To Fight Discrimination, First You Must Understand How To Measure It

“Too often ‘racism’ is the catch-all explanation, and can be simply implicitly accepted rather than explicitly examined. The evidence shows that geography, family influence, socio-economic background, culture and religion have more significant impact on life chances than the existence of racism.” (Commission on Race and Ethnic Disparities: The report, page 8).

This sentence, along with the assertion there is no institutional racism in the UK, summarises the conclusion of the 2021 Commission on Race and Ethnic Disparities (CRED). The report drew the ire of the press and media and ignited a political imbroglio that divided opinion both within the government and British public.

A few, close to the government, defended it; many stressed that other data point to the existence of widespread discrimination in the UK: Black women in the UK are four times more likely to die in pregnancy or childbirth than their white counterparts. Black students are significantly less likely to make it into higher education -even when A-Level results are on a par with white applicants, they are up to 14% less likely to receive an offer from Russel Group universities (The Conversation, 2021).

Racism is a thorny issue. It’s an issue that can inflame tensions, controversy and political storm unlike any other. But things are not made any easier by the reality that racism – and discrimination in general – is something inherently hard to measure in any objective sense.

When we try to quantify discrimination, typically we use one of two approaches: broad or narrow. The broad approach, as the names suggests, tends to look at simple differences in outcomes between different groups. It is often used to measure several socio-economic outcomes, such as wage gaps, differential unemployment rates, health, school attainments, or violence suffered from police. The narrow approach, on the other hand, makes more complicated comparisons. Rather than simply looking at difference in outcomes across different groups, as the broad approach does, the narrow approach recognizes that groups differ in more than one dimensions. Members of different groups tend to live in different locations, experience different family experience, are from different socio-economic background. All those can also affect the outcomes studied. In the narrow approach, the effect of race, sexual orientation, or other markers of discrimination is measured only after the potential impact of other attributes has been accounted for.

In general, though not always, the broad approach yields a higher measure of discrimination than the narrow approach. For example, back in 2018, the Office of National Statistics (ONS) published a report on “ethnic pay gaps”. Using big data and statistical regression techniques to find broader patterns, the ONS were able to share some interesting data points. For instance, when using the broad approach, employees from the Bangladeshi ethnic group, on average, earned 20.2% less than White British employees, while the percentage difference in median hourly pay between people of a White ethnicity and all those who belong to an ethnic minority group is largest in London at 21.7% (ONS, 2019). However, it was also found from the same report that “the ethnicity pay gap between White British employees and most other ethnic groups narrows once other characteristics such as education and occupation are taken into account”, suggesting that the pay gap tends to be smaller when measured with the narrow approach.

Returning to the debate over the CRED report, with these elements in line, we can see that the commission chose a narrow approach, as illustrated by the first quote, while the critiques adopted a broad approach. As such, the commissioners and their opponents have talked past each other rather than debated one another.

So, is one approach better than the other? The short answer is: it depends. It depends on what type of discrimination the person or organisation is looking to measure. And it depends on the circumstances.

Take recruitment, for instance. If you are looking to spot possible discrimination in a company’s hiring processes, it’s likely smarter to adopt a narrow approach; one that digs into the specifics around like-for-like candidates. It makes little sense to compare the experience of very different applicants who are unlikely to be competing for the same job. A better way to measure discrimination in this setting is to check for differences in outcomes between applicants with similar characteristics, who are competitive for the position, and who might be treated differently because of their identity. In other words, it seems best to take a narrow approach. And research has yielded some practical solutions for testing for discrimination in recruitment.

Conversely, if you are looking to see bigger-picture evidence of discrimination – you’re looking at the effect of race on quality of life or on the kinds of opportunities open to people – taking a narrow approach can be tricky. You need to be certain that other contributing or historical factors in people’s lives are not the result of discrimination at some other point or moment in time. Let’s put that another way: could someone’s living arrangements or place of residence be a consequence of past discrimination? Or could the socio-economic status of their parents be the result of unequal treatment in the past? If the answer to this is yes, then taking a narrow approach is probably not going to help you understand discrimination at a societal level. And this is precisely the kind of criticism that was levelled at the CRED report.

It’s tempting to simplify the way we think about measuring racism and other kinds of discrimination in our world. When it’s something particular or specific, take a narrow approach. When it’s more general, be broader. Unfortunately it’s not that simple. Too narrow and you fail to see the full picture. Too broad and you risk diminishing the role of personal agency in determining our outcomes – our individual choices, failures, triumphs and successes can just become a function of “society” at large.

There’s a role here for the CREDs and the ONS, for commissions and thinktanks, scholars and researchers, and anyone interested in taking the temperature of tolerance in our society – an imperative, in fact. We can’t fight discrimination without the tools and the methods for adequately measuring these things. And a good place to start is by understanding what we gain and what we lose when we use one approach over another.