Unconscious bias is everywhere. That’s because it’s an integral part of being a human being. It’s the reason why we feel comfortable with people we assume are similar to us and why organisations often hire or promote the same sort of people, leading to a mini-me culture. Even the most freethinking, open-minded people are at its mercy. And it’s the single biggest block to UK organisations achieving increased diversity, simply because they have made it a focus for the wrong reasons.

It has traditionally been thought that patterns of discriminatory behaviour in organisations are always conscious. We assume that people who know better do the right thing, and those who don’t do the right thing act because of choices made from bias. In reality, the situation is far more complex.

In 2017, psychologists Peter Jones and Tinu Cornish published research suggesting that around a quarter of employees unintentionally tend to recruit people who are similar to themselves or others in their workplace. They interviewed candidates who felt that they had been victims of stereotyping when being interviewed for a new position and found that this was most acutely felt by part-time workers, people with disabilities and those with a strong religious faith.

A 2012 study, this time by Yale University, asked more than 100 scientists to review identical CVs for a laboratory manager position that had been randomly assigned male or female names. The researchers found that the ‘male’ candidates were judged to be more competent and deserving of a higher salary than the ‘female’ and that the scientists were more likely to hire a male candidate. What’s more, the women who took part in the study were just as likely as the men to prefer a male candidate.

So, while we might be working hard to remove conscious bias through legislation and by proactively educating people to help them understand and avoid discrimination, unconscious bias is still present behind the scenes, subversively undermining these efforts. In many ways, unconscious bias is the modern challenge because it’s harder to identify and because even the most diversity-aware individuals and organisations are prey to it.

This goes a long way towards explaining why, when we’ve made significant progress in, for example, reducing conscious discrimination, the unemployment gap between ethnic minorities and the wider population has been somewhere in the region of 15 percent for the past 30 years.

But how does this unconscious bias manifest itself in practice? Well, let’s start with headhunters. As diversity gatekeepers who arrived late to the party, they of all people ought to have a genuine commitment to helping their clients improve diversity. Some of the better firms (although not that many, truth be told) provide worthwhile diversity training and guidelines so that their consultants are positioned to avoid wilful discrimination. But these recruiters still fall prey to one of the simplest forms of unconscious bias – a negative assessment of non-standard CVs and career paths.

It’s all too easy to discount a CV that isn’t set out in a traditional way, or one that describes a career path that doesn’t follow a ‘normal’ path up through the corporate hierarchy. Unfortunately, the standard CV or career path is set at the norm for the majority: good school, good university, good corporate experience (meaning someone who can be quickly placed with clients). CVs from minority candidates often don’t look like that, because minorities don’t tend to have trodden such a conventional path. So the recruiter typically excludes those applications, not deliberately but in an unconscious way, feeling that they are doing the best by their client. They’re used to assessing candidates against a ‘normal’ framework of what looks good, and they come up with the usual suspects, who are easier and faster to place.

If minority candidates do happen to make it through the process and are presented to the client, the same unconscious bias will be reapplied by the next group of people reviewing the CVs. Unconscious bias also plays a part in the way the recruitment consultant briefs different candidates prior to meeting the clients, if they get through the sift. For example, a very good-looking candidate is likely to receive a different briefing from a less attractive candidate – maybe not better, but a different one. The same applies to minority candidates. That’s unconscious bias, and it has a proven impact on the process, whether run internally or externally.

Replace the word ‘minority candidate’ with ‘female candidate’ and you quickly see why the numbers of men tend to outnumber women at each level up the management chain – even more so when you bear in mind the findings of the Yale researchers. With the majority of leadership positions still held by men, the mini-me syndrome and the belief that ‘things have worked well up until now, so why change?’ are huge barriers to increasing the number of women in senior roles.

While most organisations will look to take the brand risk, if they admit it exists, unconscious bias will continue to thwart the drive towards diverse representation for all but the most progressive. Of course, the smartest companies are those that can address both forms of bias to realise the power of collective difference: the competitive advantage of achieving diversity.

Diversity and Inclusion for Leaders is a book newly-released, by Raj Tulsiani