How To Be Better At Recruitment Than A Coin-Toss

We tend to overestimate our potential and abilities. The more complex a task is and the further away in time the consequences of our decisions are, the harder it is for us to accurately evaluate our performance.

It’s not bad will or lack of judgment, it’s a natural bias people have.

This fact is clearly visible in the hiring process, because it’s hard to evaluate someone’s knowledge, motives, potential and cultural fit based on a few calls. Whether a candidate was actually a good fit can sometimes come out only after several months.

For small companies a wrong hire can be a real hiccup

Each employee contributes to the company culture in a unique way. In a small company, not shackled by politics, the impact one person can make is more profound than in a large company, and a person who doesn’t fit can make the rest of the team miserable.

Recruitment is a lengthy process. It can take one to two months to hire a person, then another two to three months for the trial period. If it was a wrong hire, the process must be repeated to hire a replacement and hope that this time things will be different. The whole process can take six months or more.

If you happen to be an agency, as we are, and the hire is a developer, your client won’t be very happy, either, if you need to replace him.

Let’s not forget that with a bad hire, we also waste the candidate’s time. Since you know the company, it’s your responsibility to help the candidate discover whether he or she is a good match or not.

This is not a trivial decision.

At Ragnarson, every successful recruitment involves from 8 to 12 of our developers. Each developer has to be aware of the math, the rules and the mechanics of the process. There are multiple factors that are not obvious at first glance. That’s why last month we carried out a hiring workshop for our developers.

You, too, need to know your factors. Being aware of the facts and statistics and how to improve them will increase your chances of recruiting top talent.

Your personal evaluation skills

If you happen to be God, you have an accuracy of 1. This means that you’re right 100% of the time.

The very best professional interviewers have an accuracy of 0.7. Their evaluations are 70% correct. Most people fall in the 0.3 range, so they are right only 30% of the time¹. Not surprisingly, when deciding whether a candidate at hand is a match or not, they’re wrong 70% of the time².

It doesn’t matter where you land on this scale. Even if you’re better than the industry’s ‘crème de la crème’ and you have an accuracy of 0.9, the odds are still against you. And here is why.

What are your chances when aiming for the top 10% of talent?

It’s not uncommon that during any two year period, an HR specialist will have gone through hundreds of job candidates. So, let’s assume that you have a group of 100 people, and in this group, 10 are top talents and 90 are the rest.

Every company prides itself on hiring only the best, so, naturally, you aim for exactly those top 10%. However, you don’t know who these people are. So, what will be your error rate in picking the top 10%?

The harsh reality is 50%. Your error rate will be the same as a coin-toss. Keeping in mind that you have an unrealistic accuracy of evaluating a candidate at hand of 90%.

Let me explain that. If we have 10 top talents, 90 suboptimal choices and your accuracy is 90%, you’ll pick 9 top talents from the group of 10 and dismiss one of them. From the group of 90, again, you’ll rightly eliminate 81 candidates as not top 10% and wrongly pick 9. You end up with 9 right and 9 wrong choices that you think are the top 10%.

So you’re correct in 50% of the cases.

Sadly, this is not the end of the bad news. The calculations assume that your evaluations are 90% correct, which is hardly possible. If you’re in a group of the industry’s best interviewers, you’re only 70% accurate. Presented with the same group of 100 people, your error rate will top almost 80%. Have a look at the figure below.

If you have an accuracy of 30%, your error rate is close to 96%, which means you would be better off doing the opposite of what your gut tells you.

How can we improve our odds?

Two things come swiftly to mind.

We could simply improve the quality of the pool of candidates. If we had a group of 100 people in which 60 would be top performers and your accuracy were 70%, we would have an error rate of 22% instead of 80%. Not bad, but of course, that’s easier said than done.

How, then, do we improve our odds if we can’t dramatically improve the quality of the pool?

We can add filters to the recruitment process, or simply put, include more decision-makers. Have a look at the table below.

Imagine our starting scenario with a group of 100 people: 10 top performers, 90 suboptimal choices and three people evaluating them with an accuracy of 70% each. We would have an error rate of 80% on the first person (as previously mentioned). The second person would have a 61% error rate because it would only work on the pool previously filtered by the first person, and the third person would have an error rate of 41%.

We managed to get from an 80% error rate to 41%.

However, the decision-makers (the filters) don’t just sit together and come up with a democratic verdict. Each filter is applied after the other.

So, multiple interviews?

You may be thinking, well, you didn’t exactly spill the beans, here. Every recruitment process has several interviews, filters or stages: first call, homework exercise, pair programming session and a meeting with the management.

You’re right, but if those interviewers influence each other’s decisions, they all act as one filter instead of many. For example, some interviewers (like managers) might have a stronger incentive to hire because they have more work than their current team can handle.

Moreover, you don’t know who should remain as an interviewer because of excellent accuracy and who just throws spaghetti at the wall.

 Interviewers’ accuracy varies

As we saw in the table with filters, three interviewers, each with a 0.3 accuracy, have an overall error rate of 99%. To get reasonable results, we still need to strive for the highest possible evaluation accuracy of each interviewer.

Following the advice from K. Pearson and P. Drucker, ‘That which is measured, improves’, you need to start keeping individual scores of each interviewer.

At Ragnarson, we do exactly that. If a newly recruited employee leaves the company without so much as a ‘goodbye’ or we need to fire someone, we consider that person a bad hire. This will affect the accuracy rate of the team-members who approved the hire.

Engage the team

Besides looking at the personal accuracy, we encourage the people who will work with a newly hired employee to take part in the process.

They’re the ones spending the most time with a newcomer, they’re the ones understanding role requirements and the company from the inside, so they’re the ones who should make the final call.

Conclusion

Everyone wants to work with the best, and sometimes we let ourselves get carried away by guesses, hunches and wishes. In these moments, it’s good to know what the odds are. Multiple interviews working as independent filters and a simple scoring system for each interviewer could increase the effectiveness of the recruitment process.

I would love to hear about your ideas for increasing the odds of a good hire. Feel free to ping me on Twitter at @wijet.

Notes

Insights and figures in this article come from a great book on recruitment called It’s Not the How or the What But the Who: Succeed by Surrounding Yourself with the Best by Claudio Fernández-Aráoz. I recommend it to anyone dealing with hiring.

Here is the Google spreadsheet with the error rate calculations.