Two Mistakes That Matter

The amount of hiring advice can sometimes be overwhelming. You can reduce a lot of the noise by focusing on the two types of errors that are possible and then working systematically to reduce them.

1. There are two types of selection errors in hiring: hiring the wrong person (false positives) and not hiring the right person (false negatives).

2. Each of these errors have multiple “input streams” that increase their likelihood. If you address these, you will make better hiring decisions.

The Two Types of Error: False Positives and False Negatives.

False Positives: By definition, this is when you voluntarily hire someone and they don’t work out. (You can define that however you want—I think a reasonable metric is anyone who isn’t performing at or above expectations within 6 months of joining. Or anyone who voluntarily leaves within 12-18 months of joining, even if they were a high performer.)

False Negatives: This is when you don’t hire someone that you should have. (I.e. this person would have performed strongly in the role and would have chosen to stay at the company for a long time.) If we wanted to stretch the definition a bit, we could also include people that never even chose to apply to your company.

This week, we’ll focus on false positives. Later, we’ll discuss how to minimize false negatives.

Possible Causes of False Positives (Non-exhaustive):

1. You didn’t really have an interview process at all. You half-sold, half-interviewed. Or you took it on the good faith of a referral that you trusted. Or you did a trial project that seemed to go well, but didn’t pair that with interviews.

2. You and your teammates were not aligned on what you were looking for. This is way more common than people are willing to admit to themselves—we see it in virtually every workshop we do. If the interview team is not fully aligned on a written document specifying the success criteria for the role, you will add hiring mistakes.

3. Individual interviewers made holistic judgments or otherwise “went with their gut.” If you have several people doing ~1hr interviews and each making holistic yes-no (or 4-3-2-1) decisions on a candidate, you are virtually guaranteed to get snowed over by strong communicators who are good at performing during the interview. It’s also possible that you did attempt to divide-and-conquer the Target but that the individual ratings weren’t really backed up by any substantial data.

4. You cared too much about certain traits, and not enough about others. It’s possible that your team executed the process well but ultimately weighted the evidence incorrectly. This could stem from leniency (often caused by desperation—”we need someone now!”). Or it could simply be that your Target was unbalanced to begin with and you did not adequately address the root causes of why someone washes out in this role.

5. You otherwise cut corners and are “data-light” in your decision. Usually companies have enough total interviews, but the issue is that usually the interviewing team did not get enough relevant, calibrated positive and negative examples from the candidate’s past. This is downstream of interviewer training, and is outside the scope of this article. From a process perspective, two other common things we see are companies that abridge the Deep Dive (career review) and fail to conduct meaningful reference calls. Both of these reduce your data, and you’re likely to fall for a candidate whose performance conveys intelligence, likability, and strong communication skills, but who may come up short on the actual job.

6. You just got unlucky. Predicting the future performance and motivation of someone will never be perfect and at the end of the day, a perfect process will only get you somewhere in the neighborhood of 80-90% accuracy on the hires you decide to make.

In some of the cases above, no interview process is used, so the root cause is a form of laziness (or “resource scarcity” if you want to be more generous). In other cases, you did implement an interview process, but did so imperfectly. The goal, of course, is to remove these types of errors so that the only “bad beats” you have in hiring are ones where you couldn’t have done anything better.

Two Kaizen Questions:

In the spirit of continuous improvement towards that lofty goal, here are two exercises that can keep you and your teammates honest:

1. When rejecting a candidate, ask: “Could we have known this relevant information earlier in the process? Did we really need this number of interviews to make this decision?” This will tighten up your data elicitation and interpretation loops, and shave off inefficiencies in your interview process. The better you get at this, the more time you have to go deeper where it matters on candidates that are oh-so-close but not quite right for the role.

2. When you have an underperformer—or someone who leaves prematurely—ask: “What was the root cause of this, and what data did we have from the interview process on this point? Did we have adequate evidence in favor of hiring this person, or did we cut corners in some important way?” You can then adjust your data elicitation and interpretation to start weighting these types of concerns more heavily than before.

You’ll know your false positives are under control when virtually 100% of your “mistakes” fall into one of 3 buckets:

1. New grad hires (the less career experience someone has, the lower the hiring accuracy that is possible). You just need to make sure your onboarding, communication, expectation-setting, coaching, and accountability are all dialed in.

2. A gamble you knew you were taking. You had flagged something as a concern but you thought the upside risk was worth it. (Maybe it really was worth it in the aggregate across all similar cases, but it didn’t work out in this individual case.)

3. A truly "out of the blue” occurrence. Someone does something that is truly de novo and it may not have been within your power to reasonably predict it.

I challenge you to not default to any of those explanations though. It’s highly likely that around 60-80% of your current false positives were preventable and that a great hiring (and interviewing) process can eliminate them, saving you pain, time, and money.

Next week: False Negatives.