Why are interviews harder than the job?

Why?

Because the vast majority of job interviews are with terrible candidates, even if the majority of candidates are excellent. This apparent paradox has a simple explanation: excellent candidates selectively apply to a few companies and get interviews/offers at almost all of them. On the other hand, terrible candidates are rejected at every step of the hiring process, and have to constantly reenter the interview pool.

Suppose 90% of candidates are excellent and 10% are terrible. If the excellent 90% only need to interview at one company, whereas the bad 10% need to interview at 20 companies, then only 0.9/(0.1*20+0.9)=31% of interviews will be with qualified candidates. To retierate: almost 70% of interviews will be with terrible candidates, even though 90% of people applying for jobs are excellent.

Because the cost of a bad hire is so consequential, the interview process is not designed to efficiently handle a minority of qualified candidates, but rather efficiently weed out a majority of horrible candidates. It is therefore a terrible process for the people actually qualified to pass it.

best explanation i’ve read so far!

Why put the blame only on candidates? Interviewers are equally bad to interviewees. I have been to both sides of the table and can guarantee that 80% of interviewers would not be fit for my job or the process of hiring.

If I remember correctly, there's an ancient article by Joel Spolsky about this.

That great candidates are not out there doing blind interviews, only those of us who are average are interviewing and going through hoops. Engineers below average are almost constantly interviewing.

Ah, the "market for lemons" argument: https://www.jstor.org/stable/1879431

> Because the cost of a bad hire is so consequential,

This is stated all the time and I feel it is true. But is there any way to make it less consequential? That was my main argument for contract-to-hire (though I know there are downsides to that approach).

Are there any other ways to make hiring less risky?

We have to acknowledge that most companies are horrible at recognizing a bad hire and then appropriately handling the situation. A lot of companies profess to "hire fast, fire fast" but very few actually do. Until that changes, the cost of a "bad hire" will continue to be disproportionately high.

Yup, back in 2005 [0]! It's far from a new idea.

[0] https://www.joelonsoftware.com/2005/01/

Trial periods. Asking employees to do less in their first months.

Maybe not great mitigations, but that’s what I could come up with

I mean this is certainly an effect, but largely outdated reasoning. Every tech company you can think of copied verbatim a Google-style interview process out of complete reflex without understanding largely why. Now you discover that your candidate, who can invert a tree just fine, and can regurgitate quicksort, is totally useless at delivering software, managing stakeholders, or understanding the business logic.

As someone interviewing for the first time in a long time I can tell you most assuredly that there are a disproportionate number of awful interviewers too.

Of the interviews I've had I would say about 3/4 have tried to catch you out with some inane gotcha that you would never see in the wild or have a very specific solution in mind without exploring or discussing. Sometimes both.

This is closely related to the claim "We only hire the top 1%". Yes, but when your competitors reject a candidate they don't just stop existing, eventually they choose you.

After interviews, you hire Alice but Bob was a close second. Bob goes to a competitor. Alice doesn’t work out. A double whammy. Why not immediately try to get Bob? I’ve only seen this a few time.

Yes... Joel Spolsky puts it a bit uncharitably here [1]:

> I’m exaggerating a lot, but the point is, when you select 1 out of 200 applicants, the other 199 don’t give up and go into plumbing (although I wish they would… plumbers are impossible to find). They apply again somewhere else, and contribute to some other employer’s self-delusions about how selective they are.

----

[1] https://www.joelonsoftware.com/2005/01/

In the US at least it seems common to review performance and put certain benefits on hold for 90 days to see if it's working out. This wouldn't mitigate the costs of the initial hiring or the opportunity cost of not selecting another candidate, though.

My more recent interviews have been fine, but for awhile I had several that were (in the words of xkcd) "communicating badly then acting smug when you're misunderstood".

> Are there any other ways to make hiring less risky?

Professional licensing.

Many other fields require professional licenses. I don't understand there's so much opposition in our field.

(Ok, I do understand.) In general, licensing has some risks:

The lemons will get excluded from the field. (Which is kind of the point.)

Or, the lemons will decide what the criteria for a professional license is; which turns it into a BS hurdle.

---

That being said, the article gets closer to the point of what a professional license is for: "An interview is like running 100m and a job is like a 10k.". If the license is more like running a 10k, then interviewers can rely on it to do a better screening than they could ever hope to do.

I don't think the parent comment meant to 'blame' the candidates; I read this as a statistical picture. Because of how the numbers work out, the market (as measured per-interview) is flooded with bad candidates. This does not disagree with the fact that companies are _also_ usually pretty bad at interviewing.

Your point is interesting but I don’t see how it answers the question.

“The majority of interviews being statistically rejections “is not obviously related to a need to make the interview process more demanding than the actual job - Testing for the role would also weed out bad candidates.

Here’s some alternatives:

- engineers testing for the idealized version of their job, rather than the realities of it - “a true engineer must know how to balance a binary tree! Nevermind that I spend my time dealing with support tickets regarding a null pointer that slipped by in a code review”.

- companies using long processes as negotiation tactics - “you went through two months of trials so you’re less likely to reject our offer now and start over”

- design by committee - “every interested party and team must give an approval in their own step”

- interview as marketing - “see? We deal with tough challenges, this is an interesting place to work at”.

The 'cost of a bad hire' is received wisdom that needs to go away. The first order effects of your team's time investment are easy to see and make good content for your engineering leadership blog when you're aiming for promotion. The second order effects are what get debated in threads like this ad infinitum.

Paradoxically, a higher bar for hiring increases these consequences for everyone. A bad hire is only consequential in the first place because hiring managers are slow to cut them loose. Managers are slow to cut loose because they are morally culpable for the consequences to the individual they hired. When a manager extends an offer, they are accepting some responsibility for a significant change in a person's life. It's very difficult to walk that back when it's a bad fit, knowing that hiring is a slow process and every other company out there is scared of making a bad choice. But at the end of the day, interviews are an approximation of the candidate/company fit in what is ultimately a matching problem. More attempts make for better matches. Companies and candidates both would be better served by being faster to hire and faster cut loose.

The issue with professional licensing, is that it's very, very specific.

In things like civil engineering, there's usually mandated context. You have to work within certain parameters, so it's not too difficult to test with real-world criteria.

With software engineering, it's all over the place. In fact, one of the most exciting things that I used to look for, in potential candidates, was people who were not bogged down with dogma, and would bring alternative viewpoints to the team.

Since anyone coming into my team would require a ton of training; regardless of their seniority, I always had a nice, long on-ramp, in which I could evaluate people.

One additional downside to professional licensing is that it can be time consuming and costly. For example, to bring a new physician on board to a practice they must be:

* Licensed (allowed to practice in that state)

* Credentialed (degrees and experience verified)

* Enrolled (able to be imbursed via insurance)

* Priviledged (authorized to perform certain tasks/roles)

This could take weeks or months per physician and there is usually an entire team (Medical Staff Office/MSO) dedicated to the work.

Nowadays it's more like the excellent 90% need to interview at 20 companies and the terrible 10% need to interview at yet more

Relatedly, if you hire Alice and she works out, but in months or a year, you hire for the same role, why not reach out to Bob again?

You invest all that time into interviewing Bob, but then if they don't get the offer, you never reach out to them again. I don't get it.

I don't think I've ever seen this done well.

Kind of like how the CFA used to be in finance.

It’s rather older than this. Any argument about “the market is swamped by bad X in circulation and good X are much rarer in the market than you would naively expect” (software developers, online dating, used cars, debased currency…) is just a version of Gresham’s Law, which is 500 years old: https://en.m.wikipedia.org/wiki/Gresham%27s_law

I can attest. I've personally been shut out of two positions because I didn't say the "magic word" that the interviewer was looking for. In one instance, I kept using laymen's terms to describe an optical engineering process, because I knew they weren't optical engineers (far from it). They wanted me to say the precise technical phrase, but wouldn't explicitly ask that; they just kept asking questions around that phrase.

In another job where I was hired, I was tortured for over 10 minutes as one interviewer tried to get me to say the magic word. Fortunately, his manager was present, and shut down the questioning at that point. I never did learn what word the subordinate was trying to get me to say, but I was fully qualified for the job, and hired.

In my experience, the hiring managers with the best track record have these in common, which mostly boil down to the hiring teams doing their homework:

(1) The expectations for the position are clearly defined (2) The hiring team members coordinate on questions and expected responses, and they are consistent in interviews. (3) The hiring team members know how to spot potential issues (e.g., excessive bad-mouthing of previous employers, etc.) (4) The hiring teams effectively leverage their networks for references. Ideally, there are not-too-distant trusted relationships between the candidate and the hiring team. In the absence of this, references are followed up on carefully (this has become an art form in modern times.)

These reduce the risk of someone slipping through the cracks. Hiring teams also get better with experience, so any mistakes should be carefully analyzed and corrective actions incorporated into the hiring process.

> Because the vast majority of job interviews are with terrible candidates, even if the majority of candidates are excellent. This apparent paradox has a simple explanation: excellent candidates selectively apply to a few companies and get interviews/offers at almost all of them. On the other hand, terrible candidates are rejected at every step of the hiring process, and have to constantly reenter the interview pool.

Do you have actual data showing this? Or is this just your intuition?

Because if it's the latter, my intuition is pretty different.

No one is an "excellent candidate" for every position, and many people who are "excellent candidates" for a given position might not even recognize that excellence in themselves. Therefore, they're not going to be only applying for positions at the very best companies; they're going to be applying for any position they think they have a chance at, assuming they think they can actually be OK with the job (eg, they might not want to apply for a job in adtech if they are personally repulsed by the ethics of surveillance capitalism).

Additionally, your scenario sounds like it paints the candidate pool for jobs in general as a bimodal distribution, with one peak of "terrible candidates" and one peak of "excellent candidates", with very little in the middle. My intuition says that it's much more likely to resemble a normal distribution.

No; what's much, much more likely is that most people are decent candidates for many jobs in their field, and excellent candidates for a few, but their chances of actually getting the opportunity to apply to those few (between the position being open and them searching at the same time, and them finding out that it's open) are slim, so they have to apply for a great many of the positions that they're only decent candidates for. That means that they'll try many times before finding something. This can lead to a lot of frustration and even desperation, creating a willingness to engage in some shady techniques to actually get a human to talk to you and recognize your value.

Then there are a few people who are, indeed, nothing but shady techniques, and they are likely to flood all channels with as many AI-generated or otherwise low-quality applications as they can manage.

So no; even if most applications any position sees are terrible applications, most interviews are likely to be with decent-but-not-excellent candidates, and most people are still going to have to interview with a few or even tens of companies before they are actually offered a position.

There can be a halfway solution that helps: A professional software engineering license that signifies an extremely basic, barebones level of skill. Companies would still need to interview candidates, but they wouldn't have to do FizzBuzz or "write a for loop" types of interviews with candidates who literally cannot write code at all. It wouldn't guarantee the person was an expert in inverting binary trees, but it would at least represent a minimal knowledge and skill bar that one would expect any software engineer to meet.

It really, really sucks, that we have to be here.

Maybe have online application pages, that can’t be submitted, unless you fizzbuzz. :(

One of my interview questions used to involve putting up a parameterized SQL statement on the whiteboard, (select userid, email from users where userid=?), give the candidate instructions on how to use a data reader, and then expect them to construct a "user" object.

I did this because I was part of a project that failed because the leads did not know how to work with a database. They thought the ORM would just magically load objects. Without understanding the basic limitations of data readers, they painted themselves into a corner.

Licensing basically needs to filter this kind of thing. IE, you don't get a license to work with a SQL database unless you can work with a data reader.

---

The irony was that we very rarely touched SQL, but the question was one of the greatest filters out there! It showed who understood basic concepts, who was just BSing their way though, and who would get stuck in high-level abstractions.

> This could take weeks or months per physician and there is usually an entire team

That seems like a better alternative to what we have now! Some companies search forever for the "perfect" candidate, or otherwise have to sift through so much SPAM resumes that they are already taking "weeks or months."

Same story, I think. Well-paid positions at sensible low drama companies are filled quickly, while companies with glaring issues may interview and make offers to dozens of candidates before finding one who accepts the offer. So as a candidate you also see a disproportionate number of bad interviews.

I think this may be an example of Simpson’s Paradox

https://en.m.wikipedia.org/wiki/Simpson's_paradox