←back to thread

Please stop the coding challenges

(blackentropy.bearblog.dev)
261 points CrazyEmi | 3 comments | 15 Nov 24 15:32 UTC | HN request time: 0.21s | source
Show context
fishtoaster ◴[15 Nov 24 18:07 UTC] No.42149357[source]▶
I recently ran an interview process for a relatively senior eng role at a tiny startup. Because I believe different interview methods work better for different people, I offered everyone a choice:

1. Do a takehome test, targeted to take about 4 hours but with no actual time limit. This was a non-algorithmic project that was just a stripped-down version of what I'd spent the last month on in actual work.

2. Do an onsite pairing exercise in 2 hours. This would be a version of #1, but more of "see how far we get in 2 hours."

3. Submit a code sample of pre-existing work.

Based on the ire I've seen takehome tests get, I figured we'd get a good spread between all three, but amazingly, ~90-95% of candidates chose the takehome test. That matches my preference as a candidate as well.

I don't know if this generalizes beyond this company/role, but it was an interesting datapoint - I was very surprised to find that most people preferred it!

replies(7): >>42149441 #>>42149536 #>>42149571 #>>42149636 #>>42150136 #>>42150254 #>>42151318 #
1. vunderba ◴[15 Nov 24 19:38 UTC] No.42150136[source]▶
Yeah, I saw that coming a mile away. Don't you remember uni? Students always breathed a sigh of relief when the final exam was a take home project.

Now add this to the fact that a vast majority of your applicants are going to be feeding your assignment directly into an assistive LLM.

As an interviewer, you really can't win.

- If you have a take-home test, people will either complain that it's too involved, and time consuming.

- If you do whiteboarding, people will claim that it's not representative of the actual job.

- If you do paired programming, people will claim it's unfair to those who don't test well under pressure.

replies(2): >>42151004 #>>42152862 #
2. skydhash ◴[15 Nov 24 21:02 UTC] No.42151004[source]▶
That's because these three methodologies test different things. Take-home is the best representative, but it's heavily skewed by the prize. Whiteboarding only test for algorithms knowledge and a bit of problem-solving. Paired programming relies more on communication. And the last two add time pressure that rarely exists in real world scenario.

The fact is, it all depends on the person you need and the team they will be slotted in. But it seems that the ones who know the criteria is rarely involved in these matters.

3. fishtoaster ◴[16 Nov 24 00:11 UTC] No.42152862[source]▶
That's why I offered the choice: different people do well under different circumstances. I figured that it'd allow me to hire a great dev who hates timed challenges or a great dev that hates take-home busywork. In practice, though, everyone like the take-homes, so ¯\_(ツ)_/¯

As for feeding it into an LLM, I go back and forth on that. With their current capabilities, the project is slightly too complex for that to work. You could get a lot of help from an AI, but you still have to put the pieces together yourself. And I'm fine with that! If AI tooling makes you a faster/better dev on a test, I'd expect it to do the same in real work.

Longer-term, though, I worry that LLMs still won't be able to "write the whole thing" for real work, but will be able to "write the whole thing" for takehome tests. And as such, I'll have to figure out how to handle that.