Most active commenters
  • alkonaut(7)
  • bhhaskin(6)
  • John_Cena(5)

←back to thread

Please stop the coding challenges

(blackentropy.bearblog.dev)
261 points CrazyEmi | 36 comments | | HN request time: 2.262s | source | bottom
Show context
fishtoaster ◴[] No.42149357[source]
I recently ran an interview process for a relatively senior eng role at a tiny startup. Because I believe different interview methods work better for different people, I offered everyone a choice:

1. Do a takehome test, targeted to take about 4 hours but with no actual time limit. This was a non-algorithmic project that was just a stripped-down version of what I'd spent the last month on in actual work.

2. Do an onsite pairing exercise in 2 hours. This would be a version of #1, but more of "see how far we get in 2 hours."

3. Submit a code sample of pre-existing work.

Based on the ire I've seen takehome tests get, I figured we'd get a good spread between all three, but amazingly, ~90-95% of candidates chose the takehome test. That matches my preference as a candidate as well.

I don't know if this generalizes beyond this company/role, but it was an interesting datapoint - I was very surprised to find that most people preferred it!

replies(7): >>42149441 #>>42149536 #>>42149571 #>>42149636 #>>42150136 #>>42150254 #>>42151318 #
1. bhhaskin ◴[] No.42149441[source]
Why would you even do any of that for a senior role? I wouldn't waste my time with it, and it shows you don't know how to interview/evaluate for a senior position.
replies(6): >>42149512 #>>42149515 #>>42149540 #>>42149560 #>>42150471 #>>42151698 #
2. 0xffff2 ◴[] No.42149512[source]
I've come around to the idea that all people who will be expected to write code as part of their job need to be evaluated for their ability to write code as part of the interview process. I've seen too many people write convincing and impressive resumes without a single ounce of technical ability to their name.
replies(1): >>42149685 #
3. dahart ◴[] No.42149515[source]
Why wouldn’t you? How should one evaluate a senior position, in your opinion? Are you suggesting a senior dev shouldn’t need to demonstrate any coding skill? (If not, why not?) Or are you suggesting that there are other faster ways to show coding skills?
replies(1): >>42149722 #
4. stickfigure ◴[] No.42149540[source]
What would you waste your time with?

I exclusively give pairing interviews, usually just 1 hr. The variance between good and bad candidates is amazing, and you wouldn't guess at all from resumes or casual conversations. Most candidates have given me very positive feedback. Why wouldn't you want to be interviewed like this?

replies(1): >>42149773 #
5. alkonaut ◴[] No.42149560[source]
Because for a senior role you need to hire a person with senior role skills, I suppose. Just N years of experience at X company doesn't provide that. I'd rather look at someone's github repo than talk to their reference tbh.
replies(1): >>42149651 #
6. John_Cena ◴[] No.42149651[source]
I've never worked at a company with a public repo. I never understood this. I have a life outside of work
replies(1): >>42149692 #
7. bhhaskin ◴[] No.42149685[source]
You're hiring for a senior role, not a junior or mid.

Your candidates aren't fresh out of school or just starting their careers. They already have plenty of work experience. They already know how to code and have been doing it for years. A conversation will tell you much more about their approaches to problems solving, team work, and mentoring than a coding challenge will. Looking at their GitHub, their work experience and references will tell you the rest.

It's like asking a doctor to go over basic human anatomy.

replies(3): >>42149889 #>>42151583 #>>42151647 #
8. alkonaut ◴[] No.42149692{3}[source]
Neither did I. But I'd rather just "smuggle" out some past work and "rewrite it" to make it inconspicuous, if I didn't have my own personal work to show. The company I work for isn't well known so saying "I worked there in a senior role for 10 years" isn't going to get me the next job. If I didn't have anything to show for that I either made for fun or could display from a past job, I'd not apply for another until I had tbh.

People say "No I don't have any past hobby work, never contributed to OSS, and all my professional work is verboten to show" that's not unusual, but it's also not an encouraging sign to me when interviewing.

replies(2): >>42149804 #>>42149966 #
9. bhhaskin ◴[] No.42149722[source]
I responded to a different comment but it's also relevant here.

You're hiring for a senior role, not a junior or mid.

Your candidates aren't fresh out of school or just starting their careers. They already have plenty of work experience. They already know how to code and have been doing it for years. A conversation will tell you much more about their approaches to problems solving, team work, and mentoring than a coding challenge will. Looking at their GitHub, their work experience and references will tell you the rest.

It's like asking a doctor to go over basic human anatomy.

replies(2): >>42149821 #>>42150736 #
10. bhhaskin ◴[] No.42149773[source]
I'm a senior developer. Of course I know how to code. Do you know how many code challenges I've done over the course of my career? 99% of them where terrible at evaluating my ability to code. They are a waste of time at the senior level. You are much better off with a conversation, looking at past work and experience and references than you are doing code challenges.
replies(2): >>42149905 #>>42150972 #
11. bhhaskin ◴[] No.42149804{4}[source]
Wouldn't that be a red flag if they don't have any public code at all as a senior developer? They aren't fresh out of school or just starting their careers. They should have something.
replies(3): >>42149929 #>>42149936 #>>42150087 #
12. dahart ◴[] No.42149821{3}[source]
You’re underestimating the variance in senior devs, and you’re failing to look at this from the interviewer’s perspective. Sometimes doctors should be quizzed on anatomy, btw.

They need to be able to compare candidates, and senior devs will be expected to code. Their goal is to pick the best candidate. Asking them to demonstrate their coding skill isn’t just fair game, it’s what half the interviewees actually want. A lot of people prefer being evaluated on code than on soft skills (just read the threads here for tons of evidence.)

It’s useful to know whether a candidate is going to be on the principal engineer path or management path. It’s useful to know whether they are actually good at coding, and how good exactly. And it’s also useful to know if they are willing to do what needs to be done once hired. Someone interviewing for a senior coding position who refuses to code during the interview is about as big of a red flag as you can have, and I’ve been part of hiring teams that will politely excuse someone for that, and I agree with the reasoning.

This might not make sense until you’re on the hiring side of things, but having a resume does not entitle one to being hired for a higher title and more money.

13. 0xffff2 ◴[] No.42149889{3}[source]
>They already know how to code and have been doing it for years.

I'm telling you from my own experience that this is observably not true. Twice now I've been burned by internal hires who objectively had years of experience (easy to verify when they have been working within my division) and yet couldn't code their way out of a paper bag.

And tangentially, with some of the health care experiences I've had I very well might try to administer a basic anatomy quiz to potential doctors if I thought I could get away with it.

14. stickfigure ◴[] No.42149905{3}[source]
After interviewing ~50 people using this technique - I'm sorry but there's no "of course". I cannot tell by looking at your past work experience or talking to you whether you know how to code.

I know this, because I used to look at resumes and have long conversations before I ran my pairing interview. And I found the results almost uncorrelated. Now I save the fun conversations for people who make it through the screening. But by then I already know if I want to hire them.

My pairing interview (it's a pairing session, not a code challenge) covers aspects of engineering that I specifically care about. It's not terribly hard. But at the end I'll have a pretty good idea if I want to turn this person loose in my codebase.

replies(1): >>42151470 #
15. John_Cena ◴[] No.42149929{5}[source]
I'm a senior developer and my github is half guitar tabs. I'm not interested in peacocking. Maybe it's because Hackaday refused to put my name on the article with my senior design project years ago and I just don't want to play the game.
replies(2): >>42149977 #>>42151037 #
16. alkonaut ◴[] No.42149936{5}[source]
Yes exactly. The code IS the cv of a developer.
17. John_Cena ◴[] No.42149966{4}[source]
The elitism is rampant. As an embedded sector guy everyone thinks every other RTOS isn't a real RTOS and the knowledge and wisdom don't translate.

Every time I find some open source I would contribute to, what I wanted to implement is already there or is already in the works. I'm not going to change shit just to change it. I'll buy them a coffee tip and leave.

replies(1): >>42150814 #
18. bhhaskin ◴[] No.42149977{6}[source]
And that might be hurting you in the long run. ¯ \ _ ( ツ ) _ / ¯
19. tayo42 ◴[] No.42150087{5}[source]
All of the meaningful code I write is private for the company I work for. This is how most code is stored.
replies(1): >>42150620 #
20. 12_throw_away ◴[] No.42150471[source]
> Why would you even do any of that for a senior role?

I used to think like that ... then we hired a batch of 3 "senior" devs who could not do simple, everyday coding tasks, even in their ostensible languages of preference. All had come with exceptional resumés and personal recommendations.

So now, no one at any level one gets hired without demonstrating that they can at least write a fizzbuzz and commit it to a repository.

Now, I doubt that senior engineers in other fields have to do this sort of thing. But, most other engineering fields have licensing/accreditation with accompanying post-graduate academic curricula. We don't (but probably should).

21. alkonaut ◴[] No.42150620{6}[source]
Mine too. Or 99.99% of it at least. But if I were to apply for a job, I'd focus more on having at least a little repo of something to show, than on polishing my CV or doing leetcode excercises. That 0.01% code I wrote which is my github is my CV. That 99.99% I wrote for my last employer won't be seen by my next employer.
22. theamk ◴[] No.42150736{3}[source]
You'd think so, but during the interviews I did, there are plenty of candidates to senior positions who just cannot code.

They talk the great talk about problem solving and team work and mentoring, but once it's time walk the walk, they just can't seem to write any code - and we are not talking advanced algorithms, we are talking Fizz-Buzz level.

Maybe it's different for doctors, but as long as such people exist and apply to jobs, we need to ask programming questions.

This also means that if _you_ are applying to a job in a larger team for a senior position and they did not ask you for any coding questions, they either:

- Really good for firing people fast for low performance

- Have people at "senior" position which just give advice and nasty code reviews, but don't actually write any code.

Either way, it's a red flag for the company.

23. alkonaut ◴[] No.42150814{5}[source]
If I'm interviewing for people to work in some specific tech stack or business, it's just as good to see completely different things they can show. Could be a php site for their book club or whatever. But the key is to have some code to discuss that is their code. Because I want to see them describe and reason about code they know, and it works 10x better if it's code they know, rather than some code I present and say "here, let's reason about this code".
replies(1): >>42151272 #
24. sophacles ◴[] No.42150972{3}[source]
Based on my interviewing, there's a ~25% likelihood that you're flat out lying, and a ~75% likelihood that you're overstating your skills.

The number of people with impressive looking resumes, and good command of the jargon, but who still can't successfully code hello world is shocking.

I don't believe you (general you, not just the parent) can even code until you demonstrate that to me, or until the industry comes up with a reputable way to tell me that you can code. Until then I'll be making sure you can at least write code that compiles and finishes a brain-dead simple task.

25. alkonaut ◴[] No.42151037{6}[source]
That's a great signal (that you have a hobby playing guitar). If the other half is also interesting, it sounds like a great portfolio.
replies(1): >>42153390 #
26. skydhash ◴[] No.42151272{6}[source]
Why not discuss a problem you have? You're obviously hiring for someone to solve your problem. Looking to check if they can code (in a specific language) seems like the XY problem.
replies(1): >>42151585 #
27. kstrauser ◴[] No.42151470{4}[source]
I second all of that heartily. Of course the senior eng with 10 years hands-on experience at a household name tech company knows how to code... except that they don't. At all, apparently. That seems so ludicrously unlikely unless you've actually interviewed a lot of people, then it's just the sad fact.

To others reading along, I'm not talking about someone not being able to write a multitasking OS on their own. I mean, I'm not sure how these people possibly graduated college not knowing how to write a trivial program in the language of their choosing. Turns out you can get surprisingly far into a software engineering degree without every writing code. I wouldn't have believed it. Evidence proves it though.

28. WillDaSilva ◴[] No.42151583{3}[source]
Have you ever tried to hire developers? I have, many times. It's shocking how incapable most people with a history of working as senior developers for 10+ years are at basic programming questions. I'm talking about stuff at the level of Fizz Buzz, or barely more difficult, being solved in a language of their choice with ample time.

Through all of the technical interviews I've run on behalf of multiple companies, I've become convinced that our industry is full of imposters. A terrifying number of people who are employed as developers cannot code, beyond making tiny edits to existing code, or slinging StackOverflow answers and AI generated content around. They cannot think for themselves. They rely heavily on their coworkers to make any substantial progress. If anything, modern AIs have made these people more difficult to spot. I've found these imposters within the companies I've worked for too, and if it's not possible to move them to a non-technical role where they can be productive, then it's best to fire them. If that isn't possible, then keep them out of the way of any critical work, but know that they'll continue to be a drag on their team, and will decrease the moral of those on their team who have to keep covering for them.

Once I even worked on a team (as a junior) where only 3 members out of ~16 even knew the language that was exclusively used for the job. The remaining team members accomplished nothing every day. Those stand-up meetings were painful. Drawn out to 45+ minutes of standing in a circle while the unproductive team members found new ways to explain why they haven't made progress. Then I'd see them return to their desks, and not even open their IDEs for weeks at a time. They seemed to keep their jobs because their boss didn't want to fire them, and it cost him nothing personally to keep them around, offering them a sort of corporate welfare. Perhaps having such a large team even helped him politically, but it's unclear if he cared about that since he also spent almost all of his time slacking off, playing games in the break room, while the 3 productive team members carried the rest of the team.

replies(1): >>42196893 #
29. alkonaut ◴[] No.42151585{7}[source]
Because they aren't familiar with or enthusiastic about the code I give them, they could be about code they have written. It's the classic "describe one time you did something poorly/well"-question, but with code. I want to see them critique some code, or explain what's so great, or why it ended up the way it is.

Reasoning about a problem i have (even showing some code) is also a good part of an interview. But it's my side of the field. I just found it's much better to move the interview to the interviewee's side of the turf, because they are more comfortable there.

replies(1): >>42153357 #
30. unoti ◴[] No.42151647{3}[source]
> A conversation will tell you much more about their approaches to problems solving, team work, and mentoring than a coding challenge will.

Having hired many developers over multiple decades, let me assure you that you should definitely make sure they can actually code. Do definitely have those conversations, too, but trust me when I say that a technical conversation, a degree and several listed years of experience as a software developer on a resume are not enough to guarantee that they can actually code.

31. fishtoaster ◴[] No.42151698[source]
Because all evidence[0] I've found has shown that work-sample tests and repeatable, structured indicators are the best indicator of job performance. I understand that a lot of devs think they can get a feel for a candidate by just having a good ol' conversation with them, but the body of study on the subject says that that's just wrong. And so I have people do a task that's as close to the actual job as humanly possible and evaluate them on that.

https://www.researchgate.net/publication/283803351_The_Valid...

32. John_Cena ◴[] No.42153357{8}[source]
I would like to just be asked. I can't post my code on github but I can talk about my projects at length.
replies(1): >>42155809 #
33. John_Cena ◴[] No.42153390{7}[source]
I appreciate it. After thinking about this I think I just need to get over my hangups about duplicating things already done or things that aren't really important.

I just take tabs and rearrange them for personal use. Learn from my mistake and just post whatever to your github, if you fret over its usefulness or purpose you might just never grow your portfolio at all. This was a mistake. You don't have to have some gnu front-page exploratory project.

If you want to shred guitar look up Troy Grady to grok efficient mechanics that won't break your wrist (we type alot for work as well)

34. ozim ◴[] No.42155809{9}[source]
For all this thread solution is simple. Small take home assignment where you write the code to discuss during the interview.

I know people are vocal about not wanting to do take homes. But if take home is reasonable and used as a talking piece it checks all the boxes for good tool.

I think in reality I had single person that outright refused and of course bunch of people who didn’t bother to deliver - great candidates delivered it the same day, busy great candidates delivered it over the weekend.

35. StartupSage1111 ◴[] No.42196893{4}[source]
That sounds like a really challenging situation, and I can see how it would be disheartening to work in an environment like that. It makes me wonder if part of the problem lies in how companies approach technical growth and team dynamics. For example, instead of just identifying unproductive team members, do you think those people could have benefitted from structured mentorship or training programs? Or maybe clearer performance metrics that highlight where they’re struggling?

It’s also interesting to think about how managers could be supported better. If your old boss had been more engaged, perhaps they could’ve restructured the team or helped those who were falling behind find roles that suited their strengths better. Have you ever seen a company handle this type of situation well, or is this kind of dysfunction just too common?

replies(1): >>42197341 #
36. WillDaSilva ◴[] No.42197341{5}[source]
I have seen team members fall below expectations, or burnout, or otherwise become unproductive, but then recover. In every case, their manager was a key part of the solution (and sometimes a key part of the original problem). If a person is really stuck on their current work, it can be demoralizing to have it taken away and assigned to others, but it can also be a relief. One thing that I've seen help is to let devs be more self-directed. Try to nudge them towards things that are important for the business, but focus on doing that by really convincing them of the value of that work. If they don't want to do that work, try to understand why, and consider changing plans based off of that.