Most active commenters
  • Aeolun(4)
  • ben_w(3)

←back to thread

Please stop the coding challenges

(blackentropy.bearblog.dev)
261 points CrazyEmi | 59 comments | | HN request time: 1.325s | source | bottom
1. agentultra ◴[] No.42149111[source]
> When was the last time you had to debug an ancient codebase without documentation or help from a team?

All the time. 300-400k SLOC in C++. Legacy in the sense that there were no tests of any kind. Little-to-no documentation. Solo developer at the tiny company. Fix bugs and add features while keeping the system available to the tens of thousands of users.

A more recent example: here’s a patch for a critical feature we need. It was written over a year ago. The original author isn’t available anymore. You can write the code from scratch or try to resurrect the patch against master.

Being able to jump into a project and lead people towards some goal is definitely a skill for senior developer positions. Yes, you generally have a team you can lean on and have the ability to do research and all that. But how do you show that you can do all that in an interview?

Agree with the conclusion that a good thing to test for is for problem-solving.

The tech side depends a lot on what you’re doing. Although it gets ridiculous and organizations get lazy with this part. You don’t need to be white boarding graph algorithms for a junior web developer role. If your application is a social networking role and you’re interviewing a senior developer or architect? Definitely. They’re going to be teaching this stuff and need to understand it at a deep level.

replies(25): >>42149161 #>>42149283 #>>42149381 #>>42149490 #>>42149504 #>>42149546 #>>42149679 #>>42150007 #>>42150216 #>>42150233 #>>42150322 #>>42150383 #>>42150537 #>>42151066 #>>42151233 #>>42151287 #>>42151513 #>>42152496 #>>42153372 #>>42153374 #>>42155334 #>>42155491 #>>42157152 #>>42162104 #>>42194406 #
2. sanderjd ◴[] No.42149161[source]
Yeah I thought that was a weird thing to highlight. The "debug this!" kind of interview is pretty much my favorite, because it's by far the closest to what I do for like 90% of my time at work (when I'm not doing communication tasks...).
replies(1): >>42149246 #
3. ◴[] No.42149246[source]
4. stygiansonic ◴[] No.42149283[source]
+1

Jumping into an unknown codebase (which may be a library you depend on) and being able to quickly investigate, debug, and root cause an issue is an extremely invaluable skill in my experience

Acting as if this isn’t useful in the real world won’t help. The real world is messy, documentation is often missing or unreliable, and the person/team who wrote the original code might not be around anymore.

5. zimpenfish ◴[] No.42149381[source]
+1 for "all the time". Today I have been debugging a critical piece of the system which is written in Python (none of the rest of the system is) and largely hasn't been updated since 2020 and, you'll not be surprised, has no comments, no documentation, and a fucked up deployment system which makes me cry every time I have to think about it.

Last week I was debugging some similarly uncommented, undocumented, Go code from 2020 written by lunatics that is also a critical piece of the system.

It hurts.

replies(2): >>42149719 #>>42149760 #
6. wlesieutre ◴[] No.42149490[source]
> When was the last time you had to debug an ancient codebase without documentation or help from a team?

Heck, I have to debug the stuff that some idiot (me) wrote six months ago and it might as well have been someone else who wrote it for all I remember about how to debug it

replies(4): >>42149850 #>>42149920 #>>42149997 #>>42150304 #
7. psyclobe ◴[] No.42149504[source]
That’s funny; it’s literally my career description.
8. KronisLV ◴[] No.42149546[source]
> Solo developer at the tiny company.

> Fix bugs and add features while keeping the system available to the tens of thousands of users.

Don't tens of thousands of users warrant more developers? Or having enough of a budget to work on tests, or other things to improve the developer experience and be able to work without lots of stress? That's unfortunate.

replies(1): >>42149759 #
9. redundantly ◴[] No.42149679[source]
> > When was the last time you had to debug an ancient codebase without documentation or help from a team?

> All the time.

So you were forced to do it in a silo? No access to any type of documentation, whatsoever? Not able to refer to any C++ manuals or any other type of documentation? Not permitted to access the internet to look things up? Barred from using online forums to seek help?

replies(1): >>42149891 #
10. morkalork ◴[] No.42149719[source]
+1 here too. The situation always comes up after some kind of fucked up acquisition. Nothing like inheriting an entire code base built on someone else's stack when all the original developers decided to quit or were laid off. Ancient versions of Python and Java, deployment done by pull and pray. Multiple versions of the same service in different places because they were half way through some modernization when it all came to an end. Fun stuff. Getting your bearings fast is a real skill.
11. ben_w ◴[] No.42149759[source]
I think I managed about 10k users with my mac shareware games in 2009/10, and that didn't get me ramen profitability let alone modern pay scales.

So not necessarily any budget for more than they did.

replies(1): >>42153387 #
12. thatguysaguy ◴[] No.42149760[source]
Yeah from the title I thought this was going to be about leetcode problems, but this is truly something that comes up regularly.

Where are these dev jobs where _don't_ have to figure out some mysterious issue in a barely maintained GitHub repo semi-regularly?

replies(3): >>42149878 #>>42150281 #>>42152731 #
13. 6510 ◴[] No.42149850[source]
haha, then find code that loves performance and hates the reader.
14. evoke4908 ◴[] No.42149878{3}[source]
Yeah, I feel like jobs that don't require you to reverse engineer a bunch of stuff are the exception, not the inverse.

Hall, I do greenfield embedded programming. Most of the code I touch is completely new; the entire codebase has been replaced and rewritten by me over the last three years. Not one other developer has contributed any significant amount of code. Even then, in this scenario, I'm still reverse engineering some arcane undocumented code at least once a week. Nobody ever documents their libraries properly so I have to rip everything apart to see how it works. Or I need to adapt some vendor example code or a GitHub repo into a slightly different form. Even just existing in the ESP-IDF environment I have to dig into the core code to figure out what the hell it's doing and where these error messages come from.

If you can't read someone else's Gordian knot of undocumented code, I'd argue you are probably not very good at writing code. Reading bad code is a vital part of learning to write good code, and is generally just a fundamental core skill that every programmer needs to have.

replies(1): >>42150020 #
15. theamk ◴[] No.42149891[source]
You've read the post, right?

The OP says: "A “4-hour” assignment". This is take-home, so candidates are free to use any C++ manuals, documentation, AI, online forums, whatever...

replies(1): >>42151158 #
16. dylan604 ◴[] No.42149920[source]
So many times I'm reviewing code and really wish unpleasant words with the developer that created this horrible code. Problem is, people think I'm weird when I talk to myself like that.

Edit: apparently this wasn't written clearly enough that I was the original dev that I'm having words with???

replies(1): >>42151313 #
17. startupsfail ◴[] No.42149997[source]
When is the last time you’ve been trying to get help from another team, only to realize that the engineer you are talking to can’t write code, even with the AI help?
18. dakiol ◴[] No.42150007[source]
Don't understand one thing: I can read any code of any legacy code base, sure. What I cannot do is to asses if the code is supposed to do its job (e.g., the business logic behind it). Is the code I'm reading supposed to calculate the pro-rated salary according to the "law"? Just by reading the code, you cannot know that. You need help, either from other developers who perhaps know the codebase of from product experts that know the business logic. Or documentation, sure, but this is usually a luxury one doesn't have.

So, definitely, I always need help debugging any kind of code (unless it's rather code that doesn't deal with product features).

replies(1): >>42150317 #
19. tayo42 ◴[] No.42150020{4}[source]
How are you defining bad code?

Imo unreadable and undecipherabe code is one subset of bad code.

replies(1): >>42151882 #
20. derefr ◴[] No.42150216[source]
Real. Engineers who don't think they have this problem, are engineers who see their dependencies and the lower layers of their stack as ossified black boxes they "can't" touch, rather than something they can reach into and fix (or even add features to!) when necessary.

IMHO the willingness to "dig your way down" to solve a problem at the correct layer (rather than working around a bug or missing feature in a lower layer by adding post-hoc ameliorations in "your" code) is one of the major "soft skills" that distinguishes more senior engineers. This is one of those things that senior engineers see as "the natural thing to do", without thinking about it; and fixes that take this approach work subtly to ensure that the overall engineered system is robust to both changes up and down the stack, and to unanticipated future use-cases.

And contrariwise, fixes tending not to take this approach even when it'd be a very good idea, is one of the central ways that a codebase starts to "rot" when all the senior engineers quit or are let go/replaced with more-junior talent. When, for example, every input-parsing edge-case is "fixed" by adding one more naive validation regex in front of the "opaque, scary" parser — rather than updating the parser grammar itself to enforce those validation rules — your codebase is actively evolving into a Ball of Mud.

Of course, the tradeoff for solving problems at the "correct" layer, is that you/your company often ends up having to maintain long-lived, trivial, private hotfix-branch forks of various ecosystem software you use in your stack. More often than not, the upstreams of the software you use don't see your problem as a problem, and so don't want to take your fix. So you've just got to keep it to yourself.

(Funny enough, you could probably trivially measure an engineer's seniority through a tool that auths against their github profile and calculates the number of such long-lived trivial private hotfix branches they've ever created or maintained. Much simpler than a coding challenge!)

replies(1): >>42153519 #
21. Lammy ◴[] No.42150233[source]
> All the time.

Not just all the time — right now in the other browser tabs I'm ignoring in favor of this one lol

22. VyseofArcadia ◴[] No.42150281{3}[source]
> Where are these dev jobs where _don't_ have to figure out some mysterious issue in a barely maintained GitHub repo semi-regularly?

Oh, my job is one of those.

Our code is in Perforce.

replies(1): >>42153537 #
23. Unbeliever69 ◴[] No.42150304[source]
Me but a day ago.
24. agentultra ◴[] No.42150317[source]
> What I cannot do is to asses if the code is supposed to do its job

You can, to a degree. If you can get the system under test you can make an assertion about how you think it works and see if it holds. If you know what the "law" is you can test whether the system calculates it according to that specification. You will learn something from making those kinds of assertions.

Working Effectively with Legacy Code by Michael Feathers goes into detail about exactly this process: getting untested, undocumented code people rely on into a state that it can be reliably and safely maintained and extended.

Depending on the situation I often recommend going further and use model checking. A language and toolbox like TLA+ or Alloy is really useful to get from a high-level, "what should the system do?" specification down to, "what does the system actually do?" The results are some times surprising.

You're right that at some level you do need to work with someone who does understand what the system should do.

But you can figure out what the system actually does. And that is de facto what the business actually does... as opposed to what they think it does, logic errors and all.

A good senior programmer, in my opinion, thinks above the code like this.

Update: interviewing for this kind of stuff is hard. Take-home problems can be useful for this kind of thing in the right context. What would you do differently?

replies(1): >>42153533 #
25. bluedino ◴[] No.42150322[source]
> Instead, they put developers in situations they’d never face in the workplace, where collaboration and support are standard.

I thought the page was satire at first.

replies(1): >>42150598 #
26. sam0x17 ◴[] No.42150383[source]
Yeah I think OP's overall argument is right but this particular example isn't needed and has plenty of counter-examples. It's more like when was the last time you had to solve [insert leetcode problem here] without any resources, under time constraints, with someone judging you and staring over your shoulder at all times, at work. I find under these conditions I have maybe 40% of my usual brainpower..
27. wiktor-k ◴[] No.42150537[source]
> > When was the last time you had to debug an ancient codebase without documentation or help from a team?

> All the time.

I guess the minor difference was that it was part of your paid job instead of a huge time sink to get hired.

This is the biggest issue I have with all these tasks is that they take precious time that could be utilized better. In my case: working on real open-source software instead of throwaway made-up problems. (Source: I had to spend 3 days to write a transaction engine toy once and then I didn't get the job for another reason.)

28. olddustytrail ◴[] No.42150598[source]
Collaboration and support being standard seem very much like situations you'd never face in the workplace. No satire there.
29. TZubiri ◴[] No.42151066[source]
"> When was the last time you had to debug an ancient codebase without documentation or help from a team?"

I mean, even if you have a team, you are hired to do specifically this kind of work. I'm confused what OP thinks a software job is if not EXACTLY this.

30. TZubiri ◴[] No.42151158{3}[source]
4 hours!

Yes, I'll do it, that'll be $(4 hours x hourly rate - 1 cent), please

31. synergy20 ◴[] No.42151233[source]
you probably missed the whole point,the coding test might just be an effective way to filter out old engineers
32. emmelaich ◴[] No.42151287[source]
To reinforce, an task I had when interviewing was to solve an issue with a webapp written in a language I had never used and a webserver I had never used.

It wasn't that hard if you know the fundamentals (http, ssl). It was sorta fun and I got the job.

33. kstrauser ◴[] No.42151313{3}[source]
"What complete knucklehead thought that... git blame Oh, oops."
34. raydev ◴[] No.42151513[source]
> All the time

If you didn't solve your problem in 45 minutes with zero assistance from tools or Google or colleagues, were you immediately fired?

35. dahousecat ◴[] No.42151882{5}[source]
I would argue, that in that vast majority of cases, the readability of code is the single most important metric.
replies(1): >>42154515 #
36. _proofs ◴[] No.42152496[source]
another +1 from me -- my job, with some exceptions, is literally "plop this guy into an unfamiliar project and implement some user need be it a bugfix, feature, or (silently) a refactor.

agnostic programming flueny/software practices, including being comfortable debugging unfamiliar things or navigating uncharted waters without many domain experts to guide me, is the number one skill required in my role.

bonus points if i end up becoming an active maintainer to the project, or someone who can help other devs w it, the more times i dip into it.

37. icedchai ◴[] No.42152731{3}[source]
A repo would be nice. I was recently given a 500 meg .zip of "backend stuff" and asked to figure out what's going on. No repository, no history, the read me is useless, the APIs look like they were built by rabid animals...
replies(1): >>42152873 #
38. dudleypippin ◴[] No.42152873{4}[source]
My favorite description of this sort of setup is "coded by vandals".
39. nitwit005 ◴[] No.42153372[source]
I've twice gotten a job because all, or all but one of the previous team members left. Not a lot of help to be had.

Surprisingly, both jobs were fine too.

40. Aeolun ◴[] No.42153374[source]
When was the last time you needed to do it in 4 hours? :)
replies(1): >>42153880 #
41. Aeolun ◴[] No.42153387{3}[source]
Single dev codebases have a really hard time growing to 400kloc though
replies(2): >>42155027 #>>42155432 #
42. polishdude20 ◴[] No.42153519[source]
That's I'd the company you're at will let you take the time to "do it right". Usually that's not the case. They want features yesterday. You try and fix something properly by going down to the correct layer? "You shouldn't be doing that."
replies(1): >>42158261 #
43. polishdude20 ◴[] No.42153533{3}[source]
There's also the likely chance your business doesn't have good documentation on how features should work, there are no tests and the thing you're testing is fairly custom tailored to your business that you need to have someone with knowledge of the product and history of it to tell you "oh yes that's supposed to happen that way".

Sure if there is a law written down or past written regulations your business follows, that's easy. You've got docs.

44. anankaie ◴[] No.42153537{4}[source]
This is the most Software Engineer answer. Emphasizing only the distinctions that make no difference :D
replies(1): >>42160002 #
45. beart ◴[] No.42153880[source]
More recently than never, unfortunately.
46. amluto ◴[] No.42154515{6}[source]
If I encounter third party code that is terminally broken, I don’t really care how readable it is, except insofar as I can probably tell it’s terminally broken faster if it’s more readable.
47. rudasn ◴[] No.42155027{4}[source]
Hmm. I've found that it's much easier and faster to write a lot of code than to take the time and be thoughtful of what you end up shipping.

> I didn't have time to write a short letter, so I wrote a long one instead. Mark Twain [0]

[0] https://www.goodreads.com/quotes/21422-i-didn-t-have-time-to...

replies(1): >>42155643 #
48. SergeAx ◴[] No.42155334[source]
Single dev supporting mission critical 400k LOC C++ project is a direct path to a spectacular failure for the company.
replies(1): >>42175574 #
49. ben_w ◴[] No.42155432{4}[source]
The point is that one cannot be confident that 10k users maps to "enough money to engineer this properly".
replies(1): >>42155637 #
50. fpierfed ◴[] No.42155491[source]
+1 as well: 2M LOC codebase with little to no tests and a lot of lost knowledge due to disbanded teams, people moving on etc. In my experience very common state of affairs honestly.
51. Aeolun ◴[] No.42155637{5}[source]
Oh, certainly, the 10k users can’t. That’s why I’m not talking about that. I’m saying that instead of looking at the number of users, just the fact they have a 400k lines codebase suggests that they must have enough money to hire a team that builds that (or had, at some point in the past).
replies(1): >>42156738 #
52. Aeolun ◴[] No.42155643{5}[source]
That’s true to an extend, but you need to work to make that much code.
replies(1): >>42155901 #
53. ben_w ◴[] No.42155901{6}[source]
Not necessarily — I've had the misfortune of working with someone who repeatedly, rather than subclass, duplicated the entire file.

Including the comments I'd added saying "TODO: de-duplicate this method".

That only got the project up to about 120 kloc before I'd had enough of that nonsense and left.

My understanding is that line count was mostly down to that one dev, as he didn't only ignore what I had to say when I was there, he also ignored the occasional contractor.

54. JohnFen ◴[] No.42156738{6}[source]
Maybe so, but not necessarily. I have a couple of 300+kloc projects that I wrote all by my lonesome. It's far from an impossibility. On the other hand, there's a reason I only have a couple of those.
55. gregjor ◴[] No.42157152[source]
+1. Jumping into an unfamiliar code base with no docs and no access to the original developers offers a lucrative bottomless pit of work for a skilled freelancer. I made this my specialty a decade ago.
56. derefr ◴[] No.42158261{3}[source]
Well, that's what this thread is fundamentally about / the point that the GGP poster was making.

Coding challenges should measure your ability to quickly and efficiently dive into a big unknown codebase to fix something — because that's the prerequisite skill that develops with engineering seniority, that makes the use of this approach practical.

When you're a fluent speaker of the language of "arbitrary huge foreign codebases", it's usually actually much faster to fix the problem on the "correct" layer. Outside-the-encapsulation-boundary "compensating" solutions have an inherent overhead to their design and implementation, that just solving the problem at the correct layer doesn't.

To reuse the same example from earlier: the parser library is already a parser, while your own business-layer code is not already a parser. (If it was, why would it need to be invoking a parsing library?)

The parsing library is the "correct place" to fix parsing edge-cases, not only because that keeps all the parsing logic in one place, but because, as a parser, it already has all the relevant "tools" available to rapidly express the concept that your bugfix represents.

Whereas, when you attempt to fix the same bug outside the parser, you don't have all those tools. You might have some analogous primitives available to you (like regexes); but usually just the stateless ones. You almost certainly are missing any kind of "where am I in the larger state machine of what's going on, and what is the intermediate analysis state derived from the other checks that have been done so far?" information that would be available to you in e.g. a transform-callback function in a .yacc file.

In other words, a wrong-layer fix is one that requires you to derive non-exposed internal information through heuristics from external side-channel signals. You're essentially doing Signals Intelligence to the library you're wrapping the behavior of — and SIGINT is expensive! And not something a pure SWEng is likely to be very good at!

Approaching things on the wrong layer, inevitably leads to one of three outcomes — either:

1. you end up not really fixing the bug (if you run an overly-weak pre-validation regex that can't hit every case);

2. or you introduce new bugs by making some valid inputs invalid (if you run an overly-strong pre-validation regex that hits cases it shouldn't);

3. or you Greenspun half the implementation of a parser (usually a much-less-efficient recursive-descent parser, which now gives you perf problems) so that you can "do SIGINT" — i.e. copy what the state-transitions the parser would be doing, to derive "just enough mirror state" necessary so that your validation check can be invoked only on the parser-states + lexemes it should actually apply for.

---

Here is, then, a flowchart of what will end up happening in practice, depending on the team dynamics in play:

1. In a "blind leading the blind" situation with only junior programmers, you might see options 1 or 2, where nobody realizes the bug isn't actually fixed, and the system is actually getting less robust with every change.

2. When a junior programmer is working for senior programmers, and the senior programmers aren't doing a very good job of guiding the junior, but are pointing out in code review that the junior hasn't truly solved the problem, then you almost always eventually get option 3 (after much iteration and wasted time.)

3. When a junior programmer is working for senior programmers, and the senior programmers do guide the junior, then what happens likely depends on time pressure and concurrent workload.

3.a. If there's enough capacity, then they'll likely nudge the junior into trying to fix the problem on the "correct" layer — even though the junior doesn't have this skill yet, and so will take longer to do this than they'd take even for option 3 above. The senior programmers want to give the junior the opportunity to learn this skill!

3.b. If there isn't enough capacity among the junior programmers, but one of the senior programmers has strong principles about code quality and at least a few off-hours to dedicate, then the senior programmer will likely steal the bug from the junior and fix the problem themselves, quickly, on the correct layer. (This might only happen after the junior has already flailed around for quite a while; the fact that this looks like "wasted time" to PMs is something senior programmers are very conscious of, and so will often actively defend the competence of the juniors they steal these bugs from — usually explaining that they couldn't have been expected to solve this problem, and it should have been assigned to someone more senior in the first place.)

3.c. If there isn't enough capacity — and that includes overworked senior programmers or senior programmers all pulling double-duty as PMs or such, leaving them no mindspace for implementation-level problems — then they'll probably just tell the junior to solve things through option 3 (because they know it'll be faster) and make a mental note of this as an intentional temporary introduction of technical debt to be repaid later (by moving the solution to the correct place) when they have more time.

4. And if a senior programmer gets directly assigned the problem... then they'll just go directly to fixing the problem at the correct layer.

(These are all from my personal experiences over my career as a junior engineer, senior engineer, and now CTO.)

57. deterministic ◴[] No.42162104[source]
Yep all the time. 30+ years old C/C++ large scale software used by companies you have heard about around the world.
58. int_19h ◴[] No.42175574[source]
"Move fast, break people."
59. fennecfoxy ◴[] No.42194406[source]
Even then, the senior developer/architect "social networking app" graph algorithms stuff has its limits as well. Mathematicians come up with great algorithms, not software developers; we just use 'em. And this makes sense because those are two entirely different roles.

Imo it's more important that you can break down an existing best-in-class algorithm for x task and have your prospective dev answer questions about it/how they would use it, etc. Expecting every senior dev to be a best-in-class top mathematician as well as having a handle on the ever-changing language/library/ops/etc stacks is just crazy.

I'd never expect a senior dev on my teams to just know something like that, but I would expect that given a task they can do their research, find good candidates and understand the pros/cons of each approach. Because our power is in our ability to learn & use; the stuff that we already know is a bonus.