Most active commenters

    ←back to thread

    The Therac-25 Incident (2021)

    (thedailywtf.com)
    449 points lemper | 18 comments | 27 Aug 25 06:57 UTC | HN request time: 0.001s | source | bottom
    Show context
    benrutter ◴[27 Aug 25 08:18 UTC] No.45036836[source]▶
    > software quality doesn't appear because you have good developers. It's the end result of a process, and that process informs both your software development practices, but also your testing. Your management. Even your sales and servicing.

    If you only take one thing away from this article, it should be this one! The Therac-25 incident is a horrifying and important part of software history, it's really easy to think type-systems, unit-testing and defensive-coding can solve all software problems. They definitely can help a lot, but the real failure in the story of the Therac-25 from my understanding, is that it took far too long for incidents to be reported, investigated and fixed.

    There was a great Cautionary Tales podcast about the device recently[0], one thing mentioned was that, even aside from the catasrophic accidents, Therac-25 machines were routinely seen by users to show unexplained errors, but these issues never made it to the desk of someone who might fix it.

    [0] https://timharford.com/2025/07/cautionary-tales-captain-kirk...

    replies(13): >>45036898 #>>45037054 #>>45037090 #>>45037874 #>>45038109 #>>45038360 #>>45038467 #>>45038827 #>>45043421 #>>45044645 #>>45046867 #>>45046969 #>>45047517 #
    1. AdamN ◴[27 Aug 25 08:31 UTC] No.45036898[source]▶
    This is true but there also needs to be good developers as well. It can't just be great process and low quality developer practices. There needs to be: 1/ high quality individual processes (development being one of them), 2/ high quality delivery mechanisms, 3/ feedback loops to improve that quality, 4/ out of band mechanisms to inspect and improve the quality.
    replies(1): >>45037053 #
    2. Fr3dd1 ◴[27 Aug 25 08:56 UTC] No.45037053[source]▶
    I would argue that a good process always has a good self correction mechanism built in. This way, the work done by a "low quality" software developer (this includes almost all of us at some point in time), is always taken into account by the process.
    replies(6): >>45037082 #>>45037902 #>>45037927 #>>45038864 #>>45045154 #>>45050022 #
    3. quietbritishjim ◴[27 Aug 25 09:01 UTC] No.45037082[source]▶
    Right, but if everyone is low quality then there's no one to do that correction.

    That may seem a bit hypothetical but it can easily happen if you have a company that systematically underpays, which I'm sure many of us don't need to think hard to imagine, in which case they will systematically hire poor developers (because those are the only ones that ever applied).

    replies(3): >>45037428 #>>45037524 #>>45038431 #
    4. ZaoLahma ◴[27 Aug 25 09:51 UTC] No.45037428{3}[source]▶
    Replace the "hire poor developers" with "use LLM driven development", and you have the rough outline for a perfect Software Engineering horror movie.

    It used to be that the poor performers (dangerous hip-shootin' code commitin' cowpokes) were limited in the amount of code that they could produce per time unit, leaving enough time for others to correct course. Now the cowpokes are producing ridiculous amount of code that you just can't keep up with.

    replies(2): >>45045739 #>>45050049 #
    5. anal_reactor ◴[27 Aug 25 10:05 UTC] No.45037524{3}[source]▶
    Sad truth is that average dev is average, but it's not polite to say this out loud. This is particularly important at scale - when you are big tech at some point you hit a wall and no matter how much you pay you can't attract any more good devs, simply because all good devs are already hired. This means that corporate processes must be tailored for average dev, and exceptional devs can only exist in start-ups (or hermetically closed departments). The side effect of that is that whole job market promotes the skill of fitting into corporate environment over the skill of programming. So an a junior dev, for me it makes much more sense to learn how to promote my visibility during useless meetings, rather than learn a new technology. And that's how the bar keeps getting lower.
    replies(2): >>45038943 #>>45040412 #
    6. varjag ◴[27 Aug 25 10:54 UTC] No.45037902[source]▶
    My takeaway from observing different teams over years is the talent by a huge margin is the most important component. Throw a team of A performers together and it really doesn't matter what process you make them jump through. This is how a waterfall team got the mankind to the Moon with handwoven core memory but an agile team 10x the size can't fix the software for a family car.
    replies(1): >>45038557 #
    7. rcxdude ◴[27 Aug 25 10:58 UTC] No.45037927[source]▶
    This only works with enough good developers involved in the process. I've seen how the sausage is made, and code quality is often shockingly low in these applications, just in ways that don't set off the metrics (or they do, but they can bend the process to wave them away). Also, the process often makes it very hard to fix latent problems in the software, so it rarely gets better over time, either.
    8. pjmlp ◴[27 Aug 25 11:57 UTC] No.45038431{3}[source]▶
    The correction is done by the "lucky" souls doing the onsite, customer facing roles, for the offshoring delivery. Experience from a friend....
    9. scott_w ◴[27 Aug 25 12:09 UTC] No.45038557{3}[source]▶
    You conflated, misrepresented and simply ignored so many things in your statement that I really don’t know where to start rebutting it. I’d say at least compare SpaceX to NASA with space exploration but, even then, I doubt you have anywhere near enough knowledge of both programmes to be able to properly analyse, compare and contrast to back up your claim. Hell, do you even know if SpaceX or Tesla are even using an agile methodology for their system development? I know I don’t.

    That’s not to say talent is unimportant, however, I’d need to see some real examples of high talent, no process, teams compared to low talent, high process, teams, then some mixture of the groups to make a fair statement. Even then, how do you measure talent? I think I’m talented but I wouldn’t be surprised to learn others think I’m an imbecile who only knows Python!

    replies(1): >>45038922 #
    10. franktankbank ◴[27 Aug 25 12:42 UTC] No.45038864[source]▶
    The process that makes this work would be so onerous to create. Would you think you could do this to make a low quality machinist be able to build a high quality technical part? What would this look like? Quite a lot like machine code which doesn't really reduce the requirements does it? It actually just shifted the onerous requirement somewhere else.
    11. varjag ◴[27 Aug 25 12:47 UTC] No.45038922{4}[source]▶
    Hell, do you even know if SpaceX or Tesla are even using an agile methodology for their system development?

    What I've been saying is methodology is mostly irrelevant, not that waterfall is specifically better than agile. Talent wins over the process but I can see how this idea is controversial.

    I’d need to see some real examples of high talent, no process, teams compared to low talent, high process, teams, then some mixture of the groups to make a fair statement. Even then, how do you measure talent?

    Yep, even if I made it my life's mission to run a formal study on programmer productivity (which I clearly won't) that wouldn't save the argument from nitpicking.

    replies(1): >>45040483 #
    12. ozim ◴[27 Aug 25 12:50 UTC] No.45038943{4}[source]▶
    Huh, sad truth?

    But average construction worker is also average and average doctor as well.

    World cannot be running on „best of the best” - just wrap your head around the fact whole economy and human activities are run by average people doing average stuff.

    13. orochimaaru ◴[27 Aug 25 14:42 UTC] No.45040412{4}[source]▶
    Learning new technologies wasn’t the issue with the Therac. In fact as someone who has been coding and leading sw engineering teams for the past 28 yrs, I don’t like “new technologies”. When someone does this awesome complicated async state machine using a large set of brittle components alarm bells go off and I make it my life’s mission to make it as simple as it needs to be.

    A lot of times that is boring meetings to discuss the simplification.

    I can extend the same analogy to all the gen ai bs that’s floating around right now as well.

    14. scott_w ◴[27 Aug 25 14:47 UTC] No.45040483{5}[source]▶
    Except your only example was nonsensical on the face of it.

    > Yep, even if I made it my life's mission to run a formal study on programmer productivity (which I clearly won't) that wouldn't save the argument from nitpicking.

    I didn't ask for this, I just asked for sensible examples, either from your experience or from publicly available information.

    15. vjvjvjvjghv ◴[27 Aug 25 20:56 UTC] No.45045154[source]▶
    “ This way, the work done by a "low quality" software developer (this includes almost all of us at some point in time), is always taken into account by the process”

    That’s a horrible take. There is no amount of reviews, guidelines and documentation that can compensate for low quality devs. You can’t throw garbage into the pipeline and then somehow process it to gold.

    16. ponector ◴[27 Aug 25 21:48 UTC] No.45045739{4}[source]▶
    Isn't it a future we are moving to? To hire poor (cheap) developers to do LLM-driven development.
    17. Cthulhu_ ◴[28 Aug 25 09:05 UTC] No.45050022[source]▶
    > (this includes almost all of us at some point in time)

    I'd say this includes all of us all the time; a good developer never trusts their own work blindly, and spends more time gathering requirements and verifying their and others' work than writing code.

    18. Cthulhu_ ◴[28 Aug 25 09:09 UTC] No.45050049{4}[source]▶
    This is why at every software project I've done in the past 15 odd years, steps were taken to prevent this in an automated and standardized fashion; code reviews of course, but they're more for functionality. Unit test requirements, integration / end-to-end tests based on acceptance criteria, visual regression tests, linting, type systems, OTAP, CI/CD, audit log via Git and standardized commit messages, etc etc etc.

    My job hasn't significantly changed with AI, as AI generated code still has to pass all the hurdles I've set up while setting up this project.