←back to thread

War story: the hardest bug I ever debugged

(www.clientserver.dev)
462 points jakevoytko | 1 comments | 24 Mar 25 14:37 UTC | HN request time: 0.234s | source
Show context
BobbyTables2 ◴[27 Mar 25 03:27 UTC] No.43490119[source]▶
Interesting writeup, but 2 days to debug “the hardest bug ever”, while accurate, seems a bit overdone.

Though abs() returning negative numbers is hilarious.. “You had one job…”

To me, the hardest bugs are nearly irreproducible “Heisenbugs” that vanish when instrumentation is added.

I’m not just talking about concurrency issues either…

The kind of bug where a reproduction attempt takes a week, not parallelizable due to HW constraints, and logging instrumentation makes it go away or fail differently.

2 days is cute though.

replies(13): >>43490149 #>>43490287 #>>43490459 #>>43490557 #>>43491079 #>>43491823 #>>43492539 #>>43492555 #>>43492647 #>>43493115 #>>43493245 #>>43493811 #>>43497018 #
mystified5016 ◴[27 Mar 25 14:04 UTC] No.43493811[source]▶
In hardware, you regularly see behavior change when you probe the system. Your oscilloscope or LA probes affect the system just enough to make a marginal circuit work. It's absolutely maddening.
replies(1): >>43496221 #
1. fuzzfactor ◴[27 Mar 25 18:05 UTC] No.43496221[source]▶
The closer you get to natural science, eventually reliance on logical troubleshooting can be "illogical".

The more abundant the undefined (mis)behavior, the more you're going to be tearing your hair out.

Almost the kind of frustration where you're supposed to have a logic-based system, and it rears it ugly head and defies logic anyway :\