War story: the hardest bug I ever debugged

My own story: I spent >10 hours debugging an Emacs project that would occasionally cause a kernel crash on my machine. Proximate cause was a nonlocal interaction between two debug-print statements. (Wasn't my first guess). The Elisp debug-print function #'message has two effects: it appends to a log, and also does a small update notification in the corner of the editor window. If that corner-of-the-window GUI object is thrashed several hundred times in a millisecond, it would cause the GPU driver on my specific machine to lock up, for a reason I've never root-caused.

Emacs' #'message implementation has a debounce logic, that if you repeatedly debug-print the same string, it gets deduplicated. (If you call (message "foo") 50 times fast, the string printed is "foo [50 times]"). So: if you debug-print inspect a variable that infrequently changes (as was the case), no GUI thrashing occurs. The bug manifested when there were *two* debug-print statements active, which circumvented the debouncer, since the thing being printed was toggling between two different strings. Commenting out one debug-print statement, or the other, would hide the bug.

> If that corner-of-the-window GUI object is thrashed several hundred times in a millisecond, it would cause the GPU driver on my specific machine to lock up, for a reason I've never root-caused.

Until comparatively recently, it was absurdly easy to crash machines via their graphics drivers, even by accident. And I bet a lot of them were security concerns, not just DoS vectors. WebGL has been marvellous at encouraging the makers to finally fix their drivers properly, because browsers declared that kind of thing unacceptable (you shouldn’t be able to bring the computer down from an unprivileged web page¹), and developed long blacklists of cards and drivers, and brought the methodical approach browsers had finally settled on to the graphics space.

Things aren’t perfect, but they are much better than ten years ago.

—⁂—

¹ Ah, fond memories of easy IE6 crashes, some of which would even BSOD Windows 98. My favourite was, if my memory serves me correctly, <script>document.createElement("table").appendChild(document.createElement("div"))</script>. This stuff was not robust.