War story: the hardest bug I ever debugged

> I do it a few more times. It’s not always the 20th iteration, but it usually happens sometime between the 10th and 40th iteration. Sometimes it never happend. Okay, the bug is nondeterministic.

That’s an incorrect assumption. Just because your test case isn’t triggering the bug reliably, it does not mean the bug is nondeterministic.

That is like saying the “OpenOffice can’t print on Tuesdays” is non deterministic because you can’t reproduce it everyday. It is deterministic, you just need to find the right set of circumstances.

https://beza1e1.tuxen.de/lore/print_on_tuesday.html

From the writing it appears the author found one way to reproduce the bug sometimes and then relied on it for every test. Another approach would have been to tweak their test case until they found a situation which reproduced the bug more or less often, trying to find the threshold that causes it and continuing to deduce from there.

"Deterministic" is .. something of a moveable feast. We'd generally agree that "software is deterministic in that if you provide the same inputs to the same executable machine code it will return the same value", which is nearly always true unless someone is irradiating your processor or trying to voltage-glitch it.

But there's a lot hidden in "same inputs", because that includes everything that's an input to your program from the operating system. Which includes things like "time" (bane of reproduction), memory layout, execution scheduling order of multithreaded code, value of uninitialized memory, and so on.

> Another approach would have been to tweak their test case until they found a situation which reproduced the bug more or less often, trying to find the threshold that causes it and continuing to deduce from there.

Yes - when dealing with unknowns in a huge problem space it can be very effective to play hotter-colder and climb up the hill.