Fun one:
I work on a server software of online backups for customers. We do daily thousands of mount/umount of a particular filesystem. Once every month or so, we get an issue where a file timestamp fails to save, the error happens at the filesystem level.
Hard to reproduce! It's a filesystem bug! So it's full theorical, reading code and seeing how it would happen.
Found out after a while, the conditions were fun. I don't remember exactly, but it was like, you need to follow these steps : 1/ Create a folder 2/ Create in it 99 files (no more no less) 3/ Create a new folder 4/ Copy the first of the 99 files in the new folder
The issue was linked to some data structure caching, and cache eviction.
Had fun finding it out!