Meh, these are just examples of the inability to correctly root cause issues. There is a good lesson in here about the real cause being lack of testing (the teammate’s DOM change should have never merged) and lack of monitoring (upstream mail provider failure should have been setting off alerts a long time ago).
The changes only had adjacency to the causes and that’s super common on any system that has a few core pieces of functionality.
I think the core lesson here is that if you can’t fully explain the root cause, you haven’t found the real reason, even if it seems related.
replies(1):