←back to thread

Details of the Cloudflare outage on July 2, 2019

(blog.cloudflare.com)
698 points jgrahamc | 1 comments | 12 Jul 19 15:46 UTC | HN request time: 0.205s | source
Show context
_wmd ◴[12 Jul 19 17:50 UTC] No.20422740[source]▶
So in response to a catastrophic failure due to testing in prod, they're going to push out a brand new regex engine with an ETA of 2 weeks. Can anyone say testing in prod?

The constant use of 'I' and 'me' (19 occurrences in total) deeply tarnishes this report, and repeatedly singling out a responsible engineer, nameless or not, is a failure in its own right. This was a collective failure, any individual identity is totally irrelevant. We're not looking for an account of your superman-like heroism, sprinting from meeting rooms or otherwise, we want to know whether anything has been learned in the 2 years since Cloudflare leaked heap all across the Internet without noticing, and the answer to that seems fantastically clear.

replies(6): >>20422871 #>>20422873 #>>20422891 #>>20422903 #>>20422924 #>>20424743 #
1. staticassertion ◴[12 Jul 19 18:06 UTC] No.20422924[source]▶
I don't see why switching to a new regex implementation would be so scary. 2 weeks to test that your regexes don't break seems fine? Seems like a long time tbh.

On top of that they're switching to more constrained regex engines. Rust's regex engine makes guarantees about its running time, something that would have directly mitigated a portion of the issue. And it isn't as if RE2/Rust regex aren't in use anywhere, rust's regex engine is integrated into vscode, for example.