←back to thread

Details of the Cloudflare outage on July 2, 2019

(blog.cloudflare.com)
698 points jgrahamc | 1 comments | 12 Jul 19 15:46 UTC | HN request time: 0s | source
Show context
dsr_ ◴[12 Jul 19 19:31 UTC] No.20423664[source]▶
9. We had difficulty accessing our own systems because of the outage and the bypass procedure wasn’t well trained on.

Suggestion for future, learned from bitter experience: separate your control plane from your data plane. In this case, make sure that the tools you use to manage your infrastructure don't depend on that infrastructure being functional.

That way you won't have to remember how to use a bypass procedure -- it will just be your normal procedure.

replies(2): >>20424720 #>>20428852 #
vayne ◴[12 Jul 19 21:33 UTC] No.20424720[source]▶
Yeah, that is true in most cases. However, here is Cloudflare was using Cloudflare on dash.cloudflare.com as well.

This calls for not using Cloudflare for their web dashboard.

replies(1): >>20426150 #
jjeaff ◴[13 Jul 19 02:31 UTC] No.20426150[source]▶
Then comes the inevitable tweets every few weeks. "cloudflare doesn't event trust cloudflare to run their own control panel"
replies(1): >>20426675 #
1. vegiraghav ◴[13 Jul 19 05:42 UTC] No.20426675[source]▶
couldn't it be named in order to indicate cloudflare uses cloudflare just in a different way.