←back to thread

So you want to build your own data center

(blog.railway.com)
596 points dban | 3 comments | 17 Jan 25 20:41 UTC | HN request time: 0.428s | source
Show context
toddmorey ◴[18 Jan 25 00:09 UTC] No.42744572[source]▶
Reminds me of the old Rackspace days! Boy we had some war stories:

   - Some EMC guys came to install a storage device for us to test... and tripped over each other and knocked out an entire Rack of servers like a comedy skit. (They uh... didn't win the contract.)
   - Some poor guy driving a truck had a heart attack and the crash took our DFW datecenter offline. (There were ballards to prevent this sort of scenario, but the cement hadn't been poured in them yet.)
   - At one point we temporarily laser-beamed bandwidth across the street to another building
   - There was one day we knocked out windows and purchased box fans because servers were literally catching on fire.
Data center science has... well improved since the earlier days. We worked with Facebook on the OpenCompute Project that had some very forward looking infra concepts at the time.
replies(11): >>42744904 #>>42745196 #>>42745385 #>>42745719 #>>42745862 #>>42746580 #>>42748445 #>>42749131 #>>42749468 #>>42749547 #>>42751611 #
1. kryogen1c ◴[18 Jan 25 04:02 UTC] No.42745719[source]▶
> Data center science has... well improved since the earlier days

You say that, but...

> There was one day we knocked out windows and purchased box fans because servers were literally catching on fire

This happened to Equinix's CH1 datacenter in Chicago Jan24 (not the literal fire part). Took down Azure ExpressRoute.

Apparently it got too cold and the CRACs couldn't take it? I'm told they had all the doors and windows open trying to keep things cold enough, but alas. As the CRAC goes, so goes the servers

replies(2): >>42748486 #>>42750060 #
2. inopinatus ◴[18 Jan 25 14:16 UTC] No.42748486[source]▶
running European ISPs in summer we’d nick desk fans off the telco folks to cool down our walls of USR Sportsters, distracting them first with snarky remarks about ATM overhead

absolutely do not miss those days

3. Henchman21 ◴[18 Jan 25 18:00 UTC] No.42750060[source]▶
I’ve worked in CH1 for years now. The glycol in the chillers froze. Thats how cold it was!

It was also 115 degrees ambient temp inside CH1. Techs were dipping in and out 5-10 minutes at a time to avoid heat stroke