←back to thread

SSDs have become fast, except in the cloud

(databasearchitects.blogspot.com)
589 points greghn | 3 comments | 20 Feb 24 16:59 UTC | HN request time: 0.538s | source
Show context
pclmulqdq ◴[20 Feb 24 17:22 UTC] No.39443994[source]▶
This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions.

SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, which was the backing technology when a lot of these network attached storage systems were invented, because they are fundamentally slow compared to networks, but it is a problem for SSD.

replies(30): >>39444009 #>>39444024 #>>39444028 #>>39444046 #>>39444062 #>>39444085 #>>39444096 #>>39444099 #>>39444120 #>>39444138 #>>39444328 #>>39444374 #>>39444396 #>>39444429 #>>39444655 #>>39444952 #>>39445035 #>>39445917 #>>39446161 #>>39446248 #>>39447169 #>>39447467 #>>39449080 #>>39449287 #>>39449377 #>>39449994 #>>39450169 #>>39450172 #>>39451330 #>>39466088 #
vlovich123 ◴[20 Feb 24 17:24 UTC] No.39444024[source]▶
Why do they fundamentally need to be network attached storage instead of local to the VM?
replies(5): >>39444042 #>>39444055 #>>39444065 #>>39444132 #>>39444197 #
SteveNuts ◴[20 Feb 24 17:33 UTC] No.39444132[source]▶
Because even if you can squeeze 100TB or more of SSD/NVMe in a server, and there are 10 tenants using the machine, you're limited to 10TB as a hard ceiling.

What happens when one tenant needs 200TB attached to a server?

Cloud providers are starting to offer local SSD/NVMe, but you're renting the entire machine, and you're still limited to exactly what's installed in that server.

replies(3): >>39444256 #>>39444774 #>>39446160 #
1. jalk ◴[20 Feb 24 17:41 UTC] No.39444256[source]▶
How is that different from how cores, mem and network bandwidth is allotted to tenants?
replies(2): >>39444337 #>>39444566 #
2. pixl97 ◴[20 Feb 24 17:49 UTC] No.39444337[source]▶
Because a fair number of customers spin up another image when cores/mem/bandwidth run low. Dedicated storage breaks that paradigm.

Also, adding, if I am on an 8 core machine and need 16, network storage can be detached from host A and connected to host B. In dedicated storage it must be fully copied over first.

3. baq ◴[20 Feb 24 18:06 UTC] No.39444566[source]▶
It isn't. You could ask for network-attached CPUs or RAM. You'd be the only one, though, so in practice only network-attached storage makes sense business-wise. It also makes sense if you need to provision larger-than-usual amounts like tens of TB - these are usually hard to come by in a single server, but quite mundane for storage appliances.