←back to thread

SSDs have become fast, except in the cloud

(databasearchitects.blogspot.com)
589 points greghn | 2 comments | | HN request time: 0s | source
Show context
pclmulqdq ◴[] No.39443994[source]
This was a huge technical problem I worked on at Google, and is sort of fundamental to a cloud. I believe this is actually a big deal that drives peoples' technology directions.

SSDs in the cloud are attached over a network, and fundamentally have to be. The problem is that this network is so large and slow that it can't give you anywhere near the performance of a local SSD. This wasn't a problem for hard drives, which was the backing technology when a lot of these network attached storage systems were invented, because they are fundamentally slow compared to networks, but it is a problem for SSD.

replies(30): >>39444009 #>>39444024 #>>39444028 #>>39444046 #>>39444062 #>>39444085 #>>39444096 #>>39444099 #>>39444120 #>>39444138 #>>39444328 #>>39444374 #>>39444396 #>>39444429 #>>39444655 #>>39444952 #>>39445035 #>>39445917 #>>39446161 #>>39446248 #>>39447169 #>>39447467 #>>39449080 #>>39449287 #>>39449377 #>>39449994 #>>39450169 #>>39450172 #>>39451330 #>>39466088 #
jsnell ◴[] No.39444096[source]
According to the submitted article, the numbers are from AWS instance types where the SSD is "physically attached" to the host, not about SSD-backed NAS solutions.

Also, the article isn't just about SSDs being no faster than a network. It's about SSDs being two orders of magnitude slower than datacenter networks.

replies(3): >>39444161 #>>39444353 #>>39448728 #
pclmulqdq ◴[] No.39444161[source]
It's because the "local" SSDs are not actually physically attached and there's a network protocol in the way.
replies(14): >>39444222 #>>39444248 #>>39444253 #>>39444261 #>>39444341 #>>39444352 #>>39444373 #>>39445175 #>>39446024 #>>39446163 #>>39446271 #>>39446742 #>>39446840 #>>39446893 #
candiddevmike ◴[] No.39444253[source]
Depends on the cloud provider. Local SSDs are physically attached to the host on GCP, but that makes them only useful for temporary storage.
replies(3): >>39444326 #>>39444754 #>>39445986 #
pclmulqdq ◴[] No.39444326[source]
If you're at G, you should read the internal docs on exactly how this happens and it will be interesting.
replies(3): >>39444529 #>>39450240 #>>39450805 #
rfoo ◴[] No.39444529[source]
Why would I lose all data on these SSDs when I initiate a power off of the VM on console, then?

I believe local SSDs are definitely attached to the host. They are just not exposed via NVMe ZNS hence the performance hit.

replies(2): >>39444859 #>>39445006 #
1. manquer ◴[] No.39444859[source]
It is because on reboot you may not get the same physical server . They are not rebooting the physical server for you , just the VM

Same VM is not allocated for a variety of reasons , scheduled maintenance, proximity to other hosts on the vpc , balancing quiet and noisy neighbors so on.

It is not that the disk will always wiped , sometimes the data is still there on reboot just that there is no guarantee allowing them to freely move between hosts

replies(1): >>39448758 #
2. mr_toad ◴[] No.39448758[source]
Data persists between reboots, but not shutdowns:

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-inst...

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-inst...