←back to thread

420 points speckx | 4 comments | | HN request time: 0.016s | source
Show context
dylan604 ◴[] No.44533476[source]
SSD speeds are nothing short of miraculous in my mind. I come from the old days of striping 16 HDDs together (at a minimum number) to get 1GB/s throughput. Depending on the chassis, that was 2 8-drive enclosures in the "desktop" version or the large 4RU enclosures with redundant PSUs and fans loud enough to overpower arena rock concerts. Now, we can get 5+GB/s throughput from a tiny stick that can be used externally via a single cable for data&power that is absolutely silent. I edit 4K+ video as well, and now can edit directly from the same device the camera recorded to during production. I'm skipping over the parts of still making backups, but there's no more multi-hour copy from source media to edit media during a DIT step. I've spent many a shoot as a DIT wishing the 1s&0s would travel across devices much faster while everyone else on the production has already left, so this is much appreciated by me. Oh, and those 16 device units only came close to 4TB around the time of me finally dropping spinning rust.

The first enclosure I ever dealt with was a 7-bay RAID-0 that could just barely handle AVR75 encoding from Avid. Just barely to the point that only video was saved to the array. The audio throughput would put it over the top, so audio was saved to a separate external drive.

Using SSD feels like a well deserved power up from those days.

replies(8): >>44533735 #>>44534375 #>>44535266 #>>44535471 #>>44536311 #>>44536501 #>>44539458 #>>44539872 #
bob1029 ◴[] No.44536311[source]
The latency of modern NVMe is what really blows my mind (as low as 20~30 uS). NVMe is about an order of magnitude quicker than SAS and SATA.

This is why I always recommend developers try using SQLite on top of NVMe storage. The performance is incredible. I don't think you would see query times anywhere near 20uS with a hosted SQL solution, even if it's on the same machine using named pipes or other IPC mechanism.

replies(3): >>44536919 #>>44540452 #>>44542264 #
Numerlor ◴[] No.44536919[source]
Then there's optane that got ~10us with. The newest controllers and nand is inching closer with randoms but optane is still the most miraculous ssd tech that's normally obtainable
replies(3): >>44537013 #>>44537056 #>>44538546 #
teaearlgraycold ◴[] No.44537056[source]
Eventually we'll have machines with unified memory+storage. You'll certainly have to take a bit of a performance hit in certain scenarios but also think about the load time improvements. If you store video game files in the same format they'd be needed at runtime you could be at the main menu in under a second.
replies(3): >>44537445 #>>44538862 #>>44543140 #
bob1029 ◴[] No.44537445[source]
At a minimum, we should be able to get everything to DRAM speeds. Beyond that you start to run into certain limitations. Achieving L1 latency is physically impossible if the storage element is more than a few inches away from the CPU.
replies(1): >>44537754 #
1. devmor ◴[] No.44537754{3}[source]
Most new motherboards do already have the highest throughput M.2 connector very near the CPU.

The most recent desktop I built has it situated directly below the standard formfactor x16 PCI slot.

replies(1): >>44538706 #
2. lmz ◴[] No.44538706[source]
I think part of the reason why it's so close is also for signal integrity reasons.
replies(2): >>44539020 #>>44539190 #
3. thfuran ◴[] No.44539020[source]
Far more so than latency.
4. privatelypublic ◴[] No.44539190[source]
Also: routing PCIe lanes is a pain. Being able to take 4 pairs and terminate them makes routing everything else simple