←back to thread

An Empirical Evaluation of Columnar Storage Formats [pdf]

(www.vldb.org)
77 points eatonphil | 1 comments | 15 May 24 18:29 UTC | HN request time: 0s | source
Show context
0cf8612b2e1e ◴[15 May 24 19:50 UTC] No.40371539[source]▶

  Third, faster and cheaper storage devices mean that it is better to use faster decoding schemes to reduce computation costs than to pursue more aggressive compression to save I/O bandwidth. Formats should not apply general-purpose block compression by default because the bandwidth savings do not justify the decompression overhead.
Not sure I agree with that. Have a situation right now where I am bottlenecked by IO and not compute.
replies(6): >>40372011 #>>40372288 #>>40372399 #>>40372660 #>>40373077 #>>40373820 #
1. jauntywundrkind ◴[16 May 24 00:06 UTC] No.40373820[source]▶
I struggle to imagine being bandwidth limited in this day and age. Kioxia makes some mean mean SSDs, for not a wild price. A 1u can fit dozens of thee monsters easily.