(www.vldb.org)

77 points eatonphil | 1 comments | 15 May 24 18:29 UTC | HN request time: 0.213s | source

Show context

0cf8612b2e1e ◴[15 May 24 19:50 UTC] No.40371539[source]▶

  Third, faster and cheaper storage devices mean that it is better to use faster decoding schemes to reduce computation costs than to pursue more aggressive compression to save I/O bandwidth. Formats should not apply general-purpose block compression by default because the bandwidth savings do not justify the decompression overhead.

Not sure I agree with that. Have a situation right now where I am bottlenecked by IO and not compute.

replies(6): >>40372011 #>>40372288 #>>40372399 #>>40372660 #>>40373077 #>>40373820 #

1. apavlo ◴[15 May 24 21:11 UTC] No.40372399[source]▶

>>40371539 #

> Have a situation right now where I am bottlenecked by IO and not compute.

Can you describe your use-case? Are you reading from NVMe or S3?

↑

An Empirical Evaluation of Columnar Storage Formats [pdf]