←back to thread

Zlib-rs is faster than C

(trifectatech.org)
341 points dochtman | 1 comments | | HN request time: 0.218s | source
Show context
jrockway ◴[] No.43382147[source]
Chromium is kind of stuck with zlib because it's the algorithm that's in the standards, but if you're making your own protocol, you can do even better than this by picking a better algorithm. Zstandard is faster and compresses better. LZ4 is much faster, but not quite as small.

Some reading: https://jolynch.github.io/posts/use_fast_data_algorithms/

(As an aside, at my last job container pushes / pulls were in the development critical path for a lot of workflows. It turns out that sha256 and gzip are responsible for a lot of the time spent during container startup. Fortunately, Zstandard is allowed, and blake3 digests will be allowed soon.)

replies(4): >>43383191 #>>43383456 #>>43385563 #>>43392411 #
1. cesarb ◴[] No.43392411[source]
> Zstandard is faster and compresses better.

However, keep in mind that zstd also needs much more memory. IIRC, it uses by default 8 megabytes as its buffer size (and can be configured to use many times more than that), while zlib uses at most 32 kilobytes, allowing it to run even on small 16-bit processors.