←back to thread

Hitting Peak File IO Performance with Zig

(steelcake.com)
129 points ozgrakkurt | 3 comments | 04 Sep 25 19:15 UTC | HN request time: 0s | source
Show context
mgerdts ◴[07 Sep 25 18:31 UTC] No.45160907[source]▶
> The result I get from fio is 4.083 GB/s write, … The result for the diorw example in zig is 3.802 GB/s write….

4.083 / 3.802 = 1.0739

2^30 / 10^9 = 1.0737

I think the same rate was likely achieved but there is confusion between GiB and GB.

replies(1): >>45163287 #
ozgrakkurt ◴[07 Sep 25 23:46 UTC] No.45163287[source]▶
Yeah, seems like fio might be mixing them up or it is just faster than the zig code.

Fio seems to interpret `16g` as 16GiB so it creates a 16GiB ~= 17.2GB file. But not sure if it is reading/writing the whole thing.

It seems like the max performance of the SSD is 7GB/s in spec so it is kind of confusing.

replies(1): >>45167222 #
mgerdts ◴[08 Sep 25 11:58 UTC] No.45167222[source]▶
By default fio will read or write the whole thing. There are options to control how much will be read or written, perhaps bounded by time rather than space. The default output format of fio displays the results in both SI (10^x) and IEC (2^x) units per second. Other output formats are available that give values in bytes.

NVMe drive vendors always market size in GB (or TB) and data rates in GB/s.

replies(1): >>45167696 #
1. ozgrakkurt ◴[08 Sep 25 12:58 UTC] No.45167696[source]▶
Ahh, just realised it also interprets block size as KiB instead of KB.

I have this config:

bs=512KB size=16GB

But it interprets KiB and GiB so this was causing my confusion.

The IOPS and timing is basically identical.

So output seems fine but it always interprets parameters as SI.

Edit: Actually after looking into it more. It seems like there is a good chance that fio reports GiB and KiB in output and it also does the calculation based on that but in reality it uses GB/KB so measurements are a bit wrong.

replies(1): >>45173724 #
2. mgerdts ◴[08 Sep 25 20:43 UTC] No.45173724[source]▶
fio is a mature widely used tool. It is the only tool that I’ve seen used in the disclosures for performance measurement in NVMe drive datasheets. I’ve used it regularly and see a strong correlation between what it reports and what I can see from other sources measuring the same load.

I categorize suggestions by new fio users that they found a bug in fio similarly to how I categorize claims by new C programmers that they found a bug in malloc.

replies(1): >>45175084 #
3. ozgrakkurt ◴[08 Sep 25 22:43 UTC] No.45175084[source]▶
Ok It was because what I think is a KB is really a KiB.

I though KB = 1024 bytes and KiB = 1000 bytes but it is the other way around