←back to thread

io_uring is faster than mmap

(www.bitflux.ai)
283 points ghuntley | 3 comments | 04 Sep 25 22:01 UTC | HN request time: 0s | source
Show context
jared_hulbert ◴[04 Sep 25 23:18 UTC] No.45133330[source]▶
Cool. Original author here. AMA.
replies(5): >>45133433 #>>45133597 #>>45133666 #>>45133764 #>>45135337 #
Jap2-0 ◴[04 Sep 25 23:33 UTC] No.45133433[source]▶
Would huge pages help with the mmap case?
replies(2): >>45133546 #>>45133572 #
jared_hulbert ◴[04 Sep 25 23:49 UTC] No.45133546[source]▶
Oh man... I'd have look into that. Off the top of my head I don't know how you'd make that happen. Way back when I'd have said no. Now with all the folio updates to the Linux kernel memory handling I'm not sure. I think you'd have to take care to make sure the data gets into to page cache as huge pages. If not then when you tried to madvise() or whatever the buffer to use huge pages it would likely just ignore you. In theory it could aggregate the small pages into huge pages but that would be more latency bound work and it's not clear how that impacts the page cache.

But the arm64 systems with 16K or 64K native pages would have fewer faults.

replies(1): >>45133578 #
1. inetknght ◴[04 Sep 25 23:54 UTC] No.45133578{3}[source]▶
> I'd have look into that. Off the top of my head I don't know how you'd make that happen.

Pass these flags to your mmap call: (MAP_HUGETLB | MAP_HUGE_1GB)

replies(1): >>45133626 #
2. jared_hulbert ◴[05 Sep 25 00:02 UTC] No.45133626[source]▶
Would this actually create huge page page cache entries?
replies(1): >>45133675 #
3. inetknght ◴[05 Sep 25 00:11 UTC] No.45133675[source]▶
It's right in the documentation for mmap() [0]! And, from my experience, using it with an 800GB file provided a significant speed-up, so I do believe the documentation is correct ;)

And, you can poke around in the linux kernel's source code to determine how it works. I had a related issue that I ended up digging around to find the answer to: what happens if you use mremap() to expand the mapping and it fails; is the old mapping still valid or not? Answer: it's still valid. I found that it was actually fairly easy to read linux kernel C code, compared to a lot (!) of other C libraries I've tried to understand.

[0]: https://www.man7.org/linux/man-pages/man2/mmap.2.html