←back to thread

lsr: ls with io_uring

(rockorager.dev)
335 points mpweiher | 1 comments | | HN request time: 0s | source
Show context
SillyUsername ◴[] No.44604307[source]
Love it.

I'm trying to understand why all command line tools don't use io_uring.

As an example, all my nvme's on usb 3.2 gen 2 only reach 740MB/s peak.

If I use tools with aio or io_uring I get 1005MB/s.

I know I may not be copying many files simultaneously every time, but the queue length strategies and the fewer locks also help I guess.

replies(9): >>44604401 #>>44604434 #>>44604490 #>>44604735 #>>44604738 #>>44604905 #>>44605976 #>>44607467 #>>44608653 #
tyingq ◴[] No.44604434[source]
Probably historical preference for portability without a bunch of #ifdef means platform+version-specific stuff is very late to get adopted. Though, at this point, the benefit of portability across various posixy platforms is much lower.
replies(1): >>44604570 #
Retr0id ◴[] No.44604570[source]
Has anyone written an io_uring "polyfill" library with fallback to standard posix-y IO? It could presumably be done via background worker threads - at a perf cost.
replies(1): >>44604729 #
vlovich123 ◴[] No.44604729{3}[source]
Seems like a huge lift since io_uring is an ever growing set of interfaces that is encompassing more and more of the kernel surface area. Also, the problem tends to not necessarily be that the io_uring interface isn’t available at compile time but a) the version you distribute to has a kernel with it disabled or you don’t have permission to use it meaning you need to do LD_preload magic or use a framework b) the kernel you’re using supports some of the interfaces you’re trying to use but not all. Not sure how you solve that one without using a framework.

But I agree. It would be cool if it was transparent, but this is actually what a bunch of io-uring runtimes do, using epoll as a fallback (eg in Rust monoio)

replies(1): >>44608547 #
1. namibj ◴[] No.44608547{4}[source]
You can just ask io_uring what commands you have available to you. Though the way of the background thread should be readily available/usable by just indirectly calling the syscall (-helper) and replacing it with a futex-based handshake/wrapper. If you're not using the backend-ordering-imposing link bit, you could probably even use minor futex trickery to dispatch multiple background threads to snatch up from the submission queue in a "grab one at a time" fashion.