The real issue is actually runtime ELF (and PE) which are obsolete on modern hardware architecture.
replies(1):
On modern large hardware architecture, for executable files/dynamic libraries, ELF(PE[+]) has overkill complexity.
I am personnally using a executable file format of my own I do wrap into an "ELF capsule" on linux kernel. With position independent code, you kind of only need memory mapped segments (which dynamic libraries are in this very format). I have two very simple partial linkers I wrote in plain and simple C, one for risc-v assembly, one for x86_64 assembly, which allow me to link into such executable file some simple ELF object files (from binutils GAS).
There is no more centralized "ELF loader".
Of course, there are tradeoffs, 1 billion times worth it in regards of the accute simplicity of the format.
(I even have a little vm which allows me to interpret simple risc-v binaries on x86_64).
A good analogy to "feel" that, it is a bit like "json vs xml" but for executable binary formats.
But, I keep in mind, those formats (excrutiatingly simple) can work only on modern hardware architectures.