TPDE-LLVM: Faster LLVM -O0 Back-End

IMO the worst problem with LLVM isn't that it's slow, the worst problem is that its IR has poorly defined semantics or its team doesn't actually deliver those semantics and a bug ticket saying "Hey, what gives?" goes in the pile of never-never tickets, making it less useful as a compiler backend even if it was instant.

This is the old "correctness versus performance" problem and we already know that "faster but wrong" isn't meaningfully faster it's just wrong, anybody can give a wrong answer immediately and so that's not at all useful.

What is the alternative though for a new language though? Transpiring to C or hacking s.th. by using the GCC backend?

The easy alternative? There isn't one.

The really difficult thing would be to write a new compiler backend with a coherent IR that everybody understands and you'll stick to. Unfortunately you can be quite certain that after you've done the incredible hard work to build such a thing, a lot of people's assessment of your backend will be:

1. The code produced was 10% slower than LLVM, never use this, speed is all that matters anyway and correctness is irrelevant.

2. This doesn's support the Fongulab Splox ZV406 processor made for six years in the 1980s, whereas LLVM does, therefore this is a waste of time.

Or native code generation. Depends on what your performance goals are. It would be cool if there was a standard IR that languages could target - something more suitable than C.

Hm. SPIR-V is a standard IR, but AFAIU not really the kind of IR that you need for communication inside a compiler. It wasn't designed for that.

Being pursued since UNCOL in 1958, each attempt eventually only works out for a specific set of languages, due to politics or market forces.

Produce a dumb machine code quality, enough to bootstrapt it, and go from there.

Move away from classical UNIX compiler pipelines.

However in current times, I would rather invest into LLM improvements into generating executables directly, the time to mix AI into compiler development has come, and classical programming languages are just like doing yet another UNIX clone, in terms of value.

mm, a non deterministic compiler with no way to verify correctness, what could go wrong lol

WASM ?

Ok, but then if this were done, then you could also emit LLVM after. It probably get worse timings, but, allow to make the transition palatable

WASM !

Ask C and C++ developers, they are used to it, and still plenty of critical software keeps being written with them.

> The really difficult thing would be to write a new compiler backend with a coherent IR that everybody understands and you'll stick to.

But why would you bother, when with those same skills and a lot less time, you could fork LLVM, correct its IR semantics yourself (unilaterally), and then push people to use your fork?

(I.e. the EGCS approach to forcing the upstream to fix their shit.)

> This doesn's support the Fongulab Splox ZV406 processor made for six years in the 1980s, whereas LLVM does, therefore this is a waste of time.

AFAIK, the various Fongulab Sploxes that LLVM has targets for, are mostly there to act as forcing functions to keep around features that no public backend would otherwise rely on, because proprietary, downstream backends rely on those features. (See e.g. https://q3k.org/lanai.html — where the downstream ISA of interest is indeed proprietary, but used to be public before an acquisition; so the contributor [Google] upstreamed an implementation of the old public ISA target.)

C and C++ compilers are deterministic and have guarantees of correctness similar to that of other languages (esp ones that share share the same llvm backend)

Provided you're clever enough to avoid UB land mines, and compiler specific implementation non portable behaviours.

C++ compilers are not required to be deterministic and in practice are not, at least as far as "same source code produces same observable behavior". Things that can introduce non-determinism include the order in which symbols are linked, static variable initialization, floating point operations (unless you use strict mode, which is not mandated by the standard), and this is ignoring the obvious stuff like unspecified behavior which is specifically defined as behavior which can differ between different runs on the same system.

Also correctness guarantees? Hahaha... I'll pretend you didn't just claim C++ has correctness guarantees on par with other languages, LLVM or otherwise. C++ gives you next to nothing with respect to correctness guarantees.

Thanks for the link about Lanai although that site's cert has expired (very recently too) so it's slightly annoying (or of course bad guys are attacking me)

As to the first point, I suspect this is a foundational problem. Like, suppose you realise the concrete used to make a new skyscraper was the wrong mixture. In a sense this is a small change, there's nothing wrong with the elevators, the windows, cabling, furnishing, air conditioning, and so on. But, to "fix" this problem you need to tear down the skyscraper and replace it. Ouch.

I may be wrong, I have never tried to solve this problem. But I fear...

It's probably not too bad of an option these days tbf. Are there any optimising WASM compilers that can get close to native performance?

Excellent point, undefined behavior is exactly like an LLM. Surely this is what “alignment” in the standard is talking about.

Can you point to cases where you feel this has caused harm that you feel outweighs the collective time people spend waiting for LLVM builds?