Why is the Rust compiler so slow?

Go has sub-second build times even on massive code-bases. Why? because it doesn't do a lot at build time. It has a simple module system, (relatively) simple type system, and leaves a whole bunch of stuff be handled by the GC at runtime. It's great for its intended use case.

When you have things like macros, advanced type systems, and want robustness guarantees at build time.. then you have to pay for that.

One of the primary features of Rust is the extensive compile-time checking. Monomorphization is also a complex operation, which is not exclusive to Rust.

C compile times should be very fast because it's a relatively low-level language.

On the grand scale of programming languages and their compile-time complexity, C code is closer to assembly language than modern languages like Rust or Swift.

The overall principle is sound though: it's true that doing some work is more than doing no work. But the borrow checker and other safety checks are not the root of compile time performance in Rust.

Do you really believe that nobody over the course of Rust's lifetime has ever taken a look at C compilers and thought about if techniques they use could apply to the Rust compiler?

Unity builds are useful for C programs because they tend to reduce header processing overhead, whereas Rust does not have the preprocessor or header files at all.

They also can help with reducing the number of object files (down to one from many), so that the linker has less work to do, this is already sort of done (though not to literally one) due to what I mentioned above.

In general, the conventional advice is to do the exact opposite: breaking large Rust projects into more, smaller compilation units can help do less "spurious" rebuilding, so smaller changes have less overall impact.

Basically, Rust's compile time issues lie elsewhere.

* Make no nested types - these slow compiler time a lot

* Include no crates, or ones that emphasize compiler speed

C is still v. fast though. That's why I love it (and Rust).

A big reason that amalgamation builds of C and C++ can absolutely fly is because they aren't reparsing headers and generating exactly one object file so the linker has no work to do.

Once you add static linking to the toolchain (in all of its forms) things get really fucking slow.

Codegen is also a problem. Rust tends to generate a lot more code than C or C++, so while the compiler is done doing most of its typechecking work, the backend and assembler has a lot of things to chuck through.

The compiler is optimized for compilation speed, not runtime performance. Generally speaking, it does well enough. Especially because it's usecase is often applications where "good enough" is good enough (IE, IO heavy applications).

You can see that with "gccgo". Slower to compile, faster to run.

The rust compiler is actually pretty fast for all the work it's doing. It's just an absolutely insane amount of additional work. You shouldn't expect it to compile as fast as C.

I can believe that, but even so it's caused by the type system monomorphising everything. When it use qsort from libc, you are using per-compiled code from a library. When you use slice::sort(), you get custom assembler compiled to suit your application. Thus, there is a lot more code generation going on, and that is caused by the tradeoffs they've made with the type system.

Rusts approach give you all sorts of advantages, like fast code and strong compile time type checking. But it comes with warts too, like fat binaries, and a bug in slice::sort() can't be fixed by just shipping of the std dynamic library, because there is no such library. It's been recompiled, just for you.

FWIW, modern C++ (like boost) that places everything in templates in .h files suffers from the same problem. If Swift suffers from it too, I'd wager it's the same cause.

I was all excited to conduct the "cargo check; mrustc; cc" is 100x faster experiment, but I think at best, the multiple is going to be pretty small.

1. Use pointers and do not include header file for class, if you need pointer to that class. I think that's a pretty established pattern in C++. So if you want to declare pointer to a class in your header, you just write `class SomeClass;` instead of `#include "SomeClass.hpp"`.

2. Do not use STL or IOstreams. That project used only libc and POSIX API. I know that author really hated STL and considered it a huge mistake to be included to the standard language.

3. Avoid generic templates unless absolutely necessary. Templates force you to write your code in header file, so it'll be parsed multiple times for every include, compiled to multiple copies, etc. And even when you use templates, try to split the class to generic and non-generic part, so some code could be moved from header to source. Generally prefer run-time polymorphism to generic compile-time polymorphism.

Stuff like inserting bounds checking puts more work on the optimization passes and codegen backend as it simply has to deal with more instructions. And that then puts more symbols and larger sections in the input to the linker, slowing that down. Even if the frontend "proves" it's unnecessary that calculation isn't free. Many of those features are related to "safety" due to the goals of the language. I doubt the syntax itself really makes much of a difference as the parser isn't normally high on the profiled times either.

Generally it provides stricter checks that are normally punted to a linter tool in the c/c++ world - and nobody has accused clang-tidy of being fast :P

This has tradeoffs: increased ABI stability at the cost of longer compile times.

There's some other dependencies in there that are only used when building for test/benchmarking like serde, zstd, and criterion. You would need to be certain you're building only the library and not the test harness to be sure those aren't being built too.

Maybe it's a MSVC thing - it does seem to have some multi-threading stuff. In any case raddbg non-clean builds take longer than any of my rust projects.

For pure computational workloads, it'll be faster. However, anything with heavy allocation will suffer as apparently the gccgo GC and GC related optimizations aren't as good as cgo's.

https://codeload.github.com/EpicGamesExt/raddebugger/tar.gz/...

I’d like to see tooling for this to pinpoint bottlenecks - it’s not always obvious what’s making builds slow.

Since fast compilation was a goal, every part of the design was looked at through a rough "can this be a horrible bottleneck?", and discarded if so. For example, the import (package) system was designed to avoid the horrible, inefficient mess of C++. It's obvious that you never want to compile the same package more than once and that you need to support parallel package compilation. These may be blindingly obvious, but if you don't think about compilation speed at design time, you'll get this wrong and will never be able to fix it.

As far as optimizations vs compile speed goes, it's just a simple case of diminishing returns. Since Rust has maximum possible perfomance as a goal, it's forced to go well into the diminishing returns territory, sacrificing a ton of compile speed for minor performance improvements. Go has far more modest performance goals, so it can get 80% of the possible performance for only 20% of the compile cost. Rust can't afford to relax its stance because it's competing with languages like C++, and to some extent C, that are willing to go to any length to squeeze out an extra 1% of perfomance.

If you want to see the difference download unreal engine and compile the editor with and without unity builds enabled.

My experience has been the polar opposite of yours - similar size rust projects are an order of magnitude slower than C++ ones. Could you share an example of a project to compare with?

I second this enthusiastically.

That's my 2000s development experience. Fortunately I've spent a good chunk of the 2010s and most of the 2020s using other languages.

The classic XKCD compilation comic exists for a reason.

https://github.com/rust-lang/rustc_codegen_cranelift

Wouldn't you say a lot of that comes from the macros and (by way of monomorphisation) the type system?

Even this can lead to unworkable compile times, to the point that code is rewritten.

I wouldn't like it that much

It's not that it can't be done but that it usually is not worth the hassle and our goal should be for compilation to be fast despite not everything being in one file.

Turbo Pascal is a prime example for a compiler that won the market not least because of its - for the time - outstanding compilation speed.

In the same vein, a language can be designed for fast compilation. Pascal in general was designed for single-pass compilation which made it naturally fast. All the necessary forward declarations were a pain though and the victory of languages that are not designed for single-pass compilation proofs that while doable it was not worth it in the end.

I'm not sure what Rust or docker have to do with this basic issue, it just feels like young blood attempting 2020 solutions before exploring 1970 solutions.

But not having to is a win, as the monomorphised sorts are just much faster at runtime than having to do an indirect call for each comparison.

But it _is_ about the sheer volume of stuff passed to LLVM, as you say, which comes from a couple of places, mostly related to monomorphization (generics), but also many calls to tiny inlined functions. Incidentally, this is also what makes many "modern" C++ projects slow to compile.

In my experience, similarly sized Rust and C++ projects seem to see similar compilation times. Sometimes C++ wins due to better parallelization (translation units in Rust are crates, not source files).

The simple truth is a C compiler doesn’t need to do very much!

Unless you use sqlite, in which case your build takes a million years.

As an example, say your function takes anything that can be turned into a String. You'd write a generic wrapper that does the ToString step, then change the existing function to just take a String. That way when your function is called, only the thin outer function is monomorphised, and the bulk of the work is a single implementation.

It's not _that_ commonly known, as it only becomes a problem for a library that becomes popular.

Have you tried troubleshooting a compiler error in a unity build?

Yeah.

Could you expand on that, please? Every time you run dynmically linked program, it is linked at runtime. (unless it explicitly avoids linking unneccessary stuff by dlopening things lazily; which pretty much never happens). If it is fine to link on every program launch, linking at build time should not be a problem at all.

If you want to have link time optimization, that's another story. But you absolutely don't have to do that if you care about build speed.

If it improves compile time, that sounds like a bug in the compiler or the design of the language itself.

DLLs got their start when early windowing systems didn't quite fit on the workstations of the era in the late 80s / early 90s.

In about 4 minutes both Microsoft and GNU were like, "let me get this straight, it will never work on another system and I can silently change it whenever I want?" Debian went along because it gives distro maintainers degrees of freedom they like and don't bear the costs of.

Fast forward 30 years and Docker is too profitable a problem to fix by the simple expedient of calling a stable kernel ABI on anything, and don't even get me started on how penetrated everything but libressl and libsodium are. Protip: TLS is popular with the establishment because even Wireshark requires special settings and privileges for a user to see their own traffic, security patches my ass. eBPF is easier.

Dynamic linking moves control from users to vendors and governments at ruinous cost in performance, props up bloated industries like the cloud compute and Docker industrial complex, and should die in a fire.

Don't take my word for it, swing by cat-v.org sometimes and see what the authors of Unix have to say about it.

I'll save the rant about how rustc somehow manages to be slower than clang++ and clang-tidy combined for another day.

Compilation speed depends on what you do with a language. "Fast" is not an absolute, and for most people it depends heavily on community habits. Rust habits tend to favor extreme optimizability and/or extreme compile-time guarantees, and that's obviously going to be slower than simpler code.

https://www.youtube.com/watch?v=dOfucXtyEsU

…

Dynamic linking moves control from users to vendors and governments at ruinous cost in performance, props up bloated industries...

This is ridiculous. Not everything is a conspiracy!

  for i in 0..10 {}

  let mut iter = Range { start: 0, end: 10 }.into_iter();
  while let Some(i) = iter.next() {}

EDIT: i signed up to get access to unreal so take a look at how they do unity builds and turns out they have their own build tool (not CMake) that orchestrates the build. so does anyone know (can someone comment) whether unity builds for them (unreal) means literally one file for literally all project sources files or if it's "higher-granularity" like UNITY_BUILD in CMake (i.e., single file per object).

There are multiple caveats on providing this to users (we can't assume that macro invocations are idempotent, so the new behavior would have to be opt in, and this only benefits incremental compilation), but it's in our radar.

  fn foo<S: Into<String>>(s: S) {
      fn inner(s: String) { ... }
      inner(s.into())
  }

UE doesn't use a full unity build, it groups some files together into small "modules". I can see how this approach may have some benefits; you're trading off a faster clean build for a slower incremental build.

I tested compiling UnrealFrontend, and a default setup with the hybrid unity build took 148s. I noticed it was only using half my cores due to memory constraints. I disabled unity and upped the parallelism and got 182s, so 22% slower while still using less memory. A similarly configured unity build was 108s, so best case is ~2x.

On the other hand only changing the file TraceTools/SFilterPreset.cpp resulted in 10s compilation time under a unity build, and only 2s without unit.

I can see how this approach has its benefits (and drawbacks). But to be clear this isn't what projects like raddbg and sqlite3 are doing. They're doing a single translation unit for the entire project. No parallelism, no incremental builds, just a single compiler invocation. This is usually what I've seen people mean by a unity build.

> My experience has been the polar opposite of yours - similar size rust projects are an order of magnitude slower than C++ ones. Could you share an example of a project to compare with?

I just did a release build of egui in 35s, about the same as raddbg's release build. This includes compiling dependencies like wgpu, serde and about 290 other dependencies which add up to well over a million lines of code.

Note I do have mold configured as my linker, which speeds things up significantly.

Nah. Slow type checking in Swift is primarily caused by the fact that functions and operators can be overloaded on type.

Separately-compiled generics don't introduce any algorithmic complexity and are actually good for compile time, because you don't have to re-type check every template expansion more than once.

I suspect this leaks into both compile-time and run-time costs.

If fact, if there was anything remotely controversial about a bunch of extremely specific, extremely falsifiable claims I made, one imagines your rebuttal would have mentioned at least one.

I said inflmatory things (Docker is both arsonist and fireman at ruinous cost), but they're fucking true. That Alpine in the Docker jank? Links musl!

https://en.wikipedia.org/wiki/Advanced_persistent_threat

https://en.wikipedia.org/wiki/Operation_Olympic_Games

https://simple.wikipedia.org/wiki/Stuxnet

https://en.wikipedia.org/wiki/Cozy_Bear

https://en.wikipedia.org/wiki/Fancy_Bear

https://en.wikipedia.org/wiki/Tailored_Access_Operations

But people should make an informed choice, and there isn't any noble or high minded or well-meaning reason to try to shout that information down.

Don't confidently assert falsehoods unless you're prepared to have them refuted. You're entitled to peddle memes and I'm entitled to reply with corrections.

Rust is generally faster to compile on a per-sourceline basis than c++. But rust projects compile all their dependencies as well.

Go and Dlang compilers were designed by those that are really good at compiler design and that's why they're freaking fast. They designed the language around the compiler constraints and at the same managed to make the language intuitive to use. For examples, Dlang has no macro and no unnecessary symbols look-up for the ambiguous >>.

Because of these design decisions both Go and Dlang are anomaly for fast compilation. Dlang in particular is notably more powerful and expressive compared to C++ and Rust even with its unique hybrid GC and non-GC compilation.

In automotive industry it's considered a breakthrough and game changing achievement if you have a fast transmission for seamless auto and manual transmission such as found in the latest Koenigsegg hypercar [1]. In programming industry however, nobody seems to care. Walter Bright the designer of Dlang has background in mechanical engineering and it shows.

[1] Engage Shift System: Koenigsegg new hybrid manual and automatic gearbox in CC850:

https://www.topgear.com/car-news/supercars/heres-how-koenigs...

Not all programmers of course - if you look at std there are many places that split types into generic and non-generic parts so the compiler will reuse as much code as possible, but it does come at the cost of additional complexity. Worse if you aren't already aware of why they are doing it, the language does a marvellous job of hiding the reason that complexity is there. I'd wager a novice Rust programmer is as befuddled by it as a JavaScript programmer coming across his first free() call in C.

I have this dream of a language like Rust that makes the trade-off plain, so the programmer is always aware of "this is a zero cost abstraction - you're just making it plain via the type system your doing the right thing" and "I'm going to have to generate a lot of code for this". Then go a step further and put the types and source you want to export to other libraries in a special elf section in the .so so you don't need the source to link against it, then go another step further and make the programmer using the .so explicitly instantiate any stuff that does require a lot of code generated so he is aware of what is happening.

That said, I don't think it would help the compile time problem in most cases. C++ already does something close by forcing you to put exported stuff in .h files, and they ended up with huge .h files and slow compiles anyway.

Nevertheless doing that would make for a Rust like language, that, unlike Rust, supported an eco-system of precompiled libraries just like C does. Rust is so wedded to transparent monomorphisation it looks near impossible now.

Go got famous compile times, because for a decade a new generation educated in scripting languages and used to badly configured C and C++ projects, took for innovation, what was actually a return to old values in compiler development.

Unfortunately that doesn't seem to ever be a scenario cargo will support out of the box.

Same applies to point 3, when used alongside external templates.

Most likely treating C and C++ as scripting languages with header only libraries, only to complain about build times afterwards.

I think lazily linking is the default even if you don't use dlopen, i.e. every symbol gets linked upon first use. Of course that has the drawback, that the program can crash due to missing/incompatible libraries in the middle of work.

I’m the tech director for a game studio using unreal and we spec the dev machines with enough memory to avoid that pressure - we require 2.5GB/Core, rounded up. Right now we’re using i9 14900k’s with 128GB RAM. We were on 5950x with 64GB before this.

I tried egui last night but failed to build it due to missing dependencies, but I’ll see if I can get it running on Monday. I’d also love to see what raddbg would be like if we split it into even 4 compilation units

Anyway, while what you said is theoretically half-true, a fairly large number of libraries are not designed/encapsulated well. This means almost all of their symbols are exported dynamically, so, the idea that there are only "few public exported symbols" is unfortunately false.

However, something almost no one ever mentions is that ELF was actually designed to allow dynamic libraries to be fairly performant. It isn't something I would recommend, as it breaks many assumptions on Unices, (while you don't get the benefits of LTO) you can achieve code generation almost equivalent to static linking by using something like "-fno-semantic-interposition -Wl,-Bsymbolic,-z,now". MaskRay has a good explanation on it: https://maskray.me/blog/2021-05-16-elf-interposition-and-bsy...

Most certainly an unfair comparison; I did not know that "unity build" also referred to a hybrid setup. I've only seen people advocate for a single translation unit approach, and that's always just plain slow.

Personally I'll almost always prefer faster incremental compilation; thankfully my work's clean builds only take 30 seconds.

> I’d also love to see what raddbg would be like if we split it into even 4 compilation units

Should be a minimum of 32x faster just building in parallel with our hardware ;)

Unfortunately raddbg suffers from unity-build-syndrome: It has headers but they're missing definitions, and .c files are missing includes.

> Unfortunately raddbg suffers from unity-build-syndrome: It has headers but they're missing definitions, and .c files are missing includes.

We run a CI build once a week with unity disabled to catch these issues.

> Should be a minimum of 32x faster just building in parallel with our hardware ;)

My experience here is that parallelism is less than linear and avoiding repeated work is super-linear - there’s a happy medium. If you had 128 files and 32 cores it’s probably fast to compile in parallel, but likely even faster if you can just dispatch 32 “blobs” and link them at the end.

I belong to the school of thought that C style macros in C++ should be nuked.

Exception being source code I don't own.