A tail calling interpreter for Python (already landed in CPython)

This is a great summary. When Mike wrote the message you linked, his conclusion was that you have to drop to assembly to get reasonable code for VM interpreters. Later we developed the "musttail" technique which was able to match his assembly language sequences using C. This makes C a viable option for VM interpreters, even if you want best performance, as long as your compiler supports musttail.

> they are able to perform more/better optimizations when looking at normal-sized functions rather than massive blocks

It's not the size of the function that is the primary problem, it is the fully connected control flow that gums everything up. The register allocator is trying to dynamically allocate registers through each opcode's implementation, but it also has to connect the end of every opcode with the beginning of every opcode, from a register allocation perspective.

The compiler doesn't understand that every opcode has basically the same set of "hot" variables, which means we benefit from keeping those hot variables in a fixed set of registers basically all of the time.

With tail calls, we can communicate a fixed register allocation to the compiler through the use of function arguments, which are always passed in registers. When we pass this hot data in function arguments, we force the compiler to respect this fixed register allocation, at least at the beginning and the end of each opcode. Given that constraint, the compiler will usually do a pretty good job of maintaining that register allocation through the entire function.

thanks for the explanation!

I feel like using calling conventions to massage the compiler's register allocation strategy is a hack. If the problem is manual control over register allocation, then the ideal solution should be... well, exactly that and no more? An annotation for local variables indicating "always spill this" (for cold-path locals) or "never spill this or else trigger a build error" (for hot-path locals). Isn't that literally why the "register" keyword exists in C? Why don't today's C compilers actually use it?

If the tail calling pattern made the code ugly, I would be more inclined to agree with this. But putting each opcode in its own function isn't so bad: it seems just as readable, if not more so, than a mondo function that implements every opcode.

By contrast, a mondo function that also has a bunch of register allocation annotations seems less readable.

I don't see how a hypothetical __attribute__((never_spill)) annotation on local variables would preclude splitting opcode logic into separate functions. It just means those functions would have to be inlined into the interpreter loop to avoid conflicts with calling convention constraints.