A surprising enum size optimization in the Rust compiler

I don't think it's actually "flattening" the enums discriminant spaces (though maybe it is). My guess is this is just 32-bit alignment requirement (ie. bytes 1:4 are padding) + sub-byte discriminant sizes. The surprising thing here is that the ordering of Outer's variants doesn't match the ordering as defined, instead having variant D's discriminant be 0 (0 << size_of::<Inner's discriminant>). size_of::<Inner> is actually 33 bits and size_of::<Outer> is 34 bits and then you just apply alignment requirements to each field. Actual size_of calls will account for alignment and padding, but the compiler knows the truth.

What's cool about this in general is nested match statements can be flattened into a jumplist (idk if rustc does this, but it's possible in theory).

> I don't think it's actually "flattening" the enums discriminant spaces (though maybe it is).

It is. One easy way to see this is with an Option<Option<bool>> [0]: if both options are Some, it takes the value 0 or 1 depending on the boolean; if the inner Option is None, it takes the value 2; and if the outer Option is None, it takes the value 3. And as you keep adding more Options, they take values 4, 5, 6, etc. to represent None, while still only taking up 1 byte.

[0] https://play.rust-lang.org/?version=stable&mode=debug&editio...

I think that's just niche optimization. If you change from bool to a u8 it doesn't use the invalid bit pattern as a discriminant even though it could: https://play.rust-lang.org/?version=stable&mode=debug&editio...

But there's no invalid u8 bit pattern.

But there is for Option<u8> which is the entirety of the u8s bits if the Option is None.