Can your terminal do emojis? How big?

In my fever dreams of maintaining utf8 supporting text widgets that work and never need to be updated, there's a zero-width whitespace grapheme cluster that represents the number of codepoints in the next grapheme cluster if they're different from the previous.

The situation today is basically the same as null terminated C strings. Except worse, because you can define that problem and solve it in linear time/space without needing to keep an up to date list of tables.

Interesting! Alternatively you could encode that in variation selectors on a zero width space: https://paulbutler.org/2025/smuggling-arbitrary-data-through...

This has nothing to do with UTF-8 which doesn't and shouldn't care about anything beyond mapping bytes to code points.

But even for adding it to Unicode, your proposal would make text stateful (even over long distances) which is a really bad idea.

Combining characters have already made Unicode text stateful.

Although I agree that encoding length hints into it seems like a bad idea - it creates an opportunity for the encoding to disagree with the reality of the text. You need _some_ way of handling it if it says that the next grapheme cluster is 4 characters long but it's actually only three.

Combining characters and joiners should have been prefix rather than suffix/infix operators (and preferably in blocks by arity) so you'd always know without lookahead whether a grapheme cluster was complete.

(Prefix combining accents would also have made dead keys trivial rather than painful.)

It's already stateful