The bitter lesson is coming for tokenization

‘Bytes’ is tokenization.

There’s no reason to assume it’s the best solution. It might be the case that a better tokenization scheme is needed for math, reasoning, video, etc models.