Kapathy's points are correct (of course).
One thing I like about text tokens though is that it learns some understanding of the text input method (particularly the QWERTY keyboard).
"Hello" and "Hwllo" are closer in semantic space than you'd think because "w" and "e" are next to each other.
This is much easier to see in hand coded spelling models, where you can get better results by including a "keybaord distance" metric along with a string distance metric.
replies(2):