3× rare in "strawberry"
Numbers instead of letters
A language model computes with numbers, not text. So every text is first cut into pieces, and each piece gets a fixed number. A piece can be a whole word or just part of one. The model doesn't see the individual letters directly.
What a study found
In 2024 a team led by Tairan Fu tested many models on many words. Result: the models recognize the letters but are bad at counting them. They are wrong most often when a letter appears more than once. How common a word was in training hardly mattered.
_single2.jpg){kind=link}