From Early Encoding to Late Suppression: Interpreting LLMs on Character Counting Tasks — mentioned by nearbuy

Fri, 04 Sep 2026 04:03:14 GMT

People assume that's the reason because it's intuitive and "strawberry" is one token. But that doesn't explain why those models would also often get it wrong for "StRaWbErRy" or even "s-t-r-a-w-b-e-r-r-y", where the r's are not combined into one token. We don't know what was going on inside the closed source GPT models, but this paper investigated on some of the open-weight models and found it's not due to tokenization: https://arxiv.org/abs/2604.00778

Paper and discussion history. A mention is not an endorsement.

From Early Encoding to Late Suppression: Interpreting LLMs on Character Counting Tasks · HackerPapers discussions

From Early Encoding to Late Suppression: Interpreting LLMs on Character Counting Tasks — mentioned by nearbuy