Grokking modular arithmetic — mentioned by p-e-w

Mon, 19 Jan 2026 12:47:45 GMT

> The Turkey was an LLM. It predicted the future based entirely on the distribution of the past. It had no "understanding" of the purpose of the farmer. But we already know that LLMs can do much better than that. See the famous “grokking” paper[1], which demonstrates that with sufficient training, a transformer can learn a deep generalization of its training data that isn’t just a probabilistic interpolation or extrapolation from previous inputs. Many of the supposed “fundamental limitations” of LLMs have already been disproven in research. And this is a standard transformer architecture; it…

Paper and discussion history. A mention is not an endorsement.

Grokking modular arithmetic · HackerPapers discussions

Grokking modular arithmetic — mentioned by p-e-w