How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2305.00586/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTHow does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model — mentioned by theptiphttps://news.ycombinator.com/item?id=36778528urn:hackerpapers:mention:2305.00586:36778528Tue, 18 Jul 2023 20:19:01 GMT<p>&gt; GPT-4 is not a pure LLM I’ve looked a bit into the GPT architecture and haven’t seen anything suggesting it’s doing special-case experts for maths. It has MoE over 16 language models, and an image modality bolted on. If you have any evidence that there is a separate trained logic/math model I’d love to see that, as it would be interesting. (I don’t recall reading anything like that in the GPT papers for example, and this seems to claim there is no “calculator” hooked up in GPT-4 https://ai.stackexchange.com/a/40090). &gt; To put it another way: a (pure) language model has no way of encoding…</p><p><a href="https://hackerpaper.kaipfister.de/p/2305.00586/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CL