https://hackerpaper.kaipfister.de/p/2407.15160/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=42572588urn:hackerpapers:mention:2407.15160:42572588Thu, 02 Jan 2025 07:50:47 GMT<p>When Can Transformers Count to n? https://arxiv.org/html/2407.15160v2 The Expressive Power of Transformers with Chain of Thought https://arxiv.org/html/2310.07923v5 Transformer needs to retrieve letters per each token while forced to keep internal representation still aligned in length with the base tokens (each token also has finite embedding, while made out of multiple letters), and then it needs to count the letters within misaligned representation. Autoregressive mode completely alleviate the problem as it can align its internal representation with the letters and it can just keep…</p><p><a href="https://hackerpaper.kaipfister.de/p/2407.15160/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CL