https://hackerpaper.kaipfister.de/p/2106.01506/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=49088467urn:hackerpapers:mention:2106.01506:49088467Tue, 28 Jul 2026 19:07:18 GMT<p>Overall, all the different linear attentions out there are approximations the original (quadratic) attention and this is important for the whole &quot;AI&quot; enterprise[2]. Original attention involves (very crudely) an approach of scanning how every token (roughly a word) relates every other token and training a classic neural network on related tokens - to get either language translation or next word prediction (and next word prediction is what &quot;seems intelligent&quot; in LLMs). [1] The problem is that since original attention is &quot;everything to everything else&quot; it scales quadratically (O(n^2)) with the…</p><p><a href="https://hackerpaper.kaipfister.de/p/2106.01506/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LG