Tropical Attention: Neural Algorithmic Reasoning for Combinatorial Algorithms · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2505.17190/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTTropical Attention: Neural Algorithmic Reasoning for Combinatorial Algorithms — mentioned by kingstnaphttps://news.ycombinator.com/item?id=45698420urn:hackerpapers:mention:2505.17190:45698420Fri, 24 Oct 2025 19:48:46 GMT<p>There are many many problems with attention. The softmax has issues regarding attention sinks [1]. The softmax also causes sharpness problems [2]. In general this decision boundary being Euclidean dot products isn&#x27;t actually optimal for everything, there are many classes of problem where you want polyhedral cones [3]. Positional embedding are also janky af and so is rope tbh, I think Cannon layers are a more promising alternative for horizontal alignment [4]. I still think there is plenty of room to improve these things. But a lot of focus right now is unfortunately being spent on…</p><p><a href="https://hackerpaper.kaipfister.de/p/2505.17190/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LG