Rodimus*: Breaking the Accuracy-Efficiency Trade-Off with Efficient Attentions · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2410.06577/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTRodimus*: Breaking the Accuracy-Efficiency Trade-Off with Efficient Attentions — mentioned by juank10https://news.ycombinator.com/item?id=44296651urn:hackerpapers:mention:2410.06577:44296651Tue, 17 Jun 2025 07:48:38 GMT<p>Funnily enough, the code was deleted in the repo, but can still be seen in the commits. It&#x27;s what you would expect from the paper :D On the general topic of non-attention LLMs, I recommend checking out the MesaNet [1], Rodimus [2], Gated DeltaNet [3], or Mamba2 [4]. They are currently SOTA. However, I have yet to see a compelling non attention based model that achieves good performance on code, math, reasoning, or multi-turn QA tasks. I do not think we are getting rid of attention soon, I believe the ability to look back is crucial in certain tasks. [1] https://arxiv.org/abs/2506.05233 [2]…</p><p><a href="https://hackerpaper.kaipfister.de/p/2410.06577/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CL