RADLADS: Rapid Attention Distillation to Linear Attention Decoders at Scale · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2505.03005/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTRADLADS: Rapid Attention Distillation to Linear Attention Decoders at Scale — mentioned by killerstormhttps://news.ycombinator.com/item?id=43972401urn:hackerpapers:mention:2505.03005:43972401Tue, 13 May 2025 12:54:17 GMT<p>Another paper related to attention distillation, although doing something far more radical: transformer attention is distilled onto RWKV-like model: https://huggingface.co/papers/2505.03005</p><p><a href="https://hackerpaper.kaipfister.de/p/2505.03005/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CL