Adaptive Attention Span in Transformers · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/1905.07799/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTAdaptive Attention Span in Transformers — mentioned by voxgenhttps://news.ycombinator.com/item?id=35280802urn:hackerpapers:mention:1905.07799:35280802Thu, 23 Mar 2023 20:23:21 GMT<p>One can also make a model to learn the necessary context length for each layer and head to save a huge amount of FLOPs: https://arxiv.org/abs/1905.07799</p><p><a href="https://hackerpaper.kaipfister.de/p/1905.07799/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LG