https://hackerpaper.kaipfister.de/p/2411.12537/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=42219673urn:hackerpapers:mention:2411.12537:42219673Sat, 23 Nov 2024 07:34:22 GMT<p>Abstract: Linear Recurrent Neural Networks (LRNNs) such as Mamba, RWKV, GLA, mLSTM, and DeltaNet have emerged as efficient alternatives to Transformers in large language modeling, offering linear scaling with sequence length and improved training efficiency. However, LRNNs struggle to perform state-tracking which may impair performance in tasks such as code evaluation or tracking a chess game. Even parity, the simplest state-tracking task, which non-linear RNNs like LSTM handle effectively, cannot be solved by current LRNNs. Recently, Sarrof et al. (2024) demonstrated that the failure of…</p><p><a href="https://hackerpaper.kaipfister.de/p/2411.12537/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LGhttps://news.ycombinator.com/item?id=42219672urn:hackerpapers:submission:2411.12537:42219672Sat, 23 Nov 2024 07:34:21 GMT<p>Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues</p><p></p><p>2 points; 1 discussion reply on HN. <a href="https://hackerpaper.kaipfister.de/p/2411.12537/">Paper and source history</a>. A submission is not an endorsement.</p>cs.LG