RNNs can generate bounded hierarchical languages with optimal memory · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2010.07515/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTRNNs can generate bounded hierarchical languages with optimal memory — mentioned by canjobearhttps://news.ycombinator.com/item?id=35830545urn:hackerpapers:mention:2010.07515:35830545Fri, 05 May 2023 15:31:34 GMT<p>An RNN with infinite-precision rational weights can implement a counter with a bounded number of neurons. With finite precision the number of neurons scales linearly in your desired depth of matching brackets: https://arxiv.org/pdf/2010.07515.pdf For a Transformer you need to add a new layer of self-attention to increase bracket matching depth by 1. https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00306... Transformers are formally a lot weaker than RNNs/LSTMs.</p><p><a href="https://hackerpaper.kaipfister.de/p/2010.07515/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CL