https://hackerpaper.kaipfister.de/p/1605.02226/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=40612735urn:hackerpapers:mention:1605.02226:40612735Fri, 07 Jun 2024 20:39:20 GMT<p>People also often forget &quot;orderless autoregression&quot;, which was introduced a while back and has been reinvented many times since. See Sec 4 (pg 8) of &quot;Neural Autoregressive Distribution Estimation&quot; [https://arxiv.org/abs/1605.02226]. The main difference from current work is that this 2016 paper used MLPs and convnets on fixed-length observations/sequences, so sequence position is matched one-to-one with position in the network&#x27;s output, rather than conditioning on a position embedding. Of course, Transformers make this type of orderless autoregression more practical for a variety of reasons…</p><p><a href="https://hackerpaper.kaipfister.de/p/1605.02226/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LG