Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2104.06644/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTMasked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little — HN submissionhttps://news.ycombinator.com/item?id=26827693urn:hackerpapers:submission:2104.06644:26827693Fri, 16 Apr 2021 00:03:26 GMT<p>Bert pre-training appears to do well even with shuffled words</p><p></p><p>4 points; 0 discussion replies on HN. <a href="https://hackerpaper.kaipfister.de/p/2104.06644/">Paper and source history</a>. A submission is not an endorsement.</p>cs.CL