https://hackerpaper.kaipfister.de/p/1606.06996/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=40588596urn:hackerpapers:mention:1606.06996:40588596Wed, 05 Jun 2024 18:33:14 GMT<p>Maybe 10 bits is the average over the dictionary – which is what matters here, but over normal text it is significantly less. Our best current estimation for relatively high-level text (texts published by the EU) is 6 bits per word[1]. However, as our methods of predicting text improve, this number is revised down. LLMs ought to have made a serious dent in it, but I haven&#x27;t looked up any newer results. Anyway, all of this to say is that which words are chosen matters, but how they are put together matters perhaps more. [1]: http://arxiv.org/pdf/1606.06996</p><p><a href="https://hackerpaper.kaipfister.de/p/1606.06996/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CLhttps://news.ycombinator.com/item?id=35814876urn:hackerpapers:submission:1606.06996:35814876Thu, 04 May 2023 13:22:36 GMT<p>The Word Entropy of Natural Languages (2016)</p><p></p><p>2 points; 0 discussion replies on HN. <a href="https://hackerpaper.kaipfister.de/p/1606.06996/">Paper and source history</a>. A submission is not an endorsement.</p>cs.CL