Language Model Tokenizers Introduce Unfairness Between Languages · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2305.15425/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTLanguage Model Tokenizers Introduce Unfairness Between Languages — HN submissionhttps://news.ycombinator.com/item?id=49650798urn:hackerpapers:submission:2305.15425:49650798Thu, 10 Sep 2026 22:09:54 GMT<p>Language Model Tokenizers Introduce Unfairness Between Languages (2023)</p><p></p><p>2 points; 0 discussion replies on HN. <a href="https://hackerpaper.kaipfister.de/p/2305.15425/">Paper and source history</a>. A submission is not an endorsement.</p>cs.CLLanguage Model Tokenizers Introduce Unfairness Between Languages — mentioned by fnordpiglethttps://news.ycombinator.com/item?id=36981894urn:hackerpapers:mention:2305.15425:36981894Thu, 03 Aug 2023 05:50:26 GMT<p>Given the performance of tokenizers by language the answer must be “worse,” there are a variety of sources for this .. they typically focus on cost, but it’s also borne out in quality. https://denyslinkov.medium.com/why-is-gpt-3-15-77x-more-expe... https://arxiv.org/pdf/2305.13707 https://arxiv.org/pdf/2305.15425 A quick Kagi can find a lot more.</p><p><a href="https://hackerpaper.kaipfister.de/p/2305.15425/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CL