https://hackerpaper.kaipfister.de/p/2507.06378/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=47830894urn:hackerpapers:mention:2507.06378:47830894Mon, 20 Apr 2026 06:04:35 GMT<p>There is currently very little evidence that morphological tokenizers help model performance [1]. For languages like German (where words get glued together) there is a bit more evidence (eg a paper I worked on [2]), but overall I start to suspect the bitter lesson is also true for tokenization. [1] https://arxiv.org/pdf/2507.06378 [2] https://pieter.ai/bpe-knockout/</p><p><a href="https://hackerpaper.kaipfister.de/p/2507.06378/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CL