A Theoretical Analysis of the Repetition Problem in Text Generation · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2012.14660/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTA Theoretical Analysis of the Repetition Problem in Text Generation — mentioned by andreykhttps://news.ycombinator.com/item?id=29387740urn:hackerpapers:mention:2012.14660:29387740Tue, 30 Nov 2021 01:38:42 GMT<p>This is a common problem with language models in general, and the reason is not that well understood. This paper A Theoretical Analysis of the Repetition Problem in Text Generation (https://arxiv.org/abs/2012.14660) seems to offer a principled answer. Basically the probability maximizing search procedure for text generation can get stuck in loops where the most likely next statement is similar or same to before. I&#x27;m no NLP researcher so I don&#x27;t have easy intuition on it, but that paper seems like a good read.</p><p><a href="https://hackerpaper.kaipfister.de/p/2012.14660/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CL