Revisiting Self-Distillation · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2206.08491/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTRevisiting Self-Distillation — mentioned by phreezahttps://news.ycombinator.com/item?id=44682567urn:hackerpapers:mention:2206.08491:44682567Fri, 25 Jul 2025 12:51:40 GMT<p>One mind-bending thing is that self-distillation, meaning distilling one model into another of the same architecture, number of parameters, etc., also often works! https://arxiv.org/abs/2206.08491</p><p><a href="https://hackerpaper.kaipfister.de/p/2206.08491/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LG