Teacher-Student Architecture for Knowledge Learning: A Survey · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2210.17332/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTTeacher-Student Architecture for Knowledge Learning: A Survey — mentioned by PrayagBhakarhttps://news.ycombinator.com/item?id=39039104urn:hackerpapers:mention:2210.17332:39039104Thu, 18 Jan 2024 07:47:24 GMT<p>If we’re talking about model distillation[0] I don’t think the student can ever be better than the teacher as optimising for speed and smaller model sizes inherently means that there will be precision loss. Even if the student is as big as the teacher, there is still data loss. [0] https://arxiv.org/pdf/2210.17332.pdf</p><p><a href="https://hackerpaper.kaipfister.de/p/2210.17332/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LG