https://hackerpaper.kaipfister.de/p/2311.03099/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=39952826urn:hackerpapers:submission:2311.03099:39952826Sat, 06 Apr 2024 14:39:05 GMT<p>Language models are Super Mario: Absorbing abilities from homologous models</p><p></p><p>105 points; 70 discussion replies on HN. <a href="https://hackerpaper.kaipfister.de/p/2311.03099/">Paper and source history</a>. A submission is not an endorsement.</p>cs.CLhttps://news.ycombinator.com/item?id=38716423urn:hackerpapers:mention:2311.03099:38716423Thu, 21 Dec 2023 02:43:38 GMT<p>This is not so surprising if you consider the fact that finetuning is extremely sparse and barely imparts any new knowledge to the model. The paper &quot;Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch&quot;[1] made this clear: &gt; We initially demonstrate that SFT LM (either encoder- or decoder-based) always tends to acquire excessively redundant delta parameters. To be specific, we present DARE, which randomly resets some delta parameters to zeros based on a drop rate p and subsequently scales the remaining parameters by a factor of 1/(1 − p). Despite its…</p><p><a href="https://hackerpaper.kaipfister.de/p/2311.03099/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CL