How Much Knowledge Can You Pack Into the Parameters of a Language Model? · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2002.08910/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTHow Much Knowledge Can You Pack Into the Parameters of a Language Model? — mentioned by patrick-lewishttps://news.ycombinator.com/item?id=24259807urn:hackerpapers:mention:2002.08910:24259807Mon, 24 Aug 2020 12:07:37 GMT<p>Hi, first author here. &gt; Given that nearest-neighbor outperforms on closed book, is it reasonable to suspect the model is doing NN itself internally (which would explain the good performance on close duplicates?) I think this is definitely the case for the BART model. It is essentially acting as a QA-pair memorizer over the training data, and at test time, it just matches the question onto those seen at training time. Note that the T5-11B+SSM closed-book model was able to do a little better on NQ, so very large models with task-specific pretraining objectives do seem to do something slightly…</p><p><a href="https://hackerpaper.kaipfister.de/p/2002.08910/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CL