https://hackerpaper.kaipfister.de/p/2105.03536/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=35148832urn:hackerpapers:mention:2105.03536:35148832Tue, 14 Mar 2023 08:51:13 GMT<p>This is really interesting, thank you for the reference! Having worked more with images based NN than language models before, I wonder: are LLM inherently more suited to aggressive quantisation, due to their very large size? I see people suggesting here 4b is pretty good, and 3b should be the target. I remember ResNets etc can of course also be quantized, and up to 8-6b you get pretty good results with very little effort, with low-ish degradation in performance. Trying to go down to 4b is more challenging, though this paper claims with quantisation aware training 4b is possible indeed, but…</p><p><a href="https://hackerpaper.kaipfister.de/p/2105.03536/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LG