Reliability Scaling Laws for Quantized Large Language Models — mentioned by armcat

Sun, 08 Nov 2026 13:21:59 GMT

Something to keep in mind with aggressive quantisation, is that it's been empirically proven [1] that (1) quantisation has non-linear performance behaviour, and (2) generally the peak size-to-performance ratio occurs at 4-bit; anything lower and you will see significant reduction in performance. Also, I've been playing with speech models for the past 2 years, and for me anything below 8-bit is a huge drop in quality; so quantisation for LLMs and for other bespoke modality models, is not 1:1. [1] "Reliability Scaling Laws for Quantized Large Language Models", TUM…

Paper and discussion history. A mention is not an endorsement.

Reliability Scaling Laws for Quantized Large Language Models · HackerPapers discussions

Reliability Scaling Laws for Quantized Large Language Models — mentioned by armcat