SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models — mentioned by kingstnap

Wed, 25 Mar 2026 15:56:18 GMT

Other people have answered here but the real answer is that deep neural networks don't learn isotropic distributions of activations. What happens is that you get very spikey activations, there are so called "outlier" activations. A easy to read paper that tells you about this is SmoothQuant [0]. Another source from Anthropic and the Mechanistic Interperability people is calling these "privileged basis" [1]. Now based on the weight symmetries of a typical transformer, these actually don't need to exist. Weight symmetries means the ways you can change the weights without actually affecting the…

Paper and discussion history. A mention is not an endorsement.

SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models · HackerPapers discussions

SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models — mentioned by kingstnap