Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling — mentioned by augment_me

Sun, 08 Nov 2026 17:28:44 GMT

This is not fully true. 4 over 6 paper(https://arxiv.org/pdf/2512.02010) for example showed that the initial and last layers of transformers that expand into the high-dimensional vocabulary are hurt much more than the center ones. These layers are kept in fp16 and are hurt much more than others by quantization. If you think of it from an informational perspective, you can imagine that text tokens are much more high-dimensional than a transformed/down-projected variant. With this reasoning the central layers are mostly building onto the residual stream, and can do fine with 1b and 2b weights.

Paper and discussion history. A mention is not an endorsement.

Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling · HackerPapers discussions

Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling — mentioned by augment_me