Microscaling Data Formats for Deep Learning · HackerPapers discussions

Microscaling Data Formats for Deep Learning — mentioned by buildbot

Tue, 07 Apr 2026 15:22:56 GMT

Super cool to tape out a nearly 500MHz systolic array!!! I wish sometimes I had gone far enough with my hardware education to be able to do more than simple FOGA designs. A really cool thing to see would be the newer block scaled fp4/fp8 data types. Maybe for their next asic they can try it haha - https://arxiv.org/abs/2310.10537

Paper and discussion history. A mention is not an endorsement.

Microscaling Data Formats for Deep Learning — mentioned by buildbot

Wed, 25 Mar 2026 18:06:52 GMT

You can even train in 4 & 8 bits with newer microscaled formats! From https://arxiv.org/pdf/2310.10537 to gpt-oss being trained (partially) natively in MXFP4 - https://huggingface.co/blog/RakshitAralimatti/learn-ai-with-... To Nemotron 3 Super, which had 25T of nvfp4 native pretraining! https://docs.nvidia.com/nemotron/0.1.0/nemotron/super3/pretr...

Paper and discussion history. A mention is not an endorsement.

Microscaling Data Formats for Deep Learning — mentioned by opcode84

Tue, 19 Mar 2024 01:39:10 GMT

https://arxiv.org/pdf/2310.10537.pdf Discussed in a previous post https://news.ycombinator.com/item?id=37930663

Paper and discussion history. A mention is not an endorsement.

Microscaling Data Formats for Deep Learning — mentioned by buildbot

Sun, 10 Mar 2024 16:22:04 GMT

FP19 is perfectly fine for training - llama is trained in fp16, many train in bf16, with microscaling exponents, you can do 6 bit training : https://arxiv.org/abs/2310.10537

Paper and discussion history. A mention is not an endorsement.

Microscaling Data Formats for Deep Learning — mentioned by buildbot

Wed, 06 Dec 2023 18:30:14 GMT

Even below FP8 works: https://arxiv.org/abs/2310.10537 (6 bit and lower)

Paper and discussion history. A mention is not an endorsement.

Microscaling Data Formats for Deep Learning — mentioned by opcode84

Wed, 15 Nov 2023 19:24:46 GMT

The Microsoft chip has MX data types: https://news.ycombinator.com/item?id=37930663 https://arxiv.org/abs/2310.10537

Paper and discussion history. A mention is not an endorsement.

Microscaling Data Formats for Deep Learning — mentioned by opcode84

Wed, 18 Oct 2023 16:04:19 GMT

Earlier this year, AMD, Arm, Intel, Meta, Microsoft, NVIDIA, and Qualcomm Technologies, Inc. formed the Microscaling Formats (MX) Alliance with the goal of creating and standardizing next-generation 6- and 4-bit data types for AI training and inferencing. The key enabling technology that enables sub 8-bit formats to work, referred to as microscaling, builds on a foundation of years of design space exploration and research. MX enhances the robustness and ease-of-use of existing 8-bit formats such as FP8 and INT8, thus lowering the barrier for broader adoption of single digit bit training and…

Paper and discussion history. A mention is not an endorsement.