HackerPapers

arXiv:2402.17764cs.CL

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits

Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, Jilong Xue, Furu Wei

Abstract

Recent research, such as BitNet, is paving the way for a new era of 1-bit Large Language Models (LLMs). In this work, we introduce a 1-bit LLM variant, namely BitNet b1.58, in which every single parameter (or weight) of the LLM is ternary {-1, 0, 1}. It matches the full-precision (i.e., FP16 or BF16) Transformer LLM with the same model size and training tokens in terms of both perplexity and end-task performance, while being significantly more cost-effective in terms of latency, memory, throughput, and energy consumption. More profoundly, the 1.58-bit LLM defines a new scaling law and recipe for training new generations of LLMs that are both high-performance and cost-effective. Furthermore, it enables a new computation paradigm and opens the door for designing specific hardware optimized for 1-bit LLMs.

arXiv comments
Work in progress
Journal reference
DOI
Subjects
cs.CLComputation and Language cs.LGMachine Learning

On Hacker News

Linked by 26 identifiable commenters in 26 comments across 21 threads, from Feb 2024 to Sep 2026. Found in 5 HN submissions from Feb 2024 to Jun 2024. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments21 threads

  1. Breaking the 1.58-bit Barrier for Ternary LLMs 1 comment

    • sounds like a perfect fit for ASIC-optimized models (where matrix ops could be supported directly in BITCOS format, potentially) & achieving record power efficiency for on-device inference. And it looks like per [0], a model needs only ~30% more weights to be at comparable…

      yalokView on HNParent context

  2. Neutrino-1 8B 1 comment

    • There's a really interesting trend of labs using "proprietary" methods to convert existing models to a compressed ternary format. PrismML actually targeted the same Qwen 8b model and got it down to 1.75gb here: https://prismml.com/news/ternary-bonsai I wonder how proprietary it…

      kamranjonView on HNParent context

  3. How to pack ternary numbers in 8-bit bytes 1 comment

    • Amusing that the link to an article on ternary numbers was posted by Mr Triplett. (: The article is well-written and illustrated. The technique described is used in llama.cpp for running language models like BitNet b1.58 whose weights are stored as ternary types. > ..in which…

      lioetersView on HNParent context

  4. 1-Bit Bonsai Image 4B Image Generation for Local Devices 1 comment

    • fair, i think i was referring more to 1.58 bit architecture in general since the original paper (Figure 3) shows that we eliminate FP16 multiplication and addition just for INT8 addition. I need to dive deeper into bonsai overall if it differs https://arxiv.org/pdf/2402.17764

      c0rruptbytesView on HNParent context

  5. BitNet: 100B Param 1-Bit model for local CPUs 1 comment
  6. I built P2P network where every CPU becomes an AI inference node 89 tks/s no GPU 1 comment

    • Hey HN, I've been working on ARIA Protocol — an open-source P2P network for distributed AI inference using 1-bit quantized models (ternary weights: -1, 0, +1). The key insight: multiplications become additions/subtractions, so any CPU can run LLMs efficiently without a GPU. Real…

      anthonymuView on HNParent context

  7. Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model 1 comment

    • Didn't this paper demonstrate that you only need 1.58 bits to be equivalent to 16 bits in performance? https://arxiv.org/abs/2402.17764

      codexonView on HNParent context

  8. Why can't transformers learn multiplication? 1 comment

    • > It doesn't show it can execute it at scale. Without writable state and bit exact ops, Well, modern LLM coding agent products (eg. Claude Code) are able to store state in files in the current repository. So, you could have the model keep the "CPU State", and the files in the…

      kovekView on HNParent context

  9. The Trinary Dream Endures 1 comment

    • Ternary quantized weights for LLMs are a thing. Most of the weights in Bitnet b1.58[1] class models[2][3] are ternary (-1/0/1). [1]: https://arxiv.org/abs/2402.17764 [2]: https://huggingface.co/tiiuae/Falcon-E-3B-Instruct [3]: https://huggingface.co/microsoft/bitnet-b1.58-2B-4T

      woadwarrior01View on HNParent context

  10. AMD's Freshly-Baked MI350: An Interview with the Chief Architect 1 comment

    • Will 1.58 bits be in the MI400? Or is it not established as a widely useful technology yet? See https://arxiv.org/abs/2402.17764

      kristianpView on HNParent context

  11. Microsoft researchers developed a hyper-efficient AI model that can run on CPUs 2 comments

    • Repo with demo video and benchmark: https://github.com/microsoft/BitNet "...It matches the full-precision (i.e., FP16 or BF16) Transformer LLM with the same model size and training tokens in terms of both perplexity and end-task performance, while being significantly more…

      hu3View on HNParent context

    • The original BitNet paper (https://arxiv.org/pdf/2310.11453) BitNet: Scaling 1-bit Transformers for Large Language Models was actually binary (weights of -1 or 1), but then in the follow-up paper they started using 1.58bit weights (https://arxiv.org/pdf/2402.17764) The Era of…

      AzN1337c0d3rView on HNParent context

  12. Run DeepSeek R1 Dynamic 1.58-bit 1 comment

    • That was just improper rounding from 1.58 to 1.5. They say 1.58 in other places and explicitly link to https://arxiv.org/abs/2402.17764

      cubefoxView on HNParent context

  13. Galois Theory 1 comment
  14. Researchers run high-performing LLM on the energy needed to power a lightbulb 1 comment

    • For reference, this is a real prototype for Bitnet 1.58b which uses (-1,0,1) as the weights which simplifies the matrix multiplication [0]. [0] https://arxiv.org/abs/2402.17764

      syntaxingView on HNParent context

  15. Scalable MatMul-Free Language Modeling 1 comment

    • The quantization approach is basically identical to the 1.58bit LLM paper: https://arxiv.org/abs/2402.17764 The main addition of the new paper seems to be the implementation of optimized and fused kernels using triton, as seen here…

      cpldcpuView on HNParent context

  16. AMD Unveils Ryzen 9000 CPUs for Desktop, Zen 5 1 comment

    • No problem, glad you found it interesting, I think it's pretty exciting stuff. This is a more recent paper by the same people, where they tweak slightly to ternary (-1, 0, 1), and they seem to be able to maintain network performance: https://arxiv.org/pdf/2402.17764

      ericdView on HNParent context

  17. GPUs Go Brrr 1 comment

    • > Then there's a trend towards much shorter numbers. 16 bit floating point? 8 bit? 2 bit? 1 bit? There was that recent paper titled "The Era of 1-bit LLMs" [0] which was actually suggeting a 1.58 bit LLM (2 bits in practice). > Someone reading this is probably writing it in VHDL…

      UncleOxidantView on HNParent context

  18. FPGA Architecture for Deep Learning: Survey and Future Directions 1 comment

    • The big challenge when it comes to using FPGAs for deep learning is pretty simple: all of that reprogrammability comes at a performance cost. If you're doing something highly specific that conventional GPUs are bad at, like genomics research [1] or high-frequency trading [2]…

      zachbeeView on HNParent context

  19. Towards 1-bit Machine Learning Models 1 comment

    • In both the '1-bit Model' and '2-bit Model' tables, the forward time (sec) for Llama2-7B with FP16 (full precision) is 0.1 s, whereas it's ~0.231, ~0.257, ~0.353 s respectively for HQQ (1-bit) / HQQ+ (1-bit) / Quip# (2-bit) meaning the FP16 model has ~3x lower inference time. On…

      jstmmView on HNParent context

  20. Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’ 1 comment

    • > - Training isn’t done at 4-bits, to date this small size has only been for inference. Wasn't there a paper from Microsoft two weeks ago or so where they trained on log₂(3) bits? Edit: https://arxiv.org/pdf/2402.17764.pdf

      tmalsburg2View on HNParent context

  21. The Era of 1-bit LLMs: ternary parameters for cost-effective computing 5 comments

    • Read the pdf https://arxiv.org/pdf/2402.17764.pdf they call it 1-bit everywhere. I don't know why do they do this, 1-bit seems to be a very wrong name for {-1, 0, 1}.

      bmachoView on HNParent context

    • +1 On this, the real proof would have been testing both models side-by-side. It seems that it may be published on GitHub [1] according to HuggingFace [2]. [1] https://github.com/microsoft/unilm/tree/master/bitnet [2] https://huggingface.co/papers/2402.17764

      bArrayView on HNParent context

    • You're both right, I skimmed the paper, saw large model numbers but didn't notice it was for speed. On the HF page they say those models are being trained. https://huggingface.co/papers/2402.17764 "We haven't finished the training of the models beyond 3B as it requires much much…

      imjonseView on HNParent context

    • Discussion on HF [1] implies that no, conversion is not helpful. It would take training the model from scratch. 1: https://huggingface.co/papers/2402.17764

      btbuildemView on HNParent context

    • From the BitNet paper: "Straight-through estimator. To train our 1-bit model, we employ the straight-through estimator (STE)[BLC13] to approximate the gradient during backpropagation. This method bypasses the nondifferentiable functions, such as the Sign (Eq. 2) and Clip (Eq. 5)…

      spyderView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.