The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits · HackerPapers discussions

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by yalok

Thu, 17 Sep 2026 00:16:52 GMT

sounds like a perfect fit for ASIC-optimized models (where matrix ops could be supported directly in BITCOS format, potentially) & achieving record power efficiency for on-device inference. And it looks like per [0], a model needs only ~30% more weights to be at comparable quality, if quantization-aware training is done... 0. https://arxiv.org/pdf/2402.17764 - The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by kamranjon

Tue, 28 Jul 2026 05:53:25 GMT

There's a really interesting trend of labs using "proprietary" methods to convert existing models to a compressed ternary format. PrismML actually targeted the same Qwen 8b model and got it down to 1.75gb here: https://prismml.com/news/ternary-bonsai I wonder how proprietary it all is though, since the BitNet b1.58 paper has been out for a couple years now: https://arxiv.org/abs/2402.17764 From the wikipedia on 1.58 bit llms: "BitNet derives its performance from being trained natively in 1.58 bit instead of being quantized from a full-precision model after training. Still, training is an…

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by lioeters

Tue, 21 Jul 2026 12:25:37 GMT

Amusing that the link to an article on ternary numbers was posted by Mr Triplett. (: The article is well-written and illustrated. The technique described is used in llama.cpp for running language models like BitNet b1.58 whose weights are stored as ternary types. > ..in which every single parameter (or weight) of the LLM is ternary {-1, 0, 1} > significantly more cost-effective in terms of latency, memory, throughput, and energy consumption The original paper on this technique was published in Feb 2024. (Also linked from the article) The Era of 1-bit LLMs: All Large Language Models are in…

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by c0rruptbytes

Sun, 31 May 2026 18:59:42 GMT

fair, i think i was referring more to 1.58 bit architecture in general since the original paper (Figure 3) shows that we eliminate FP16 multiplication and addition just for INT8 addition. I need to dive deeper into bonsai overall if it differs https://arxiv.org/pdf/2402.17764

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by throwa356262

Wed, 11 Mar 2026 18:52:08 GMT

The paper has performance comparisons towards the end. https://arxiv.org/abs/2402.17764

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by anthonymu

Wed, 11 Feb 2026 21:03:35 GMT

Hey HN, I've been working on ARIA Protocol — an open-source P2P network for distributed AI inference using 1-bit quantized models (ternary weights: -1, 0, +1). The key insight: multiplications become additions/subtractions, so any CPU can run LLMs efficiently without a GPU. Real benchmarks (AMD Ryzen 9 7845HX, 8 threads): 0.7B model: 89.65 tokens/s — ~11 mJ/token 2.4B model: 36.94 tokens/s — ~27 mJ/token 8.0B model: 15.03 tokens/s — ~66 mJ/token Memory: 10x reduction (2B model: 4.0 GB → 0.4 GB) Based on Microsoft Research's BitNet b1.58 (arXiv:2402.17764) and bitnet.cpp. These are…

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by codexon

Tue, 27 Jan 2026 21:55:15 GMT

Didn't this paper demonstrate that you only need 1.58 bits to be equivalent to 16 bits in performance? https://arxiv.org/abs/2402.17764

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by kovek

Sat, 25 Oct 2025 08:01:16 GMT

> It doesn't show it can execute it at scale. Without writable state and bit exact ops, Well, modern LLM coding agent products (eg. Claude Code) are able to store state in files in the current repository. So, you could have the model keep the "CPU State", and the files in the repository be the "RAM". Also, could this https://arxiv.org/html/2402.17764v1 possibly reduce errors when doing inference? There is no floating point operations

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by woadwarrior01

Sun, 19 Oct 2025 20:33:20 GMT

Ternary quantized weights for LLMs are a thing. Most of the weights in Bitnet b1.58[1] class models[2][3] are ternary (-1/0/1). [1]: https://arxiv.org/abs/2402.17764 [2]: https://huggingface.co/tiiuae/Falcon-E-3B-Instruct [3]: https://huggingface.co/microsoft/bitnet-b1.58-2B-4T

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by kristianp

Sat, 21 Jun 2025 00:09:12 GMT

Will 1.58 bits be in the MI400? Or is it not established as a widely useful technology yet? See https://arxiv.org/abs/2402.17764

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by AzN1337c0d3r

Thu, 17 Apr 2025 01:17:02 GMT

The original BitNet paper (https://arxiv.org/pdf/2310.11453) BitNet: Scaling 1-bit Transformers for Large Language Models was actually binary (weights of -1 or 1), but then in the follow-up paper they started using 1.58bit weights (https://arxiv.org/pdf/2402.17764) The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits This seems to be first source of the confounding of "1-bit LLM" and ternary weights that I could find. In this work, we introduce a 1-bit LLM variant, namely BitNet b1.58, in which every single parameter (or weight) of the LLM is ternary {-1, 0, 1}.

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by hu3

Wed, 16 Apr 2025 23:16:28 GMT

Repo with demo video and benchmark: https://github.com/microsoft/BitNet "...It matches the full-precision (i.e., FP16 or BF16) Transformer LLM with the same model size and training tokens in terms of both perplexity and end-task performance, while being significantly more cost-effective in terms of latency, memory, throughput, and energy consumption..." https://arxiv.org/abs/2402.17764

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by cubefox

Tue, 28 Jan 2025 13:19:28 GMT

That was just improper rounding from 1.58 to 1.5. They say 1.58 in other places and explicitly link to https://arxiv.org/abs/2402.17764

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by klyrs

Thu, 15 Aug 2024 20:13:32 GMT

Somebody did a paper on GF(3)... https://arxiv.org/abs/2402.17764

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by syntaxing

Tue, 25 Jun 2024 13:32:06 GMT

For reference, this is a real prototype for Bitnet 1.58b which uses (-1,0,1) as the weights which simplifies the matrix multiplication [0]. [0] https://arxiv.org/abs/2402.17764

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by cpldcpu

Sun, 09 Jun 2024 15:41:44 GMT

The quantization approach is basically identical to the 1.58bit LLM paper: https://arxiv.org/abs/2402.17764 The main addition of the new paper seems to be the implementation of optimized and fused kernels using triton, as seen here: https://github.com/ridgerchu/matmulfreellm/blob/master/mmfre... This is quite useful, as this should make training this type of LLMs much more efficient. So this is a ternary weight LLM using quantization aware training (QAT). The activations are quantized to 8 bits. The matmal is still there, but it is multiplying the 8 bit activations by one bit values.…

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — HN submission

Sat, 08 Jun 2024 20:08:45 GMT

Show HN: A Small BitNet GPT in MLX

For those of us nerds who like to tinker with language models, I thought of sharing an example ipynb that shows how to build a small gpt (kudos to @karpathy for teaching us minGPT) where you replace the nn.Linear layer with a BitLinear layers as introduced in the paper on 1.58 bit LLMs https://arxiv.org/abs/2402.17764

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by ericd

Wed, 05 Jun 2024 00:21:18 GMT

No problem, glad you found it interesting, I think it's pretty exciting stuff. This is a more recent paper by the same people, where they tweak slightly to ternary (-1, 0, 1), and they seem to be able to maintain network performance: https://arxiv.org/pdf/2402.17764

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by UncleOxidant

Mon, 13 May 2024 17:30:47 GMT

> Then there's a trend towards much shorter numbers. 16 bit floating point? 8 bit? 2 bit? 1 bit? There was that recent paper titled "The Era of 1-bit LLMs" [0] which was actually suggeting a 1.58 bit LLM (2 bits in practice). > Someone reading this is probably writing it in VHDL right now, or will be soon. Yeah, I think I'm in the "will be soon" camp - FPGA board has been ordered. Especially with the 2-bit data types outlined in that paper [0] and more details in [1]. There's really a need for custom hardware to do that 2-bit math efficiently. Customizing one of the simpler open source RISC-V…

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by zachbee

Tue, 23 Apr 2024 00:27:21 GMT

The big challenge when it comes to using FPGAs for deep learning is pretty simple: all of that reprogrammability comes at a performance cost. If you're doing something highly specific that conventional GPUs are bad at, like genomics research [1] or high-frequency trading [2], the performance tradeoff is worth it. But for deep learning, GPUs and AI ASICs are highly optimized for most of these computations, and an FPGA won't offer huge performance increases. The main advantage FPGAs offer is being able to take advantage of new model optimizations much earlier than ASIC implementations could.…

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by jstmm

Fri, 29 Mar 2024 21:47:21 GMT

In both the '1-bit Model' and '2-bit Model' tables, the forward time (sec) for Llama2-7B with FP16 (full precision) is 0.1 s, whereas it's ~0.231, ~0.257, ~0.353 s respectively for HQQ (1-bit) / HQQ+ (1-bit) / Quip# (2-bit) meaning the FP16 model has ~3x lower inference time. On the contrary, in the BitNet b1.56 paper [0] the authors report their 7b model has 2.9x reduced inference latency. It's not clear to me what's happening here. Can someone explain why the 1/2bit HQQ/HQQ+ models are so much slower than the BitNet b1.56 models? [0] https://arxiv.org/pdf/2402.17764.pdf

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — HN submission

Mon, 25 Mar 2024 13:33:10 GMT

The Era of 1-Bit LLMs: All Large Language Models Are in 1.58 Bits

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by tmalsburg2

Tue, 19 Mar 2024 08:53:32 GMT

> - Training isn’t done at 4-bits, to date this small size has only been for inference. Wasn't there a paper from Microsoft two weeks ago or so where they trained on log₂(3) bits? Edit: https://arxiv.org/pdf/2402.17764.pdf

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — HN submission

Fri, 01 Mar 2024 10:52:25 GMT

1.58-bit LLMs: thousands of tokens/sec, reduced computing footprint

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by spyder

Wed, 28 Feb 2024 14:54:24 GMT

From the BitNet paper: "Straight-through estimator. To train our 1-bit model, we employ the straight-through estimator (STE)[BLC13] to approximate the gradient during backpropagation. This method bypasses the nondifferentiable functions, such as the Sign (Eq. 2) and Clip (Eq. 5) functions, during the backward pass. STE allows gradients to flow through the network without being affected by these non-differentiable functions, making it possible to train our quantized model." also the author's (@shumingma) answer in the comments: https://huggingface.co/papers/2402.17764#65df17ed4d436404cdc...

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by btbuildem

Wed, 28 Feb 2024 13:12:16 GMT

Discussion on HF [1] implies that no, conversion is not helpful. It would take training the model from scratch. 1: https://huggingface.co/papers/2402.17764

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by imjonse

Wed, 28 Feb 2024 13:11:45 GMT

You're both right, I skimmed the paper, saw large model numbers but didn't notice it was for speed. On the HF page they say those models are being trained. https://huggingface.co/papers/2402.17764 "We haven't finished the training of the models beyond 3B as it requires much much more resources. However, we're optimistic about the results because we have verified that BitNet follows a similar performance-parameter scaling law as the full-precision LLMs. We'll update the results on larger models once they're ready."

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by bArray

Wed, 28 Feb 2024 13:04:26 GMT

+1 On this, the real proof would have been testing both models side-by-side. It seems that it may be published on GitHub [1] according to HuggingFace [2]. [1] https://github.com/microsoft/unilm/tree/master/bitnet [2] https://huggingface.co/papers/2402.17764

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — mentioned by bmacho

Wed, 28 Feb 2024 12:44:38 GMT

Read the pdf https://arxiv.org/pdf/2402.17764.pdf they call it 1-bit everywhere. I don't know why do they do this, 1-bit seems to be a very wrong name for {-1, 0, 1}.

Paper and discussion history. A mention is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — HN submission

Wed, 28 Feb 2024 12:32:24 GMT

The Era of 1-Bit LLMs: All Large Language Models Are in 1.58 Bits

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — HN submission

Wed, 28 Feb 2024 09:28:43 GMT

The Era of 1-bit LLMs: ternary parameters for cost-effective computing

1040 points; 447 discussion replies on HN. Paper and source history. A submission is not an endorsement.