QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models · HackerPapers discussions

QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models — mentioned by renonce

Fri, 31 May 2024 10:28:37 GMT

I study LLM quantization and I have surveyed GPTQ and QuIP# and lots of quantization algorithms (specifically PTQ, post-training quantization) to develop my own, and my experience has led me to become extremely skeptical of many of the papers. I've seen lots of headlines like "1-bit quantization" (including this one and https://arxiv.org/abs/2310.16795). What I've found in this space is that the headlines can often be intentionally misleading about what is actually achieved. If you read closer the abstract of this paper, it claims 8.41 perplexity on LLaMA2-70B at 1 bit, which is a HUGE…

Paper and discussion history. A mention is not an endorsement.

QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models — mentioned by Cieric

Fri, 29 Mar 2024 16:34:58 GMT

I assume this is ment to be a joke, but isn't this actually being worked on? (minus the probabilistic portion) https://arxiv.org/abs/2310.16795

Paper and discussion history. A mention is not an endorsement.

QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models — HN submission

Wed, 13 Dec 2023 18:57:12 GMT

QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models

43 points; 12 discussion replies on HN. Paper and source history. A submission is not an endorsement.

QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models — mentioned by janwas

Sun, 10 Dec 2023 19:29:58 GMT

Yes :) https://arxiv.org/abs/2310.16795

Paper and discussion history. A mention is not an endorsement.

QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models — HN submission

Thu, 26 Oct 2023 11:56:17 GMT

QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models

1 point; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.