BiLLM: Pushing the Limit of Post-Training Quantization for LLMs — mentioned by light_hue_1

Fri, 31 May 2024 04:35:19 GMT

The performance loss from BiLLM is disastrous. It's basically useless. No one would ever want to use the resulting models. They hide their main results in the appendix: Table 8. page 15. https://arxiv.org/pdf/2402.04291 I won't go over the entire table in detail, but PIQA, BoolQ, HellaSwag, and WinoGrande should in the mid-to-high 70s for LLaMa2-7B. They drop that to 58, 62, 32, and 51. There are 700M parameter models that perform much better. What they should have reported is effective number of parameters. Does LLaMa2-7B with their quantization method outperform a model that has amount of…

Paper and discussion history. A mention is not an endorsement.

BiLLM: Pushing the Limit of Post-Training Quantization for LLMs · HackerPapers discussions

BiLLM: Pushing the Limit of Post-Training Quantization for LLMs — mentioned by light_hue_1