Compressing LLMs: The Truth is Rarely Pure and Never Simple — mentioned by janwas

Fri, 28 Jun 2024 05:22:45 GMT

Yes, PPL and certain benchmarks do not detect differences from quantization. But recent work gives cause for concern, e.g., https://arxiv.org/pdf/2310.01382, https://arxiv.org/pdf/2405.18137.

Paper and discussion history. A mention is not an endorsement.

Compressing LLMs: The Truth is Rarely Pure and Never Simple · HackerPapers discussions

Compressing LLMs: The Truth is Rarely Pure and Never Simple — mentioned by janwas