An empirical study of LLaMA3 quantization: from LLMs to MLLMs — mentioned by sbierwagen

Tue, 07 May 2024 23:05:05 GMT

Interestingly enough, Llama3 suffers more performance loss than Llama2 did at identical quantizations. https://arxiv.org/abs/2404.14047 There's some speculation that a net trained for more epochs on more data learns to pack more information into the weights, and so does worse when weight data is degraded.

Paper and discussion history. A mention is not an endorsement.

An empirical study of LLaMA3 quantization: from LLMs to MLLMs · HackerPapers discussions

An empirical study of LLaMA3 quantization: from LLMs to MLLMs — mentioned by sbierwagen