Accuracy is Not All You Need — mentioned by BoorishBears

Fri, 25 Apr 2025 20:12:33 GMT

I do? I spend a ton of time post-training models for creative tasks. The effects of model quantization are usually qualified in terms of performance on benchmaxxed tasks with strong logit probabilities, temp 0, and a "right" answer the model has to pick. Or even worse they'll be measured on metrics that don't map to anything except themselves like perplexity (https://arxiv.org/pdf/2407.09141) I agree Q8 is strong but I also think the effects of quantization are constantly being underappreciated. People are often talking about how these models perform while fundamentally using 10+ variants of…

Paper and discussion history. A mention is not an endorsement.

Accuracy is Not All You Need — HN submission

Tue, 16 Jul 2024 11:21:13 GMT

Accuracy is Not All You Need

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Accuracy is Not All You Need · HackerPapers discussions

Accuracy is Not All You Need — mentioned by BoorishBears

Accuracy is Not All You Need — HN submission