From Benchmarks to Skills: Low-Rank Factors for LLM Evaluation — mentioned by moritzwarhier

Sat, 08 Nov 2025 16:00:20 GMT

When people claim that there is such a thing as "X% accuracy in reasoning", it's really hard to take anything else seriously, no matter how impressive. AI (and humans!) aside, claiming that there was an oracle that could "answer all questions" is a solved problem. Such a thing cannot exist. But this is going already too deep IMO. When people start talking about percentages or benchmark scores, there has to be some denominator. And there can be no bias-free such denominator for - trivia questions - mathematical questions (oh, maybe I'm wrong here, intuitively I'd say it's impossible for…

Paper and discussion history. A mention is not an endorsement.

From Benchmarks to Skills: Low-Rank Factors for LLM Evaluation · HackerPapers discussions

From Benchmarks to Skills: Low-Rank Factors for LLM Evaluation — mentioned by moritzwarhier