On Randomness in Agentic Evals · HackerPapers discussions

On Randomness in Agentic Evals — mentioned by suttontom

Fri, 12 Jun 2026 18:40:23 GMT

You're wrong in lots of ways. Some model cards do show regressions on benchmarks for newer models on specific tasks: https://storage.googleapis.com/deepmind-media/Model-Cards/Ge... This wasn't a new model but updates to models backed by numbers being better can make the model worse: https://openai.com/index/sycophancy-in-gpt-4o/ The slight increases in performance/benchmarks may be just noise: https://arxiv.org/pdf/2602.07150

Paper and discussion history. A mention is not an endorsement.

On Randomness in Agentic Evals — HN submission

Tue, 17 Feb 2026 14:29:26 GMT

Randomness in Agentic Evals

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

On Randomness in Agentic Evals — HN submission

Tue, 10 Feb 2026 09:00:32 GMT

Randomness in Agentic Evals

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.