https://hackerpaper.kaipfister.de/p/2603.29399/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=47912207urn:hackerpapers:mention:2603.29399:47912207Sun, 26 Apr 2026 17:43:24 GMT<p>Benchmarks/evals are really hard and they become harder when there’s huge incentive to game them at an industry scale. ELT-Bench is another recent example. It was the first serious attempt at a benchmark for data engineering workloads, published about a year ago. A few days ago, a follow-up paper from a group that includes one of the original authors audited the benchmark itself. The team gfound that the benchmark has structural issues that biased results. Here’s the paper: https://arxiv.org/abs/2603.29399 None of these are new though, the industry has gone through all that before just in a…</p><p><a href="https://hackerpaper.kaipfister.de/p/2603.29399/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.AI