https://hackerpaper.kaipfister.de/p/2107.07002/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=29458753urn:hackerpapers:mention:2107.07002:29458753Mon, 06 Dec 2021 12:11:44 GMT<p>Too bad they don&#x27;t cite the paper &quot;The Benchmark Lottery&quot; (M. Dehghani et al, 2021) (https://arxiv.org/abs/2107.07002) &gt; The world of empirical machine learning (ML) strongly relies on benchmarks in order to determine the relative effectiveness of different algorithms and methods. This paper proposes the notion of &quot;a benchmark lottery&quot; that describes the overall fragility of the ML benchmarking process. The benchmark lottery postulates that many factors, other than fundamental algorithmic superiority, may lead to a method being perceived as superior. On multiple benchmark setups that are…</p><p><a href="https://hackerpaper.kaipfister.de/p/2107.07002/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LGhttps://news.ycombinator.com/item?id=27865491urn:hackerpapers:submission:2107.07002:27865491Sat, 17 Jul 2021 13:04:48 GMT<p>The Benchmark Lottery</p><p></p><p>57 points; 12 discussion replies on HN. <a href="https://hackerpaper.kaipfister.de/p/2107.07002/">Paper and source history</a>. A submission is not an endorsement.</p>cs.LG