https://hackerpaper.kaipfister.de/p/2412.05430/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=44990390urn:hackerpapers:mention:2412.05430:44990390Fri, 22 Aug 2025 22:02:08 GMT<p>I watched an interview with one of the co-founders of Anthropic where his point is that although benchmarks saturate they&#x27;re still an important signal for model development. We think the situation is similar here - one the challenges is aligning the benchmark with the function of the models. Genomic benchmarks for gLMs and RNA foundation models have been very resistant to staturation. I think in NLP the problem is that they are victims of their own success where the models can be overfit to particular benchmarks really fast. In genomics we&#x27;re a bit behind. A good paper on this is DartEval…</p><p><a href="https://hackerpaper.kaipfister.de/p/2412.05430/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LG