https://hackerpaper.kaipfister.de/p/2512.18470/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=48628839urn:hackerpapers:mention:2512.18470:48628839Mon, 22 Jun 2026 11:41:56 GMT<p>There are upcoming benchmarks aimed at measuring the ability to work with brownfield tasks. (Of course, benchmarks can be gamed, but they are still better than unrealistic toy tasks that earlier generations of benchmarks used. Frontier labs are yet to use them in their tech reports or marketing material, though.:-) * SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios https://arxiv.org/abs/2512.18470 * SWE-CI: Evaluating Agent Capabilities in Maintaining Codebases via Continuous Integration https://arxiv.org/abs/2603.03823</p><p><a href="https://hackerpaper.kaipfister.de/p/2512.18470/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.SEhttps://news.ycombinator.com/item?id=46773932urn:hackerpapers:mention:2512.18470:46773932Tue, 27 Jan 2026 00:43:56 GMT<p>The most accessible way to start experimenting would be the Ralph loop: https://github.com/anthropics/claude-code/tree/main/plugins/... You could also work backwards from this paper: https://arxiv.org/abs/2512.18470</p><p><a href="https://hackerpaper.kaipfister.de/p/2512.18470/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.SE