SWE-INTERACT: Reimagining SWE Benchmarks as User-Driven Long-Horizon Coding Sessions · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2606.30573/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTSWE-INTERACT: Reimagining SWE Benchmarks as User-Driven Long-Horizon Coding Sessions — mentioned by bisonbearhttps://news.ycombinator.com/item?id=48811785urn:hackerpapers:mention:2606.30573:48811785Mon, 06 Jul 2026 23:22:29 GMT<p>I&#x27;ve actually been working on a solution for this problem! https://www.stet.sh/ At a high level, it - Mines tasks from your merged PRs/commits - Replays them in Docker containers with different harness settings (change model / reasoning effort / AGENTS.md / etc) - Grades the patches on various attributes (tests, equivalence with human patch, code quality) The goal is to get a sense of how agents perform on your tasks, with your context, using the tools you do. This is currently one-shot but I&#x27;d definitely like to explore session-based benchmarks as well. There are some interesting papers that…</p><p><a href="https://hackerpaper.kaipfister.de/p/2606.30573/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LG