Dissecting model behavior through agent trajectories — mentioned by gaurav71531

Fri, 19 Jun 2026 17:58:14 GMT

We show that a simple harness fixing 'intent-execution gap' achieves SOTA pass@1 on 21 models (across diverse model providers of Claude, GPT, Gemini, Grok, Qwen) on agentic benchmarks (SWE-Pro, -verif, tb2). This is first time a single open-source harness reproduce/improve results on popular benchmarks for modern LLMs! The code is public to try and build-on: Code: https://github.com/strands-labs/benchmark-harnesses More importantly, we also generated 138k high-quality agent trajectories (SOTA pass@1) and present a detailed study on them "Dissecting model behavior through agent trajectories"…

Paper and discussion history. A mention is not an endorsement.

Dissecting model behavior through agent trajectories · HackerPapers discussions

Dissecting model behavior through agent trajectories — mentioned by gaurav71531