HackerPapers

arXiv:2405.15793cs.SE

SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering

John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, Ofir Press

Abstract

Language model (LM) agents are increasingly being used to automate complicated tasks in digital environments. Just as humans benefit from powerful software applications, such as integrated development environments, for complex tasks like software engineering, we posit that LM agents represent a new category of end users with their own needs and abilities, and would benefit from specially-built interfaces to the software they use. We investigate how interface design affects the performance of language model agents. As a result of this exploration, we introduce SWE-agent: a system that facilitates LM agents to autonomously use computers to solve software engineering tasks. SWE-agent's custom agent-computer interface (ACI) significantly enhances an agent's ability to create and edit code files, navigate entire repositories, and execute tests and other programs. We evaluate SWE-agent on SWE-bench and HumanEvalFix, achieving state-of-the-art performance on both with a pass@1 rate of 12.5% and 87.7%, respectively, far exceeding the previous state-of-the-art achieved with non-interactive LMs. Finally, we provide insight on how the design of the ACI can impact agents' behavior and performance.

arXiv comments
Code, data, and demo available at https://swe-agent.com
Journal reference
DOI
Subjects
cs.SESoftware Engineering cs.AIArtificial Intelligence cs.CLComputation and Language cs.HCHuman-Computer Interaction cs.LGMachine Learning

On Hacker News

Linked by 2 identifiable commenters in 2 comments across 2 threads, from Aug 2024 to Aug 2025. Found in 2 HN submissions from May 2024 to Sep 2024. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments2 threads

  1. How to build a coding agent 1 comment

    • Mini swe agent, as an academic tool, can be easily tested aimed to show the power of a simple idea against any LLM. You can go and test it with different LLMs. Tool calls didn't work fine with smaller LLM sizes usually. I don't see many viable alternatives less than 7GB, beyond…

      diminishView on HNParent context

  2. OpenAI is good at unminifying code 1 comment

    • Super interesting! Since you're generating code with LLMs, you should check out this paper: https://arxiv.org/pdf/2405.15793 It uses smart feedback to fix the code when LLMs occasionally do hiccups with the code. You could also have a "supervisor LLM" that asserts that the…

      jehna1View on HNParent context

Submitted to Hacker News

  • Show HN: Codebased, an AI Search Engine for Code17 points8 comments

    Codebased combines Tree Sitter for code awareness (find functions, data structures, constants, etc. not just lines of code), full-text search using SQLite, and semantic search using OpenAI embeddings + FAISS. Despite being implemented in Python, supporting semantic search, making multiple API calls for embedding and re-ranking, it is faster than ripgrep for runng searches against the Linux kernel (takes ~1 second vs. ~2 seconds, obviously depends on system, temperature, time of day, tidal…

  • SWE-Agent: Agent-Computer Interfaces Enable Automated Software Engineering1 point0 comments

Connected through shared HN threads, not inferred similarity or a recommendation.