HackerPapers

arXiv:2108.09293cs.CR

Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions

Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, Ramesh Karri

Abstract

There is burgeoning interest in designing AI-based systems to assist humans in designing computing systems, including tools that automatically generate computer code. The most notable of these comes in the form of the first self-described `AI pair programmer', GitHub Copilot, a language model trained over open-source GitHub code. However, code often contains bugs - and so, given the vast quantity of unvetted code that Copilot has processed, it is certain that the language model will have learned from exploitable, buggy code. This raises concerns on the security of Copilot's code contributions. In this work, we systematically investigate the prevalence and conditions that can cause GitHub Copilot to recommend insecure code. To perform this analysis we prompt Copilot to generate code in scenarios relevant to high-risk CWEs (e.g. those from MITRE's "Top 25" list). We explore Copilot's performance on three distinct code generation axes -- examining how it performs given diversity of weaknesses, diversity of prompts, and diversity of domains. In total, we produce 89 different scenarios for Copilot to complete, producing 1,689 programs. Of these, we found approximately 40% to be vulnerable.

arXiv comments
Accepted for publication in IEEE Symposium on Security and Privacy 2022
Journal reference
DOI
Subjects
cs.CRCryptography and Security cs.AIArtificial Intelligence

On Hacker News

Linked by 5 identifiable commenters in 6 comments across 5 threads, from Sep 2021 to Jan 2023. Found in 1 HN submission from Aug 2021 to Aug 2021. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments5 threads

  1. It's plausible, but is it true? 1 comment

    • It's an anecdote which is meant to illustrate a widely observed behavior among many humans: we're better at accounting for frequent errors than we are at accounting for rare ones. This is well established in the field of human reliability analysis (HRA). Take a look at the HRA…

      elil17View on HNParent context

  2. Study finds AI assistants help developers produce code likely to be buggy 1 comment

    • This is a really nice study! It is very cool that they were able to get professional programmers to participate, this is something that is really hard to set up as an academic team. And yes, 47 participants is a small number, but apparently sufficient in this case to detect the…

      moyixView on HNParent context

  3. Papers on GitHub Copilot, copyright law, and ownership for AI-generated code 1 comment

    • To be honest, I wish that better code repositories we're given more weight. Microsoft's should be given more weight, especially considering the security issues that is autosuggested by copilot. https://arxiv.org/abs/2108.09293

      abirchView on HNParent context

  4. GitHub Copilot available for JetBrains and Neovim 1 comment

    • This attack has been studied! https://arxiv.org/abs/2007.02220 Although our own work shows Copilot is pretty good at adding security flaws on its own: https://arxiv.org/abs/2108.09293

      moyixView on HNParent context

  5. GitHub Copilot Generated Insecure Code in 40% of Circumstances During Experiment 2 comments

    • I couldn't find a link to the actual study anywhere in the article: https://arxiv.org/abs/2108.09293

      wcarssView on HNParent context

    • the actual paper: https://arxiv.org/abs/2108.09293 previous discussion including comments from lead author: https://news.ycombinator.com/item?id=28279365

      mzsView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.