HackerPapers

arXiv:2211.03622cs.CR

Do Users Write More Insecure Code with AI Assistants?

Neil Perry, Megha Srivastava, Deepak Kumar, Dan Boneh

Abstract

We conduct the first large-scale user study examining how users interact with an AI Code assistant to solve a variety of security related tasks across different programming languages. Overall, we find that participants who had access to an AI assistant based on OpenAI's codex-davinci-002 model wrote significantly less secure code than those without access. Additionally, participants with access to an AI assistant were more likely to believe they wrote secure code than those without access to the AI assistant. Furthermore, we find that participants who trusted the AI less and engaged more with the language and format of their prompts (e.g. re-phrasing, adjusting temperature) provided code with fewer security vulnerabilities. Finally, in order to better inform the design of future AI-based Code assistants, we provide an in-depth analysis of participants' language and interaction behavior, as well as release our user interface as an instrument to conduct similar studies in the future.

arXiv comments
16 pages, 16 figures, update adds names of statistical tests and survey questions, full version of conference paper
Journal reference
CCS '23: Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, November 2023, Pages 2785-2799
DOI
doi:10.1145/3576915.3623157
Subjects
cs.CRCryptography and Security

On Hacker News

Linked by 7 identifiable commenters in 8 comments across 8 threads, from Dec 2022 to Oct 2026. Found in 12 HN submissions from Nov 2022 to Apr 2024. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments8 threads

  1. Is this the new cyber normal? 1 comment

    • It's been shown to be bad at secure coding _unless_ you specifically prompt models to watch out for common vulnerabilities, at which point they're actually quite good at containing the issues, though the studies here are somewhat older (eg https://arxiv.org/abs/2211.03622). The…

      CER10TYView on HNParent context

  2. Mexican government breached by solo user with Claude, 150 GB exfiltrated 1 comment

    • All references here, do your own research: References [1] SecurityWeek, "Hackers Weaponize Claude Code in Mexican Government Cyberattack," Feb. 2026. [Online]. Available: https://www.securityweek.com/hackers-weaponize-claude-code-i... [2] Anthropic, "Threat Intelligence Report…

      ReaktornanoView on HNParent context

  3. Write-only code 1 comment

    • > We observed that participants who had access to the AI assistant were more likely to introduce security vulnerabilities for the majority of programming tasks, yet were also more likely to rate their insecure answers as secure compared to those in our control group.…

      pconstantineView on HNParent context

  4. To AI or not to AI 1 comment

    • > I am not a coder (much), but I have to wonder if my experience is common in the coding world? It is, yes. Surely someone will come and tell you it doesn’t happen to them, but all that tells you is that it ostensibly isn’t universal, but still common enough you’ll find no end…

      latexrView on HNParent context

  5. Lost at C: A user study on the security of LLM coding assistants 1 comment

    • More: Do Users Write More Insecure Code with AI Assistants? (https://arxiv.org/pdf/2211.03622.pdf) More relevant discussion, from Andrew Ng's The Batch newsletter: Check the Generated Code Generates Overconfident Coders section: (https://www.deeplearning.ai/the-batch/issue-180/)

      dpflanView on HNParent context

  6. AI assistants: The convenient crutch for new developers or a recipe for disaster 1 comment

    • Without expert filtering of the suggestions, yes, disaster. Experts will benefit the most from this because they can filter incorrect suggestions and reinterpret. Novices will be more susceptible. There is certainly ongoing research on this, expert systems have been an area of…

      dpflanView on HNParent context

  7. Kickstarter shuts down campaign for Unstable Diffusion amid changing guidelines 1 comment

    • Then again, it also seems we perform worse when using AI tools, so maybe I'm wrong https://arxiv.org/abs/2211.03622

      spoilerView on HNParent context

  8. The End of Programming 1 comment

    • This paper coomparing code developed without copilot to code developed with it feels relevant: https://arxiv.org/abs/2211.03622

      thyrsusView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.