HackerPapers

arXiv:2304.13187cs.AI

AI-assisted coding: Experiments with GPT-4

Russell A Poldrack, Thomas Lu, Gašper Beguš

Early discovery. Linked by one identifiable commenter. This paper is outside the main ranking; a mention is not an endorsement.

Abstract

Artificial intelligence (AI) tools based on large language models have acheived human-level performance on some computer programming tasks. We report several experiments using GPT-4 to generate computer code. These experiments demonstrate that AI code generation using the current generation of tools, while powerful, requires substantial human validation to ensure accurate performance. We also demonstrate that GPT-4 refactoring of existing code can significantly improve that code along several established metrics for code quality, and we show that GPT-4 can generate tests with substantial coverage, but that many of the tests fail when applied to the associated code. These findings suggest that while AI coding tools are very powerful, they still require humans in the loop to ensure validity and accuracy of the results.

arXiv comments
Journal reference
DOI
Subjects
cs.AIArtificial Intelligence cs.SESoftware Engineering

On Hacker News

Linked by 1 identifiable commenter in 1 comment across 1 thread, from May 2023 to May 2023. Found in 1 HN submission from Apr 2023 to Apr 2023. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments1 thread

  1. How much of AI's recent success is due to the Forer Effect? 1 comment

    • The odd thing is that some people report much better results than others. Most literature on the subject points out the importance of a "human in the loop" https://arxiv.org/abs/2304.13187 Generally, chatbots function in direct communication with humans and do not function…

      PaulHouleView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.