The Impact of AI on Developer Productivity: Evidence from GitHub Copilot · HackerPapers discussions

The Impact of AI on Developer Productivity: Evidence from GitHub Copilot — mentioned by aspenmartin

Thu, 01 Jan 2026 22:16:11 GMT

That's why I gave you data! METR study was 16 people using Sonnet 3.5/3.7. Data I'm talking about is 10s of thousands of people and is much more up to date. Some counter examples to METR that are in the literature but I'll just say: "rigor" here is very difficult (including METR) because outcomes are high dimensional and nuanced, or ecological validity is an issue. It's hard to have any approach that someone wouldn't be able to dismiss due to some issue they have with the methodology. The sources below also have methodological problems just like METR https://arxiv.org/pdf/2302.06590 -- 55%…

Paper and discussion history. A mention is not an endorsement.

The Impact of AI on Developer Productivity: Evidence from GitHub Copilot — mentioned by keeda

Mon, 11 Aug 2025 03:34:44 GMT

There are more studies out there, but here are a couple I know of offhand, showing a 25% to 55% boost. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4945566 https://arxiv.org/abs/2302.06590 The METR study that you're likely talking about had a lot of nuances that don't get talked about, not to mention outright concerns e.g. this one participant revealed he had a pretty damning selection bias: https://xcancel.com/ruben_bloom/status/1943536052037390531

Paper and discussion history. A mention is not an endorsement.

The Impact of AI on Developer Productivity: Evidence from GitHub Copilot — mentioned by davely

Tue, 05 Aug 2025 00:31:49 GMT

The study that said devs were 19% less productive was flawed in a number of ways: small sample size, poorly chosen tasks, no prior training on AI tools. The science is by no means conclusive. A joint Harvard / MIT study found developers completed tasks up to 50% faster [0]! [0] https://arxiv.org/abs/2302.06590

Paper and discussion history. A mention is not an endorsement.

The Impact of AI on Developer Productivity: Evidence from GitHub Copilot — HN submission

Sun, 06 Jul 2025 21:09:52 GMT

The Impact of AI on Developer Productivity: Evidence from GitHub Copilot (2023)

6 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

The Impact of AI on Developer Productivity: Evidence from GitHub Copilot — HN submission

Mon, 17 Jun 2024 14:52:08 GMT

The Impact of AI on Developer Productivity: Evidence from GitHub Copilot

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

The Impact of AI on Developer Productivity: Evidence from GitHub Copilot — mentioned by NathanFulton

Thu, 01 Jun 2023 16:05:31 GMT

> Have there been any good studies on programmer productivity? Yes, but mostly from the companies developing these products: * The CoPilot productivity study by Peng et al. that is summarized in a Blog post [1] with some additional details in the Arxiv preprint [2]. They find "developers who used GitHub Copilot completed the task significantly faster–55% faster than the developers who didn’t use GitHub Copilot". (Grep for "but speed is important, too" in [1].) * Amazon's Q1 earnings report [3] includes a paragraph stating that "participants [using] CodeWhisperer completed tasks 57% faster (on…

Paper and discussion history. A mention is not an endorsement.

The Impact of AI on Developer Productivity: Evidence from GitHub Copilot — HN submission

Wed, 08 Mar 2023 23:07:44 GMT

The Impact of AI on Developer Productivity: Evidence from GitHub Copilot

4 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.