Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity · HackerPapers discussions

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by Loquebantur

Thu, 15 Oct 2026 17:39:43 GMT

> think for yourself. AI will only ruin your mind and deskill you. here are some articles: > think AI makes you super hacker? think again: https://arxiv.org/abs/2507.09089 ["Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity"] > https://blog.codeberg.org/protecting-our-floss-commons-from-... > consider that the infrastructure for AI is economically unsustainable; when the bubble bursts, and it will, we will suddenly be left with millions of people thoroughly deskilled who will be useless. lots of reeducation needed, and lots of people will just be left…

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by john_strinlai

Thu, 15 Oct 2026 13:39:55 GMT

you arent kidding! >"think AI makes you super hacker? think again: https://arxiv.org/abs/2507.09089" not that im particularly pro-ai, but it seems a bit silly to rely on a paper about early-2025 ai when people are using late-2026 ai, given how different they are.

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by ghboo0927

Mon, 12 Oct 2026 02:25:46 GMT

A double-difference comparison between GitHub projects that have adopted Cursor and paired projects that have not: https://arxiv.org/abs/2511.04427 https://arxiv.org/abs/2507.09089

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by Miladyshady

Sat, 28 Feb 2026 12:04:10 GMT

The idea that AI will replace programmers has been around since the emergence of AI. I do not know what the future holds. But I know that using in AI in software engineering reduces productivity by almost 20%. My point is, one tries to distinguish fact from opinion. Source: https://arxiv.org/abs/2507.09089

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by shakna

Sun, 25 Jan 2026 01:02:22 GMT

https://arxiv.org/abs/2507.09089

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by sockopen

Wed, 21 Jan 2026 12:51:14 GMT

https://arxiv.org/abs/2507.09089 “Before starting tasks, developers forecast that allowing AI will reduce completion time by 24%. After completing the study, developers estimate that allowing AI reduced completion time by 20%. Surprisingly, we find that allowing AI actually increases completion time by 19%--AI tooling slowed developers down.”

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by trashb

Wed, 21 Jan 2026 10:32:12 GMT

Those are some high profile (celebrity) developers. I wonder if they have measured their results? I believe that the perceived speed up of AI coding is often different from reality. The following paper backs this idea https://arxiv.org/abs/2507.09089 . Can you provide data that objects this view, based on these (celebrity) developers or otherwise?

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by yorwba

Sun, 18 Jan 2026 14:08:18 GMT

The METR study referenced (but not linked) in that paragraph has a breakdown of time spent https://arxiv.org/pdf/2507.09089 in figure 19 on page 33 which contradicts the description in the Register article. Open-source developers participating in the study who were randomly assigned to be allowed to use AI to work on an issue in their repo only spent insignificantly longer testing and debugging, but did spend much longer reviewing AI output, prompting AI, waiting for AI and "idle / overhead." In particular, time spent prompting AI alone was longer than the reduction in active coding. Lessons…

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — HN submission

Sat, 13 Dec 2025 14:54:29 GMT

Measuring Impact of Early-2025 AI on Experienced Open-Source Dev Productivity

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by lunar_mycroft

Tue, 09 Dec 2025 20:08:18 GMT

It "obviously" does based on what, exactly? For most devs (and it appears you, based on your comments) the answer is "their own subjective impressions", but that METR study (https://arxiv.org/pdf/2507.09089) should have completely killed any illusions that that is a reliable metric (note: this argument works regardless of how much LLMs have improved since the study period, because it's about how accurate dev's impressions are, not how good the LLMs actually were).

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by trashb

Wed, 26 Nov 2025 15:23:16 GMT

Have you measured? https://arxiv.org/abs/2507.09089

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by recitedropper

Wed, 19 Nov 2025 02:19:42 GMT

For seasoned maintainers of open source repos, there is explicit evidence it does slow them down, even when they think it sped them up: https://arxiv.org/abs/2507.09089 Cue: "the tools are so much better now", "the people in the study didn't know how to use Cursor", etc. Regardless if one takes issue with this study, there are enough others of its kind to suggest skepticism regarding how much these tools really create speed benefits when employed at scale. The maintenance cliff is always nigh... There are definitely ways in which LLMs, and agentic coding tools scaffolded in top, help with…

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by autoexec

Tue, 11 Nov 2025 17:29:37 GMT

Unlike higher level programing languages AI doesn't actually make programmers more efficient (https://arxiv.org/abs/2507.09089). Many people who are great programmers and love programing aren't interested in having their role reduced to being QA where they just review the bad code AI designed and wrote all day long. In a hypothetical world where AI is actually decent enough to be any good at writing software, the demand for software being infinite won't save even one programmer's job because zero programmers will be needed to create any of it. Everyone who needs software will just ask AI to…

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by meander_water

Sun, 26 Oct 2025 09:53:11 GMT

There's a few bits of information from the original sources that's left out: - The METR paper surveyed just 16 developers to arrive at their conclusion. Not sure how that got past review. [0] - The finding from the MIT report can also be viewed from a glass 5% full perspective: > Just 5% of integrated AI pilots are extracting millions in value. > Winning startups build systems that learn from feedback (66% of executives want this), retain context (63% demand this), and customize deeply to specific workflows. They start at workflow edges with significant customization, then scale into core…

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by _jsmh

Thu, 23 Oct 2025 21:41:28 GMT

"allowing AI actually increases completion time by 19%--AI tooling slowed developers down." https://arxiv.org/abs/2507.09089

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — HN submission

Tue, 21 Oct 2025 21:12:22 GMT

Measuring the Impact of Early-2025 AI on Experienced Developer Productivity

4 points; 2 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by Ianjit

Tue, 14 Oct 2025 11:09:31 GMT

"Just start iterating in chunks alongside the LLM". For those types of tasks it probably takes the same amount of time to form the idea without AI as with AI, this is what Metr found in its study of developer productivity. https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o... https://arxiv.org/abs/2507.09089

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by y0eswddl

Thu, 02 Oct 2025 15:08:33 GMT

There's literally a study out the shows when developers think LLMs are making them 20% faster, it turned out to be making them 20% less productive: https://arxiv.org/abs/2507.09089

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by pseudalopex

Wed, 01 Oct 2025 13:18:05 GMT

> Careful with the productivity claims, there's plenty of research showing that perspective is usually a distortion of some kind. I saw 1 study which suggested this.[1] Were there more? [1] https://arxiv.org/abs/2507.09089

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by thebigspacefuck

Tue, 23 Sep 2025 15:22:30 GMT

The METR study showed even though people feel more productive they weren’t https://arxiv.org/abs/2507.09089

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by dpflan

Sun, 21 Sep 2025 19:08:42 GMT

The study you are alluding to is this one by METR (Model Evaluation & Threat Research): Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity - https://arxiv.org/abs/2507.09089 """ Before starting tasks, developers forecast that allowing AI will reduce completion time by 24%. After completing the study, developers estimate that allowing AI reduced completion time by 20%. Surprisingly, we find that allowing AI actually increases completion time by 19%—AI tooling slowed developers down. This slowdown also contradicts predictions from experts in economics (39%…

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by aix1

Fri, 19 Sep 2025 04:40:21 GMT

The study you referred to sounded super interesting, so I looked it up to read later. To save others a search, here is a blog post and the paper: https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o... https://arxiv.org/abs/2507.09089 Thanks for mentioning it.

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by raincole

Sat, 13 Sep 2025 10:47:24 GMT

> It’s why the world wasted $10B+ on self driving car companies that obviously made no sense. Obviously... in what way? I feel the anti-ai pattern is clear. Self-driving cars don't work in my city so the whole concept is a hoax. LLMs don't code my proprietary language so it's a bubble. > From this study (https://arxiv.org/abs/2507.09089) I can tell this is going to be the most misquoted study in blogs and pop-sci books after the 10,000-hour mastery study. And it's just a preprint!

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by net01

Sat, 13 Sep 2025 10:45:21 GMT

This is shown in figure 5 of the paper. https://arxiv.org/pdf/2507.09089

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by jvdvegt

Wed, 10 Sep 2025 15:27:45 GMT

Full paper: https://arxiv.org/abs/2507.09089

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by mandeepj

Mon, 01 Sep 2025 15:49:34 GMT

> This also: AI slowing down completion time by 19% - https://arxiv.org/abs/2507.09089 Not sure which one is true

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by djoldman

Thu, 07 Aug 2025 12:11:34 GMT

> There’s a strange disconnect in the industry. On one hand, GitHub claims that 20 million users are on Copilot, and Sundar Pichai says that over 25% of the code at Google is now written by AI. On the other hand, independent studies show that AI actually makes experienced developers slower. From the study[0]: > 16 developers with moderate AI experience complete 246 tasks in mature projects on which they have an average of 5 years of prior experience. This study continues to get a lot of airtime on HN and elsewhere. Folks probably should be skeptical of a study with the combination of a small…

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by HarHarVeryFunny

Tue, 05 Aug 2025 16:41:49 GMT

There was a recent study concluding that AI made experienced developers 20% SLOWER to complete tasks rather than any faster ! https://arxiv.org/abs/2507.09089 Obviously it depends on what you are using the AI to do, and how good a job you do of creating/providing all the context to give it the best chance of being successful in what you are asking. Maybe a bit like someone using a leaf blower to blow a couple of leaves back and forth across the driveway for 30 sec rather than just bending down to pick them up.... It seems people find LLMs interesting, and want to report success in using them…

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — HN submission

Mon, 04 Aug 2025 09:03:06 GMT

Measuring the Impact of AI on Experienced Open-Source Developer Productivity

1 point; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by EnnEmmEss

Thu, 31 Jul 2025 05:25:14 GMT

The report is definitely worth reading although I don't think the data is strong enough to draw significant conclusions from. I personally think that this should be studied more in depth over a longer time period since the current study had a 50 hour threshold (not exactly but close enough). One thing which is mentioned in the research paper[1] is: > However, we see positive speedup for the one developer who has more than 50 hours of Cursor experience, so it’s plausible that there is a high skill ceiling for using Cursor, such that developers with significant experience see positive speedup.…

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by almog

Thu, 24 Jul 2025 16:33:08 GMT

Yet when you look beyond boilerplate code generation, it's not all that LLMs increase experienced developers productivity (even when they believe that it does): https://arxiv.org/abs/2507.09089 Edit: Hello downvoters, would love to know if you found any flawed argument, is this just because this study/comment contradicting the common narrative on HN or something else entirely?

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — mentioned by shakna

Sun, 20 Jul 2025 01:21:34 GMT

So... Attack first? That what you're going with? DKE has been confirmed more than anything else. It was one of the few things not hit by the replication crisis. You're assessing how well you do, when aided or not, by a tool. That's still self assessment, I'm afraid. And that self-assessment is flawed. [0] [0] https://arxiv.org/abs/2507.09089

Paper and discussion history. A mention is not an endorsement.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity — HN submission

Thu, 17 Jul 2025 14:03:27 GMT

Measuring the Impact of Early-2025 AI on Experienced Developer Productivity

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.