Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

you arent kidding! >"think AI makes you super hacker? think again: https://arxiv.org/abs/2507.09089" not that im particularly pro-ai, but it seems a bit silly to rely on a paper about early-2025 ai when people are using late-2026 ai, given how different they are.

john_strinlaiOct 15, 2026View on HNParent context

> think for yourself. AI will only ruin your mind and deskill you. here are some articles: > think AI makes you super hacker? think again: https://arxiv.org/abs/2507.09089 ["Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity"] >…

LoquebanturOct 15, 2026View on HNParent context

A double-difference comparison between GitHub projects that have adopted Cursor and paired projects that have not: https://arxiv.org/abs/2511.04427 https://arxiv.org/abs/2507.09089

ghboo0927Oct 12, 2026View on HNParent context

The idea that AI will replace programmers has been around since the emergence of AI. I do not know what the future holds. But I know that using in AI in software engineering reduces productivity by almost 20%. My point is, one tries to distinguish fact from opinion. Source…

MiladyshadyFeb 28, 2026View on HNParent context

https://arxiv.org/abs/2507.09089

shaknaJan 25, 2026View on HNParent context

Those are some high profile (celebrity) developers. I wonder if they have measured their results? I believe that the perceived speed up of AI coding is often different from reality. The following paper backs this idea https://arxiv.org/abs/2507.09089 . Can you provide data that…

trashbJan 21, 2026View on HNParent context

https://arxiv.org/abs/2507.09089 “Before starting tasks, developers forecast that allowing AI will reduce completion time by 24%. After completing the study, developers estimate that allowing AI reduced completion time by 20%. Surprisingly, we find that allowing AI actually…

sockopenJan 21, 2026View on HNParent context

The METR study referenced (but not linked) in that paragraph has a breakdown of time spent https://arxiv.org/pdf/2507.09089 in figure 19 on page 33 which contradicts the description in the Register article. Open-source developers participating in the study who were randomly…

yorwbaJan 18, 2026View on HNParent context

It "obviously" does based on what, exactly? For most devs (and it appears you, based on your comments) the answer is "their own subjective impressions", but that METR study (https://arxiv.org/pdf/2507.09089) should have completely killed any illusions that that is a reliable…

lunar_mycroftDec 9, 2025View on HNParent context

Have you measured? https://arxiv.org/abs/2507.09089

trashbNov 26, 2025View on HNParent context

For seasoned maintainers of open source repos, there is explicit evidence it does slow them down, even when they think it sped them up: https://arxiv.org/abs/2507.09089 Cue: "the tools are so much better now", "the people in the study didn't know how to use Cursor", etc.…

recitedropperNov 19, 2025View on HNParent context

Unlike higher level programing languages AI doesn't actually make programmers more efficient (https://arxiv.org/abs/2507.09089). Many people who are great programmers and love programing aren't interested in having their role reduced to being QA where they just review the bad…

autoexecNov 11, 2025View on HNParent context

There's a few bits of information from the original sources that's left out: - The METR paper surveyed just 16 developers to arrive at their conclusion. Not sure how that got past review. [0] - The finding from the MIT report can also be viewed from a glass 5% full perspective…

meander_waterOct 26, 2025View on HNParent context

"allowing AI actually increases completion time by 19%--AI tooling slowed developers down." https://arxiv.org/abs/2507.09089

_jsmhOct 23, 2025View on HNParent context

"Just start iterating in chunks alongside the LLM". For those types of tasks it probably takes the same amount of time to form the idea without AI as with AI, this is what Metr found in its study of developer productivity.…

IanjitOct 14, 2025View on HNParent context

There's literally a study out the shows when developers think LLMs are making them 20% faster, it turned out to be making them 20% less productive: https://arxiv.org/abs/2507.09089

y0eswddlOct 2, 2025View on HNParent context

> Careful with the productivity claims, there's plenty of research showing that perspective is usually a distortion of some kind. I saw 1 study which suggested this.[1] Were there more? [1] https://arxiv.org/abs/2507.09089

pseudalopexOct 1, 2025View on HNParent context

The METR study showed even though people feel more productive they weren’t https://arxiv.org/abs/2507.09089

thebigspacefuckSep 23, 2025View on HNParent context

The study you are alluding to is this one by METR (Model Evaluation & Threat Research): Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity - https://arxiv.org/abs/2507.09089 """ Before starting tasks, developers forecast that allowing AI will…

dpflanSep 21, 2025View on HNParent context

The study you referred to sounded super interesting, so I looked it up to read later. To save others a search, here is a blog post and the paper: https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o... https://arxiv.org/abs/2507.09089 Thanks for mentioning it.

aix1Sep 19, 2025View on HNParent context

This is shown in figure 5 of the paper. https://arxiv.org/pdf/2507.09089

net01Sep 13, 2025View on HNParent context

> It’s why the world wasted $10B+ on self driving car companies that obviously made no sense. Obviously... in what way? I feel the anti-ai pattern is clear. Self-driving cars don't work in my city so the whole concept is a hoax. LLMs don't code my proprietary language so it's a…

raincoleSep 13, 2025View on HNParent context

Full paper: https://arxiv.org/abs/2507.09089

jvdvegtSep 10, 2025View on HNParent context

> This also: AI slowing down completion time by 19% - https://arxiv.org/abs/2507.09089 Not sure which one is true

mandeepjSep 1, 2025View on HNParent context

> There’s a strange disconnect in the industry. On one hand, GitHub claims that 20 million users are on Copilot, and Sundar Pichai says that over 25% of the code at Google is now written by AI. On the other hand, independent studies show that AI actually makes experienced…

djoldmanAug 7, 2025View on HNParent context

There was a recent study concluding that AI made experienced developers 20% SLOWER to complete tasks rather than any faster ! https://arxiv.org/abs/2507.09089 Obviously it depends on what you are using the AI to do, and how good a job you do of creating/providing all the context…

HarHarVeryFunnyAug 5, 2025View on HNParent context

The report is definitely worth reading although I don't think the data is strong enough to draw significant conclusions from. I personally think that this should be studied more in depth over a longer time period since the current study had a 50 hour threshold (not exactly but…

EnnEmmEssJul 31, 2025View on HNParent context

Yet when you look beyond boilerplate code generation, it's not all that LLMs increase experienced developers productivity (even when they believe that it does): https://arxiv.org/abs/2507.09089 Edit: Hello downvoters, would love to know if you found any flawed argument, is this…

almogJul 24, 2025View on HNParent context

So... Attack first? That what you're going with? DKE has been confirmed more than anything else. It was one of the few things not hit by the replication crisis. You're assessing how well you do, when aided or not, by a tool. That's still self assessment, I'm afraid. And that…

shaknaJul 20, 2025View on HNParent context