Think AI coding tools are speeding up work? Think again – they’re actually slowing developers down
AI coding tools may be hindering the work of experienced software developers, according to new research
AI might actually be slowing down software developers, new research suggests, despite their own positive predictions that AI coding tools will help their work.
Model Evaluation & Threat Research (METR) ran a randomized, controlled trial to study the use of early-2025 AI tools used by experienced developers working on their own open source repositories.
The results weren't positive, with the study finding that developers take 19% longer on average when using AI tools.
That said, METR noted this is a single snapshot of AI capabilities in a single setting — results may vary, METR's essentially saying — and the company hopes to use the figures as a baseline to track and estimate AI acceleration and adoption.
Experienced developers aren’t seeing benefits
METR's study used 16 developers with at least five years of experience who provided a list of 246 issues, such as bug fixes or wished-for features, that would make up their real day-to-day work. The developers in the study largely used Cursor Pro and Claude Sonnet.
"When developers are allowed to use AI tools, they take 19% longer to complete issues—a significant slowdown that goes against developer beliefs and expert forecasts," the post noted.
"This gap between perception and reality is striking: developers expected AI to speed them up by 24%, and even after experiencing the slowdown, they still believed AI had sped them up by 20%."
The METR team listed a few factors they believe likely contributed to the slowdown — the first of which was simply being over-optimistic about the usefulness of AI.
Beyond that, the study showed developers accepted fewer than 44% of AI generated code, with most of those taking part in the study saying they had to make major changes to clean up code. All told, roughly 9% of their time was spent reviewing or cleaning AI output.
Developers reported that AI appeared to struggle in large, complex environments, while the human coders themselves had so much experience in their areas of expertise that the AI couldn't keep up, an issue exacerbated by the fact that it "doesn't utilize important tacit knowledge or context."
The researchers also spotted a few potential causes for the AI slowdown, though they stressed the impact was unclear. That included developers using the AI to experiment and waiting on AI to generate code.
The researchers stressed that the results don't mean that AI is never useful. "Furthermore, these results do not imply that future models will not speed up developers in this exact setting — this is a salient possibility given the rapid pace of progress in AI capabilities recently," the paper said.
"Finally, it remains possible that further improvements to current AI systems (e.g. better prompting/agent scaffolding, or domain-specific finetuning) could yield positive speed up in this setting."
Does AI help or hinder?
Studying real-world impact is necessary, METR said, as AI benchmarks are often unrealistic or can't be easily applied to real-world tasks.
"While coding/agentic benchmarks have proven useful for understanding AI capabilities, they typically sacrifice realism for scale and efficiency—the tasks are self-contained, don’t require prior context to understand, and use algorithmic evaluation that doesn’t capture many important capabilities," the post noted.
"These properties may lead benchmarks to overestimate AI capabilities."
The METR study follows research by Atlassian that suggested AI tools are saving 86% of developers a full day of work each week, letting them focus on improving code quality or creating new features.
However, half of those surveyed said they are losing that AI-gained time because of "organisational inefficiencies" — essentially, bad management is eating up any savings.
"This pressure-cooked mix of innovation and strain demands a closer look at how AI is reshaping the developer experience, and what that means for the future of software development across the industry,” said Atlassian CTO Rajeev Rajan.
Make sure to follow ITPro on Google News to keep tabs on all our latest news, analysis, and reviews.
MORE FROM ITPRO
- CEOs think workers are becoming hostile to AI tools, but they’re pushing ahead with adoption anyway
- AI coding tools are finally delivering results for enterprises
- Big tech promised developers productivity gains with AI tools – now they’re being rendered obsolete
-
How can resellers use AI to enhance, not risk, their trusted advisor statusIndustry Insights AI is seductive, but data shows people like buying from genuine people
-
Businesses need to boost cyber resilience, here’s howIn depth The government’s recently released Cyber Security Breaches Survey shows gaps in firms’ resilience. How can companies improve their approach?
-
‘The challenge now is making sure the next generation develops those same foundations before relying too heavily on AI’: Devs are swerving fundamental skills like Git and Agile because of AI – but there’s a good reasonNews O'Reilly has recorded a massive fall in programming fundamentals courses, but that’s not to suggest devs aren’t learning key skills
-
Will AI ring the death knell for open source?In-depth With projects facing fresh challenges in the AI era, the path forward remains murky
-
‘The biggest barrier to growth is not access to technology, it is access to the right people’: Demand for developers with AI skills has surged 597% – but enterprises are still struggling to find the right talentNews Hiring is shifting away from traditional software development toward specialized roles to integrate, govern, and scale AI systems
-
Forward deployed engineers are big tech’s latest gambit to drive AI adoptionNews With Microsoft and AWS placing their faith in forward deployed engineers, enterprises will gain a helping hand with tricky AI adoption projects
-
Enterprises are shipping so much AI-generated code they can't control or secure itNews As AI coding becomes commonplace, organizations are struggling to control what they are shipping
-
Why is Windows 11 so disliked by programmers – and can Microsoft do anything to change things?Windows isn't the most useful OS in the eyes of developers, with programmers preferring macOS or Linux. But is its bad reputation uncalled for?
-
The UK is betting big on the power of open source AINews The government wants to encourage open source developers to help improve public services
-
Asana wants every enterprise to have an AI ‘chief of staff’News The new Asana Dash tool was built to help guide and support teams through projects
