Think AI coding tools are speeding up work? Think again – they’re actually slowing developers down

AI might actually be slowing down software developers, new research suggests, despite their own positive predictions that AI coding tools will help their work.

Model Evaluation & Threat Research (METR) ran a randomized, controlled trial to study the use of early-2025 AI tools used by experienced developers working on their own open source repositories.

The results weren't positive, with the study finding that developers take 19% longer on average when using AI tools.

That said, METR noted this is a single snapshot of AI capabilities in a single setting — results may vary, METR's essentially saying — and the company hopes to use the figures as a baseline to track and estimate AI acceleration and adoption.

Experienced developers aren’t seeing benefits

METR's study used 16 developers with at least five years of experience who provided a list of 246 issues, such as bug fixes or wished-for features, that would make up their real day-to-day work. The developers in the study largely used Cursor Pro and Claude Sonnet.

"When developers are allowed to use AI tools, they take 19% longer to complete issues—a significant slowdown that goes against developer beliefs and expert forecasts," the post noted.

"This gap between perception and reality is striking: developers expected AI to speed them up by 24%, and even after experiencing the slowdown, they still believed AI had sped them up by 20%."

The METR team listed a few factors they believe likely contributed to the slowdown — the first of which was simply being over-optimistic about the usefulness of AI.

Beyond that, the study showed developers accepted fewer than 44% of AI generated code, with most of those taking part in the study saying they had to make major changes to clean up code. All told, roughly 9% of their time was spent reviewing or cleaning AI output.

Developers reported that AI appeared to struggle in large, complex environments, while the human coders themselves had so much experience in their areas of expertise that the AI couldn't keep up, an issue exacerbated by the fact that it "doesn't utilize important tacit knowledge or context."

The researchers also spotted a few potential causes for the AI slowdown, though they stressed the impact was unclear. That included developers using the AI to experiment and waiting on AI to generate code.

The researchers stressed that the results don't mean that AI is never useful. "Furthermore, these results do not imply that future models will not speed up developers in this exact setting — this is a salient possibility given the rapid pace of progress in AI capabilities recently," the paper said.

"Finally, it remains possible that further improvements to current AI systems (e.g. better prompting/agent scaffolding, or domain-specific finetuning) could yield positive speed up in this setting."

Does AI help or hinder?

Studying real-world impact is necessary, METR said, as AI benchmarks are often unrealistic or can't be easily applied to real-world tasks.

"While coding/agentic benchmarks have proven useful for understanding AI capabilities, they typically sacrifice realism for scale and efficiency—the tasks are self-contained, don’t require prior context to understand, and use algorithmic evaluation that doesn’t capture many important capabilities," the post noted.

"These properties may lead benchmarks to overestimate AI capabilities."

The METR study follows research by Atlassian that suggested AI tools are saving 86% of developers a full day of work each week, letting them focus on improving code quality or creating new features.

However, half of those surveyed said they are losing that AI-gained time because of "organisational inefficiencies" — essentially, bad management is eating up any savings.

"This pressure-cooked mix of innovation and strain demands a closer look at how AI is reshaping the developer experience, and what that means for the future of software development across the industry,” said Atlassian CTO Rajeev Rajan.

Make sure to follow ITPro on Google News to keep tabs on all our latest news, analysis, and reviews.

MORE FROM ITPRO

• CEOs think workers are becoming hostile to AI tools, but they’re pushing ahead with adoption anyway
• AI coding tools are finally delivering results for enterprises
• Big tech promised developers productivity gains with AI tools – now they’re being rendered obsolete