AI coding really isn't living up to expectations – "the savings have been unremarkable" but not for the reason you might think

The benefits of AI coding have been “unremarkable” so far, according to new research, with developers struggling to unlock clear-cut gains from the technology.

That's the assessment from Bain & Company after analyzing how generative AI has impacted software development – and the lack of big payoffs from implementing the technology to date.

The report found that AI coding assistants may be able to take on 40% of a developer's work, including bug fixing, maintenance, code development, review and validation.

However, that hasn't led to a serious transformation in terms of broader operational efficiency.

"Software coding was one of the first areas to deploy generative AI, but the savings have been unremarkable," the consultancy said.

One reason for that is that coding itself makes up less than 40% of a software engineer's work day, limiting the impact of generative AI if they're unable to speed up in other aspects of their role.

Bain added that two-in-three software firms have rolled out generative AI tools, but developer adoption remains low. Those that are using AI assistants are reporting up to 15% increases in productivity, but that time isn't being used in higher-value work.

"So even those modest gains don’t translate into positive returns," the consultancy added.

The Bain report fits with previous research that showed coding tools may actually slow down software engineers, in part because they had to clean up much of the code generated by AI.

Similar research from Atlassian also pointed to an "unexpected paradox" in AI software development. Developers were found to save around 10 hours a week using these tools, but "organizational inefficiencies" were hampering broader productivity gains.

How to win with AI coding

Bain argues that more AI is the answer. The study noted that companies seeing the most gains are applying AI across the software lifecycle and beyond simple coding use-cases.

"If AI speeds up coding, then code review, integration, and release must speed up as well to avoid bottlenecks," the company says, noting that Netflix has successfully "shifted left" to ensure AI code isn't held back by other parts of the process.

The best way to boost returns on AI investment was to redesign processes to use the technology more efficiently, Bain insisted, as well as taking advantage of time saved with AI coding to higher-value work.

"Leading adopters treat generative AI as a fundamental transformation of their software development life cycle rather than a one-off project," the report noted.

"They take a future-back approach to rearchitect their end-to-end software development life cycle around generative AI, embedding it deeply into workflows and scaling it enterprise-wide. They weave it into development workflows and scale it across use cases."

That should all become easier with the rise of agentic AI, the company said, as agents will be able to better manage multiple stages of development.

Leaders find success with AI

Despite that, Bain still advised that the use of generative AI across wider processes, rather than just with coding, was boosting income for those who have led in deployments.

Some companies are delivering gains of between 10% to 25% by scaling the technology across core workflows, the company added.

"If you’re still piloting, you’re dangerously behind," noted David Crawford, Chairman of Bain’s Global Technology, Media, and Telecommunications practice, in the report.

For those companies that are struggling to keep up, Bain advised moving out of "experimental mode" by following the lead of companies that have already found success, as "they established a repeatable playbook that others can now follow – grounded in robust methodology, analytic tools, and clear benchmarks.

Make sure to follow ITPro on Google News to keep tabs on all our latest news, analysis, and reviews.

MORE FROM ITPRO

• Big tech promised developers productivity gains with AI tools – now they’re being rendered obsolete
• AI in software engineering: Six ways the profession is changing
• Microsoft says AI is finally having a 'meaningful impact' on developer productivity