If your developers aren’t using an AI pair programmer already, they will be very soon. By 2028, three-quarters of enterprise software engineers will use some sort of generative AI code assistant, according to research by tech analyst Gartner.
Its survey found that 63% of organizations were piloting, deploying or had already deployed AI coding assistants by the third quarter of 2023. This represents a significant jump from the 10% of developers that used AI coding tools at the start of 2023.
Generative AI tools for developers can generate new code or suggest improvements to existing code that developers can rapidly incorporate into their applications.
These benefits, in turn, can lead to increased job satisfaction and retention, potentially saving the significant costs of finding new developers. But while there have been claims that these tools can boost developers’ efficiency by as much as 50%, the reality is more complicated.
Businesses are finding that when their teams are using coding assistants on their complex, idiosyncratic business logic and legacy codebases they may actually only get five, ten, or 15% efficiency savings, Philip Walsh, senior principal analyst at Gartner, told ITPro.
Part of the issue is that developers don’t spend all their time coding. Gartner estimates developers spend 20 to 30% of their time writing code, so even if organizations unlock significant efficiencies AI pair programmers this only applies to a fraction of a developer’s time, Walsh said.
“From starting work on a request to submitting the pull request, going through code review, testing and verification, and deployment delivering enterprise-grade software is a team sport with multiple tasks spread across the cycle,” he said.
That means if a team already has mature agile development and DevOps practices in place, introducing AI pair programmers may only lead to a 10 to 20% overall productivity gain in cycle time – others companies will see less.
Even so, ROI on these tools is relatively easy to deliver, he pointed out. With a tool like GitHub Copilot for Business available for $19 a month, even a 5% productivity gain you are effectively adding another developer to the team for $400 a month.
“There are real productivity gains to be had here and real improvements to developer experience,” Walsh said.
Gartner’s survey took in responses from 598 participants from around the world in Q3 2023.
Risks and downsides of AI pair programmers
With a debate over whether AI code generation can replace human developers ongoing, leaders are . One potential risk is that junior developers could be more reliant on AI code output even if it’s not better code than a human could produce. Recent analysis found that tools such as GitHub Copilot could be generating code containing dangerous flaws.
Walsh told ITPro that the whole idea of a pair program is that you are working with someone who questions you as much as they help you, which current AI tools cannot do.
“The thing is, the AI will never challenge you. It will always just affirm the path you are on and help you down it,” he said – whether that’s a good idea or not.
“The real value of your developers is [to be] creative critical problem solvers who look at the requirements and go back to the business stakeholders and say: ‘Are we even asking the right questions here?’” said Walsh. “The coding, that’s just the typing. The bottleneck on productivity was never keystrokes, it’s problem solving.”
While some research has suggested these tools can make developers much more efficient, completing tasks 16% to 50% faster, other research has suggested that these tools may increase so-called software churn by making it easier for less senior developers to add code – which may then need to be modified later. Developers themselves would like to use AI assistants for jobs like writing code comments and documentation and writing tests, according to a survey by JetBrains.
Evaluating the risk of AI pair programmers
Gartner said software engineering leaders must determine the return on investment (ROI) and build a business case as they scale their rollouts of AI pair programmers. Because traditional ROI frameworks steer engineering executives towards metrics focused on cost reduction, this narrow perspective fails to capture the full value of AI
To build an effective ‘value story’ that goes beyond those traditional ROI metrics, software engineering leaders must reframe the ROI conversation from cost reduction to value generation.
That means going beyond the obvious benefits, like faster coding leading to time saving and cost reduction, and to look at broader benefits. Gartner said these could include:
- Reduced task switching to help developers maintain their ‘flow’ state
- Enhanced developer experience
- Reduced staff turnover
- Improved code quality and maintainability
- Improved digital experience
- Reduced bugs and technical debt
- Faster go-to-market
“Calculating time savings on code generation is a good place to begin building a more robust value story,” said Walsh. “To convey the full enterprise value story for AI code assistants, software engineering leaders should connect value enablers to impacts, and then analyze the overall return to the organization.”
AI developer tools have been one the early and most high-profile successes of generative AI, helping companies to make developers more effective.
According to StackOverflow’s 2023 Developer Survey, GitHub Copilot, which hit general availability in June 2022, is the most used developer AI tool. GitHub has previously said more than 50,000 organizations are using its tool – including one in three Fortune 500 companies. Google Cloud’s Gemini Code Assist and Meta’s Code Llama are competitors in the space, while recent reports state Apple is working on an AI coding tool of its own.
