AI is creating more software flaws – and they're getting worse
A CodeRabbit study compared pull requests with AI and without, finding AI is fast but highly error prone
AI is helping developers write code faster, but it's also leading to more problems, marking the latest study to highlight adoption concerns among software developers.
In a study from CodeRabbit, researchers found AI makes 1.7 more times as many mistakes as human programmers.
To come to that conclusion, CodeRabbit looked at 470 open source GitHub pull requests, of which 320 had AI input and 150 were human only.
"The results? Clear, measurable, and consistent with what many developers have been feeling intuitively: AI accelerates output, but it also amplifies certain categories of mistakes," wrote CodeRabbit's director of AI David Loker in a blog post.
He admitted the study wasn't perfect, in part because it was difficult to double-check authorship, but said the findings matched with previous research. Loker pointed to a report by Cortex that found pull requests per author increased by 20%, but incidents per pull request also went up — by 23.%.
That matches with previous research suggesting AI-generated code is now the cause of one-in-five breaches, with another report noting coders' concerns around AI introducing errors to their work.
More problems and more critical flaws
The CodeRabbit study found 10.83 issues with AI pull requests versus 6.45 for human-only ones, adding that AI pull requests were far more likely to have critical or major issues.
Sign up today and you will receive a free copy of our Future Focus 2026 report - the leading resource for IT decision-maker insight on priorities and investment areas in AI, security and more.
"Even more striking: high-issue outliers were much more common in AI PRs, creating heavy review workloads," Loker said.
Logic and correctness was the worst area for AI code, followed by code quality and maintainability and security. Because of that, CodeRabbit advised reviewers to watch out for those types of errors in AI code.
"These include business logic mistakes, incorrect dependencies, flawed control flow, and misconfigurations," Loker wrote. "Logic errors are among the most expensive to fix and most likely to cause downstream incidents."
AI code was also spotted omitting null checks, guardrails, and other error checking, which Loker noted are issues that can lead to outages in the real world.
When it came to security, the most common mistake by AI was improper password handling and insecure object references, Loker noted, with security issues 2.74 times more common in AI code than that written by humans.
Another major difference between AI code and human written-code was readability. "AI-produced code often looks consistent but violates local patterns around naming, clarity, and structure," Loker added.
Beyond those issues, Loker noted flaws in concurrency, formatting, and naming inconsistencies — all of which could not only cause issues with software or apps, but also make it harder for human reviewers to spot problems.
What's happening, and how to fix it
Some of the faults are down to a lack of context, Loker suggested, and in other cases AI is sticking to "generic defaults" rather than company rules, such as with naming patterns or architectural norms.
Security problems may be down to how models recreate "legacy patterns or outdated practices" by using older code as training data.
"AI generates surface-level correctness: It produces code that looks right but may skip control-flow protections or misuse dependency ordering," Loker added.
None of this means AI shouldn't be used in coding, he stressed. Instead, companies should ensure AI has context of business rules, policies or style guides, and be given guidelines for basic security.
Such efforts would wipe out many of the faults spotted, he said. Similarly, code reviewers should be given a checklist that takes this research into account, so they know what types of errors to watch out for.
"The future of AI-assisted development isn’t about replacing developers," he added. "It’s about building systems, workflows, and safety layers that amplify what AI does well while compensating for what it tends to miss."
FOLLOW US ON SOCIAL MEDIA
Make sure to follow ITPro on Google News to keep tabs on all our latest news, analysis, and reviews.
You can also follow ITPro on LinkedIn, X, Facebook, and BlueSky.
Freelance journalist Nicole Kobie first started writing for ITPro in 2007, with bylines in New Scientist, Wired, PC Pro and many more.
Nicole the author of a book about the history of technology, The Long History of the Future.
-
TfL takes Oyster photocard systems offline following breachNews Security experts warn the Oyster photocard breach could lead to follow-up attacks on victims
-
Irish credit unions hit by data breach after IT provider compromisedNews Wellington IT reportedly delayed alerting customers by multiple weeks
-
OVHcloud launches European-trained open source AI modelNews The new lightweight LLM can run on standard desktop CPU, with plans to expand context window to 128,000 tokens
-
‘Organizations need to help developers grow alongside the technology’: Dejected devs lament career prospects amid AI advancesNews A new survey has found that AI is fundamentally changing the developer role and many are getting anxious
-
Vibe coding startup Cursor just launched a GitHub alternative – here's what users can expectNews Cursor could muscle in on GitHub’s dominance, but the startup has its work cut out
-
Software teams should take a leaf out of manufacturers books when it comes to testing codeNews Software testers are struggling to keep up with the pace of code production. UiPath thinks it has the solution
-
Microsoft may be making AI work — finally boosting its share priceMicrosoft's quarterly results featured better than expected figures on AI and cloud
-
'It delivers world-class performance at 50 percent of the cost of leading models': Microsoft unveils cut-price AI for security with latest in-house model launchNews Pairing the MAI security model with GPT-5.4 gives benchmark leading results at half the cost, according to the tech giant
-
Cisco just launched two cyber-focused small language models: Antares-350M and Antares-1B aim to supercharge codebase analysis – and they run at a “fraction of the compute expense” of popular frontier modelsNews The Antares models unveiled by Cisco aim to cut costs in codebase analysis
-
Cyber professionals are flocking to AI tools, but they’re getting tired of fixing mistakes and reviewing outputsNews Cyber pros are spending significantly more time validating AI outputs and deciding when to trust AI-generated recommendations