‘It’s time to question agile’s cult following’: Doubts cast on method’s future, with 65% of projects more likely to fail

The shine is wearing off for agile development practices, with new research showing that projects adopting this method are far more likely to fail. 

The claims follow the publication of a survey of more than 600 UK and US-based software engineers for a new book, Impact Engineering. The study found that agile software projects are 268% more likely to go wrong than those employing other methods.

65% of projects weren’t delivered within budget, the study noted, while quality standards were also found lacking.

Meanwhile, the survey found the majority of UK and US business leaders are concerned about the timely delivery of projects. And the success rates for transformation initiatives are low, with 96% of agile transformation projects failing.

The research also revealed that software engineers in the UK were 13% less likely to feel they were able to discuss and address problems than those in the US - the biggest difference of all engineering practices between the two countries.

"With 65% of projects adopting agile practices failing to be delivered on time, it’s time to question agile’s cult following," said author Dr Junade Ali, a cyber security consultant.

Agile development has its flaws

Practices listed in the Agile Manifesto include 'Working software over comprehensive documentation', 'Customer collaboration over contract negotiation' and 'Responding to change over following a plan'. 

However, the survey found projects which had a specification or requirements documented before development started were 50% more likely to succeed than those which didn’t.

Projects which didn't require significant changes to requirements late into the development process were 7% more likely to succeed.

Other practices were also said to increase success. Those where the software engineer reported feeling psychologically safe to discuss and address problems quickly when they emerged were 87% more likely to succeed compared to those which didn’t.

Similarly, projects where requirements were accurately based on a real-world problem were 54% more likely to succeed.

Interestingly, the study found no statistically significant difference between project success for those working on one project versus those working on several - despite 83% of software engineers reporting feeling burnout.

Research earlier this year showed that agile development was fading in popularity at large enterprises, and a key factor here was the significant toll this method inflicts upon developers, with many reporting burnout as a result of tight deadlines.

"Our research has shown that what matters when it comes to delivering high-quality software on time and within budget is a robust requirements engineering process and having the psychological safety to discuss and solve problems when they emerge, whilst taking steps to prevent developer burnout," Ali said.

The report echoes the conclusions of another recent book, 'Agile as a Micromanagement Tool', by ex-Google software engineer Murat Guler. In one chapter, Guler described how poor management practices can potentially become a tool for micromanagement as managers implement new processes.