“I couldn’t imagine it being more impactful,” Jon Kern, co-author of the Agile Manifesto tells ITPro. 25 years on and Kern firmly believes the manifesto – and methodology at large – has withstood the test of time.

Alongside 16 other co-authors, Kern helped curate the Agile Manifesto in February 2001 during a series of crunch meetings at Snowbird ski resort in Utah. The goal was simple: streamline development by combatting “heavyweight processes” which burdened devs and slowed down software delivery cycles.

“It promoted lighter, easier ways of developing software by capturing what a small group of practitioners had in common, and making those attitudes available to others who were struggling under heavyweight approaches,” he told ITPro.

The inspiration behind the manifesto lay in long-running gripes with traditional development approaches such as Waterfall. According to Jon Rimmer, CXO at Mercator Digital, this approach was rigid, inefficient, and at times painfully slow.

A key factor here is that under this methodology, teams worked through phases in a linear fashion, with each of these required to be completed before moving on.

“The traditional ‘Waterfall’ approach meant you could spend six months planning - and then a further three years building - only to be completely divorced from the end users and the real environment,” Rimmer told ITPro.

With Agile development, however, flexibility and adaptability were key focus areas. The 12 central principles outlined in the manifesto prioritized an array of factors, including faster delivery times through now-iconic ‘sprints’, rolling testing processes, continuous feedback, and smaller, more efficient teams.

Martin Reynolds, field CTO at Harness, told ITPro that the impact of Agile was “huge” when it hit the scene, adding that it highlighted and emphasized what efficient teams were already doing at the time.

“I’m old enough to remember when the Agile Manifesto launched – I was coding at the time – and its impact was huge,” he said.

“What it really did wasn’t to invent anything radically new, but codify what the best teams were already doing,” Reynolds added. “It gave the industry a common framework to move away from Waterfall, long release cycles, heavy documentation upfront, and very little ability to change course once things were underway.

“I can vividly remember projects where half the time was spent writing specs, only to hope everything fitted together months later. Agile flipped that on its head.”

What did Agile development actually improve?

Key focus areas of the Agile Manifesto helped drastically simplify software development, Reynolds noted.

By moving teams to smaller more regular releases, for example, this “shortened feedback loops” typically associated with Waterfall and improved flexibility throughout the development lifecycle.

“That reduced risk made it easier to respond to customer and business needs, and genuinely improved software quality,” he told ITPro. “Smaller changes meant testing could happen continuously, rather than being bolted on at the end.”

The longevity of Agile methodology is testament to its impact, and research shows it’s still highly popular. A survey from Forrester in early 2025 found 95% of respondents highlighted Agile practices as critical to their organization.

Notably, 58% of respondents said they’re still prioritizing adoption of Agile practices, underlining its continued popularity among business and technology professionals.

Manageable drawbacks

That’s not to say Agile doesn’t have some drawbacks, however. Reynolds noted that areas such as technical debt can become a burden due to the rapid nature of this approach.

“When teams are under constant pressure to deliver new features, debt can easily be pushed to the bottom of the backlog,” he told ITPro. “The problem is it doesn’t age well – it’s not like wine. It’s more like milk: the longer you leave it, the worse it smells.”

Moreover, scaling Agile across larger organizations has proved troublesome over the course of its lifespan. As ITPro previously reported, Digital.ai’s 2024 State of Agile survey found larger enterprises encountered significant challenges embracing Agile.

Simply put, the larger the company, the more fragmented teams become. This results in a scattered approach where some teams fully embrace the methodology while practices fall by the wayside in others.

Agile in the age of AI

With the rapid enterprise adoption of generative AI over the last three years, Agile methodology has once again been thrust into the spotlight.

Findings from Digital.ai’s 2025 State of Agile Report found AI is now helping supercharge Agile development, with teams using an array of tools to underpin and support processes.

More than three-quarters of respondents said they’re using AI to “save time or reduce manual effort”, for example, using the technology to auto-generate documentation or summarize retrospectives.

According to Kern, AI and Agile are “a match made in heaven” and the advent of the technology means this approach is no longer optional, albeit with a notable caveat.

“You need it more than ever,” he said. “You can build so much more in less time, which can also magnify potential pitfalls if you’re not careful. The speed of delivery with AI can easily outpace feedback, but that’s an exciting opportunity, not a flaw.”

Reynolds echoed those comments, noting that while Agile can be a force multiplier for teams, there are still risks – particularly with the influx of AI-generated code in software development.

“Those gains are often offset downstream, creating more bugs, higher cloud costs, and greater security exposure. The real value comes when AI is extended beyond code creation into testing, quality assurance, and deployment,” he said.

“Applied thoughtfully, it strengthens the fast feedback loops that Agile depends on and helps teams adapt more confidently as business requirements change,” Reynolds added.

“In that sense, AI doesn’t replace or outdate Agile – it reinforces the principles that have kept it relevant for the past quarter of a century and will ensure it remains so long into the future.”

FOLLOW US ON SOCIAL MEDIA

AI is fueling a drastic transformation of Agile development practices, according to new research, as enterprises target more efficient software delivery.

Findings from the 18th annual State of Agile Report, published by Digital.ai, suggest the methodology has reached a “major turning point” as AI moves from being a “supportive tool to an orchestrator” in delivery cycles.

“Artificial intelligence is transforming software development and delivery faster than any wave before it,” Digital.ai CEO Derek Holt commented. “In the last three years, AI has moved from research labs into every corner of the enterprise, reshaping how we plan, build, test, secure, and release software.”

The common talking points and appeal of AI tools, such as enhanced productivity, go hand in hand with the basic fundamentals of Agile methodology: more efficient development practices.

This emerging symbiotic relationship between AI and Agile comes at a critical time, the report noted. AI adoption among respondents has surged from 64% in last year’s edition to 84% this year.

With questions over return on investment (ROI) looming, IT leaders are being pushed to both justify investments in the technology and deliver positive business outcomes.

Running parallel to this AI ROI focus is rising pressure on tech leaders and developer teams alike to deliver ROI with Agile itself. More than three-quarters (76%) of respondents cited “increased scrutiny” on whether the methodology produces tangible benefits.

The study noted that this pressure is “reshaping investments and role assignments”, yet only half feel they’re capable of delivering software projects reliably and on time.

Elsewhere, teams are being stretched thin, with 79% revealing they’re being “asked to do more with less”. More than three-quarters (77%) said they feel increased pressure to accelerate innovation.

Notably, 78% of respondents said they face challenges adapting to repeated leadership and “structural changes” at their organization, which is having an adverse effect on efficiency and coherence.

Agile is adapting to a changing environment

While teams face mounting pressure, the report notes that the methodology “continues to evolve to meet modern demands”.

Investment in the practice is rising, the study found, with 41% of respondents increasing spending on this front over the last two years. This, Digital.ai noted, shows “continued confidence” in its value within the enterprise.

Flexibility is also another key positive takeaway from the report, with 74% now using hybrid, blended, or “homegrown” Agile models based on their individual needs and preferences.

Elsewhere, increased accountability and cross-functional collaboration are delivering positive results. Nearly one-third (29%) of respondents are now “accountable for connecting Agile work to business outcomes and 26% are influencing product and portfolio planning,” the study found.

Digital.ai said this means enterprises are shifting from focusing primarily on basic activity metrics to measuring broader long-term strategic impact.

Why AI could be a game changer for Agile

So where does AI actually fit into the equation with supporting Agile development practices? As with other business areas, the technology is being used as a force multiplier by teams, helping to drive productivity and streamline processes.

Over three-quarters of respondents said they’re using the technology to “save time or reduce manual effort” and simplify tasks by auto-generating documentation or summarizing retrospectives.

Teams are excited about the long-term prospects of integrating the technology within Agile workflows, the report found.

“The broader opportunity is even more compelling,” the study states. “When we asked all respondents, including those not yet using AI where it could help in their roles, the appetite extends well beyond efficiency.”

“Nearly nine-in-ten see value in detecting delivery risks earlier and surfacing real-time insights.”

Teams have a ‘pragmatic’ optimism with AI

Over the last two years, concerns about AI integration actively impacting workers have gained significant traction. In the case of Agile, however, respondents said they welcome the influx of AI tools as a way to enhance teams.

“Even with automation accelerating, the mood is overwhelmingly positive,” the study noted.

“While some worry AI could disrupt or reduce their roles, the majority see AI as enhancement rather than threat, believing it will change how they work but not replace what they do, or that it will enhance their productivity and decision-making.”

While optimism prevails across the majority of respondents, Agile teams are keen to ensure that the technology is integrated in a responsible manner.

Considerations such as security, privacy, and compliance all ranked among the top issues flagged by respondents, Digital.ai found, while the quality of AI outputs also emerged as a lingering concern.

A key factor here lies in the quality of data at the disposal of teams, with one-in-five revealing they “don’t trust their data at all” – an issue that’s affecting confidence.

“The skepticism about AI outputs makes sense,” the study said. “These systems hallucinate, present false information as fact, and need validation.”

“AI isn’t infallible – it requires governance, human oversight, and clear boundaries around where automation is safe and where human judgement remains essential.”

MORE FROM ITPRO

• Doubts cast on Agile's future, with 65% of projects more likely to fail
• Agile development has a security problem
• Four major software development lifecycle models and how they work

Java developers are encountering significant productivity barriers, according to new research, prompting businesses to take drastic measures to boost efficiency.

Perforce Software’s 2025 Java Developer Productivity report revealed barriers to Java developer productivity can take on many forms, but the most significant issue hurting their ability to work effectively was insufficient documentation (41%).

Lacking robust documentation means devs have to spend more time initially getting to grips with the code, architecture, or tools they are working with, bogging down their ability to actually get to the task at hand.

It can also feed into various further friction points when conducting maintenance and debugging, where the developer is not able to understand the previous decisions made during development and thus more time is taken to troubleshoot issues.

Communication issues between teams were also a common problem cited by businesses, with 38% of developers reporting it was hurting their productivity.

Bad communication can lead to improper implementation or developers misunderstanding the requirements of the project or feature they are working on, forcing them to spend time reworking and remediating mistakes.

Other commonly cited barriers to Java developer productivity were mismanaged timelines (32%), long redeploy times (29%), developer turnover (26%), and insufficient developer tools (24%).

Speaking to ITPro Martin Reynolds, field CTO at Harness, said many businesses face developer productivity problems. A key factor here, he noted, is that a significant portion of software developer’s time is not spent coding but on other ancillary tasks, or even simply waiting in some instances.

“About 40% of software developer time is toil not related to writing code – whether it’s waiting for builds to complete, being on call to troubleshoot, and resolving security issues,” he explained.

“A lot of the perceived productivity problems developers face stem from process inefficiencies, bottlenecks during testing and quality control, and a lack of automation across the software delivery lifecycle (SLDC).”

Development barriers strike businesses “with equal opportunity”

Generally, half of the respondents said their companies are already planning on bolstering Java teams with new developers in the coming year, but ensuring that the productivity of these expanded teams remains high is essential.

Firstly, 34% of respondents said their employer also plans on increasing their Java tool budgets this year, but how will they decide which tools are actually helpful?

When asked how they will improve developer productivity, 47% said they had been individually tasked with investigating which tools might work best, with 38% reporting they had been put into work groups for the same purpose.

This group-based approach appears more popular at larger enterprises, with the figure rising to 49% when restricted to developers working at firms with over 1,000 employees.

The most popular method to boost developer productivity was using AI tools, cited by 50% of respondents, with code completion (60%) and refactoring (39%) being the top use cases highlighted by devs.

Notably, however, 12% of respondents stated they do not use AI tools in their workflow, with a further 12% reporting that their organization was still blocking the use of AI for development purposes.

Perforce noted that giving developers access to the latest tools will help them maintain high output as other factors continue to limit their productivity.

“[Businesses] that arm their teams with appropriate tools will be the ones that will succeed working in increasingly complex development environments and demanding business conditions.”

Regardless of how businesses choose to overcome these development barriers, the report insisted they “strike with equal opportunity against businesses of all sizes”.

It added that certain approaches may help address particular challenges, others will require a more wholesale change to the organization in order to be remediated.

“While issues like insufficient documentation can easily be addressed with AI tools, long redeploy times and insufficient developer tools require a more holistic approach to realize time savings and business value.”

MORE FROM ITPRO

• Junior software developers lack coding skills because of an overreliance on AI tools
• Flaws in a popular dev library could let hackers run malicious code in your MongoDB database
• Turns out AI isn't that popular at work – just 4% of workers use the technology in the majority of daily tasks

Software development leaders say their organization did not have a robust incident response plan in place to contend with disruption before the 2024 CrowdStrike outage.

A report from IT services provider Adaptavist surveyed development leaders on how their organization reacted to the CrowdStrike outage in July 2024, which affected an estimated 8.5 million devices.

The respondents came from organizations with over $10 million in revenue based in the UK, US, and Germany.

The incident has been reported to have cost US Fortune 500 companies $5.4 billion, between £1.7 and £2.3 billion to the UK economy, and affected 98% of companies according to Adaptavist’s research.

Looking back on the event, 84% respondents admitted that their organization did not have an adequate incident response (IR) plan in place before the outage.

Of those who said they did have a plan in place, only 16% reported that they were actually effective in helping them recover from the fallout caused by the event.

Adaptavist noted that most organizations had never experienced a large-scale outage like that before, or the consequences they can have on their IT operations and, as such, struggled to understand what would be required to minimize the impact of such an incident.

But the report also found that 80% of organizations claim to have drawn positive outcomes from the crisis, which brought about what it described as a “remarkable change among businesses”.

How the CrowdStrike outage forced changes in IT resilience

Adaptavist's report claimed the CrowdStrike outage “opened IT leaders’ eyes to the wider robustness of their entire software development practices and processes”.

For example, 81% of respondents said they adopted more robust software development methodologies as a result, with one-third stating they totally overhauled their engineering processes to prevent similar disruptions moving forward.

Among the list of changes leaders made to their development practices, 54% committed to IR planning, with just under half identifying monitoring and observability, unit testing, integration testing, CI/CD, and agile methodologies, as other key change areas.

Other frequent answers included manual code reviews (47%), grey box testing (47%), white box testing (46%), and system testing (45%).

In addition to operational changes, Adaptavist found the CrowdStrike outage also prompted changes in how organizations invest in software development. It reported 86% of businesses had boosted investment in software development practices and training as a direct response to the incident.

The key investment areas listed by the respondents were agile and DevOps practices and training (89%), software testing training (89%), cybersecurity training (88%), and IR training (86%).

The incident also sparked a hiring spree, Adaptavist noted, with 99.5% of organizations stating they plan on expanding their technical teams, with quality assurance roles (36%), IT operations (34%), software developers (32%), and DevOps engineers (31%) top of the list for recruitment.

Businesses still plagued by outdated development culture

Yet cultural challenges remain, the report cautioned, with 44% of IT leaders stating that their organization still prioritizes speed over quality in software development.

Furthermore, just under two-fifths of leaders said they worry that their team’s excessive workloads will lead to another major incident.

A quarter of respondents warned their organization still advocates for a culture of fear over learning to reduce risk, with 40% stating they still fear acknowledging their mistakes.

A lack of psychological safety is also hindering innovation at their organization, according to 44% of IT leaders, with 42% claiming their firm’s security is compromised by a fear of admitting one’s mistakes.

Adaptavist suggested that skills shortages are only making this problem worse, advising businesses to invest in building a culture that seeks to address burn out and stress among engineers to attract talent.

“The ongoing war for IT talent is likely exacerbating these issues, but building a culture which invites open and honest collaboration, and addresses the issue of over-worked and over-blamed teams is the key to creating a welcoming environment for more IT professionals and delivering a far more resilient, efficient and secure IT environment,” the report advised.

AI projects fail because of technical challenges, people misunderstanding what's possible, and agile development, according to research from the RAND institute.

Research cited by RAND suggests as many as 80% of AI projects fail — that's twice the rate of other technology projects and a serious problem for the industry given the costs involved with AI. RAND notes that the US Department of Defense is spending $1.8 billion annually on military AI applications.

RAND researchers interviewed 65 AI experts, revealing five key causes for failure — what the research institute calls "anti-patterns of AI".

As is often the case, while some of the challenges relate to the technology itself, others have more to do with people.

"AI projects have two components: the technology as a platform (i.e., the development, use, and deployment of AI to complete some set of business tasks) and the organization of the project (i.e., the process, structure, and place in the overall organization)," the report notes.

"These two elements enable organizations and AI tools to work together to solve pressing business problems."

IT projects fail for many reasons beyond technology, RAND notes, pointing to poor execution, problems with how users interact with the end results, and troubles meeting high expectations.

"However, AI seems to have different project characteristics, such as costly labor and capital requirements and high algorithm complexity, that make them unlike a traditional information system," the report states.

"The high-profile nature of AI may increase the desire for stakeholders to better understand what drives the risk of IT projects related to AI."

Five points of failure

To start, RAND reports that the problem to be solved using AI isn't always well understood or communicated, so people simply don't address the real challenge. Because leadership lacks the detailed technical skills, and the data science team doesn't have the full business context, models can be developed that are just enough off point to be useless.

"[They] do not realize that the metrics measuring the success of the AI model do not truly represent the metrics of success for its intended purpose," the report notes.

As an example, the report describes business leaders wanting an algorithm to set the price for a product, but what they actually need is to know the price that offers the greatest profit margin rather than sell the most items.

"The data science team lacks this business context and therefore might make the wrong assumptions," the paper says.

Projects fail because they lack sufficient data to properly train an AI model. Organizations may collect data for compliance or to track sales figures, but that may not be the right sort of data to feed into a behavioral algorithm's many fields, for example.

"For example, an e-commerce website might have logged what links users click on — but not a full list of what items appeared on the screen when the user selected one or what search query led the user to see that item in the first place," the paper says.

Similarly, organizations focus more on using the "latest and greatest" technology than solving the problem at hand, which may need a different technology or perhaps not require the most up to date solution.

"Not every problem is complex enough to require an ML solution: As one interviewee explained, his teams would sometimes be instructed to apply AI techniques to datasets with a handful of dominant characteristics or patterns that could have quickly been captured by a few simple if-then rules," the researchers said.

The fourth issue highlighted in RAND’s report centers around the fact that organizations may lack the infrastructure to manage data and deploy an AI model, dooming the project to failure.

Supporting infrastructure can include staff with the right skills, as well as operations infrastructure to ensure quick and easy deployment.

"Some interviewees noted that they had observed cases where AI models could not be deployed from test environments to production environments because the production environments were incompatible with the requirements of the mode," the report said.

And lastly, sometimes AI simply can't solve the problem. Senior leaders have "inflated expectations" of what AI is actually capable of, exacerbated by hype-filled pitches from sales people.

"In reality, optimizing an AI model for an organization’s use case can be more difficult than these presentations make it appear," the researchers said, with the researchers adding that many leaders expect AI projects to take weeks instead of months or longer to complete, meaning many are delivered partially complete.

The agile development conundrum in AI projects

Beyond those five general concerns, RAND also raised issues with agile development, a software development methodology that breaks projects down into phases to work on flexibly.

The researchers said a large number of interviewees said that agile software development can be a poor fit for AI projects.

That's partially because of how rigidly some companies interpret agile processes, but also because of the short time scale for developing aspects of a system, known as sprints.

AI development may not fit into existing project management schedules because initial phases of data exploration and experimentation have an "unpredictable duration", the researchers said.

"Rigid interpretations of existing software development processes rarely suit the cadence of an AI project," the researchers found.

"Instead of forcing project teams to follow a uniform set of procedures designed for a different type of engineering, organizations should empower their teams to adapt their processes to fit their workloads. Ultimately, organizations will need to rediscover how to make the agile software development process be adaptive and — truly — agile."

What to do?

Some of the challenges are practical. Partnerships on data could ensure there's enough to train a model properly, while investing in infrastructure will help support AI projects.

Plus, RAND advises leaders to be patient: teams will need to be committed to a project for at least a year to see results.

More generally, organizations need to ensure they understand AI's limitations, clearly communicate the intent and purpose of a project, and focus on the problem to be solved, and choose the right technology rather than jump on the AI bandwagon.

The shine is wearing off for agile development practices, with new research showing that projects adopting this method are far more likely to fail. 

The claims follow the publication of a survey of more than 600 UK and US-based software engineers for a new book, Impact Engineering. The study found that agile software projects are 268% more likely to go wrong than those employing other methods.

65% of projects weren’t delivered within budget, the study noted, while quality standards were also found lacking.

Meanwhile, the survey found the majority of UK and US business leaders are concerned about the timely delivery of projects. And the success rates for transformation initiatives are low, with 96% of agile transformation projects failing.

The research also revealed that software engineers in the UK were 13% less likely to feel they were able to discuss and address problems than those in the US - the biggest difference of all engineering practices between the two countries.

"With 65% of projects adopting agile practices failing to be delivered on time, it’s time to question agile’s cult following," said author Dr Junade Ali, a cyber security consultant.

Agile development has its flaws

Practices listed in the Agile Manifesto include 'Working software over comprehensive documentation', 'Customer collaboration over contract negotiation' and 'Responding to change over following a plan'. 

However, the survey found projects which had a specification or requirements documented before development started were 50% more likely to succeed than those which didn’t.

Projects which didn't require significant changes to requirements late into the development process were 7% more likely to succeed.

Other practices were also said to increase success. Those where the software engineer reported feeling psychologically safe to discuss and address problems quickly when they emerged were 87% more likely to succeed compared to those which didn’t.

Similarly, projects where requirements were accurately based on a real-world problem were 54% more likely to succeed.

Interestingly, the study found no statistically significant difference between project success for those working on one project versus those working on several - despite 83% of software engineers reporting feeling burnout.

Research earlier this year showed that agile development was fading in popularity at large enterprises, and a key factor here was the significant toll this method inflicts upon developers, with many reporting burnout as a result of tight deadlines.

"Our research has shown that what matters when it comes to delivering high-quality software on time and within budget is a robust requirements engineering process and having the psychological safety to discuss and solve problems when they emerge, whilst taking steps to prevent developer burnout," Ali said.

The report echoes the conclusions of another recent book, 'Agile as a Micromanagement Tool', by ex-Google software engineer Murat Guler. In one chapter, Guler described how poor management practices can potentially become a tool for micromanagement as managers implement new processes.

The advantages of agile development have become increasingly clear in recent years, enabling developers and organizations to more efficiently build and roll out products to market. 

But in the rush to accelerate application development, developers have often overlooked critical considerations such as security, according to research.

A study from the Information Security Forum (ISF) suggested that agile development methods “do not always explicitly contemplate the need for information security, even though security should be at the forefront”.

Failing to acknowledge security considerations, the ISF warned, could present opportunities for exploitation by threat actors, enabling hackers to target rushed applications with lax security.

Gunnar Braun, technical manager at Synopsys Software Integrity Group, told ITPro that the very nature of agile development practices means that security can be overlooked frequently - but this doesn’t suggest a complete disregard for product safety.

“The fundamental concept of agile development is to work in small iterations - on one (or a few) features at a time by completing the cycle of writing code, testing it, and deploying it, so it can be consumed by an internal or external customer for feedback,” he said.

The success of security within this process depends on “whether it is considered a feature, or a property of a feature”, Braun added. If security is treated like a feature, he explained, then it competes against others in the development backlog and therefore is “likely going to lose the race”.

“Let's take the example of input validation,” he explained. “What is the product owner more likely to prioritize? Input validation or a shiny UI improvement that has been requested by a key customer?”

Braun noted, however, that in instances where security is part of the requirements of a feature, it “becomes a property of that feature”.

“Its implementation will not be considered complete before it implements proper input validation. Consequently, security is becoming integrated with the agile development process,” he explained.

Embedding security in agile development can be highly challenging

A key challenge for implementing security in an agile development process is that it requires “special knowledge that is often not available” or easily accessible for the development team, Braun said. 

The consequence of this is that vital security considerations are therefore likely to be excluded from sprints throughout the process. This reinforces the need for a more collaborative relationship between developers and security practitioners across the entirety of the development lifecycle.

The 2023 Global State of DevSecOps report highlighted “inadequate/ineffective security training” for developers and engineers as one of the leading barriers to successfully implementing DevSecOps.

Braun said that a strategy that’s proven to be effective in addressing this is the creation of a “security champions” program.

“Security champions can evolve from existing roles, for example agile coaches or DevOps engineers, and can support multiple agile development teams with security domain knowledge,” he explained.

Braun added that the benefits of this practice have been confirmed by findings from the Building Security in Maturity Model report, which found that teams with security champions score 25% higher on average than those without one.

Security can be “naturally integrated” within agile development

Braun said he believes agile development presents an opportunity for security to become more intricately woven throughout the entirety of the software development lifecycle.

However, there are challenges here. Fundamentally, developers need to view security as a “property of a feature”. Similarly, leaders should embrace the expertise that security practitioners can offer projects.

In creating a more collaborative relationship between developers and security practitioners, teams will ultimately become more self-sufficient, and this could help improve efficiency to ensure products are rolled out to market at a faster pace.

“Agile development is an opportunity for security to become naturally integrated with the software development process,” he said.

“To achieve this, it is important to treat security as a property of a feature, automate security tests in the same way as other tests, and provide security expertise to the development team to make the teams self-sufficient and own every aspect of the development process.”

Agile development methodology is facing significant obstacles as the tech industry goes through a wave of changes including developer burnout, shifting working environments, and the rise of AI.

Agile is a software development methodology first formulated back in 2001 which aims to deliver software faster by breaking monolith projects up into smaller elements. These can then be completed in ‘sprints’ of weeks or months which are monitored at daily stand-up meetings to check on progress.

The idea is that developers and business execs should work together daily on projects and that changes should be welcomed even at a late stage in a project.

Rapid iterations – and, if necessary, failures – are the best way to get projects finished, according to this approach.

“Working software is the primary measure of progress,” is one of the 12 principles of the Manifesto for Agile Software development. All of this stands in stark contrast to the ‘waterfall’ approach which was dominant previously.

This values lots of up-front planning and projects delivered in a systematic fashion - which critics argue leaves less room for adapting to change - and a greater chance of total failure.

But now a ‘State of Agile’ survey from software development company Digital.ai shows that while agile has many fans, particularly in smaller firms, it is struggling to make an impact in larger organizations.

“From AI to developer burnout, hybrid work environments and unrelenting demand, change is happening in every organization in every industry. At this moment in time, it feels like agile is having difficulty adapting,” the report said.

The study found that while smaller organizations continue to consider agile as a powerful productivity and organizational framework that exhibits “obvious benefits”, medium-sized and larger companies are less satisfied with what agile can do for them, and are more likely to pick a software development strategy that uses a number of different frameworks.

Use of agile development is still widespread

According to the survey of 788 tech workers, 69% of respondents said their IT, software development, and delivery teams use agile, as do nearly half of engineering, product, and R&D teams (48%), along with 28% of business operations and 20% of marketing teams.

Roughly 44% of respondents said enterprise agile works very or somewhat well for their organizations. Smaller companies were more likely to say they were happy (52%) compared to 43% of larger businesses.

Over two thirds (71%) of respondents said they were using agile in their software development lifecycle, while 42% said their organizations use a hybrid model that includes agile, DevOps, and others.

“The larger the organization, the more likely it is to use a hybrid model,” the report said, noting that bigger teams are also more likely to still use waterfall.

The Digital.ai report said the most successful agile implementations are in small companies. Three quarters of small companies (compared to 62% of large companies) said at least 50% of their applications were delivered on time and ‘with quality’.

But while in last year’s survey 72% of respondents said they were satisfied with agile practices in their company, this year’s study found that has dropped to 59%.

The biggest reason for this is that nearly half of survey respondents (47%) blamed a generalized resistance to organizational change or culture clash – up seven points from 2022.

They also blamed a lack of leadership participation and a failure of management support, and while a third said that business teams did not understand what agile did or could do.

Respondents who were unhappy with agile adoption rates blamed “too many mixed systems” in their companies, while second on the list for blame was siloed teams causing a culture clash and inconsistent use of agile.

Agile has been gradually spreading beyond software development, particularly as a result of demand for broader digital transformation projects which are viewed by CIOs as a way of injecting a little of the excitement and energy needed to crack open legacy systems and find new value.

But for organizations wanting to move agile from team level to company-wide, there are still a number of problems ahead.

One-in-three blamed leadership figures “not understanding and putting roadblocks up, either knowingly or unknowingly”, closely followed by “teams not empowered to be self-organized and/or self-sufficient”.

People in medium and large companies are more likely than those in small ones to say their challenge is that everything is siloed.

“What’s clear from the data is that, when agile works, it works – there are concrete benefits for the organizations who have gotten it right,” said Derek Holt, Digital.ai’s CEO.

“Agile still provides us with the best opportunity to manage these transitions and drive software delivery toward maximum business value.”

Platform engineering has become somewhat of an industry buzzword in recent years as organizations look to rapidly accelerate the delivery of applications and unlock greater business value at scale. 

This emerging discipline, as Gartner describes it, helps markedly improve developer experience and productivity by “providing self-service capabilities with automated infrastructure operations”. 

Simply put, platform engineering centers around enabling developer teams to focus on building and fine tuning products while rapidly accelerating product teams’ delivery pipeline. 

Similarly, it helps solve the age-old issue of fostering closer cooperation between software developers and operators. 

By 2026, 80% of software engineering organizations will will establish platform teams, according to Gartner research. This prompts some to question whether platform engineering will render DevOps or DevSecOps obsolete. 

Does platform engineering spell the end of DevOps?

This was a question posed to a trio of participants during a roundtable discussion at KubeCon 2023 in April. 

The panel, which included GitLab CPO David DeSanto, Sarah Polan, Field CTO EMEA at HashiCorp, and Stu Miniman, director of market insights at Red Hat, explored at length the rise of platform engineering and whether it spells the end of DevOps as we know it.

While all three agreed this emerging trend has gathered, and continues to gather, pace in recent years, it’s more complex than simply heralding a new era in product engineering and development. It’s dependent wholly on the individual needs of the business, and their ability to navigate change. 

Platform engineering isn’t for everyone, but it could unlock significant improvements to productivity and application delivery if done currently, according to Stu Miniman. 

“When we talk to our developer communities, some of the biggest problems we hear is that there’s cognitive overload and dealing with context switching,” he said. 

“It’s a challenge that all of us have, but especially with developers. Platforms are not new, but platform engineering is a thing that’s been growing for about three years now.”

Miniman noted that, fundamentally, platform engineering is about enabling developers to focus on their primary role and delegate certain aspects of platform management and governance to teams specifically catered around that task. 

In doing this, platform engineering makes things “a little bit simpler” for developers and removes – to an extent – the cognitive overload that many experience when building and managing platforms. 

What problem does platform engineering solve?

For many years, businesses globally have focused significant resources in bringing together the developer, operator, and cyber security elements that all play a key role in delivering software applications rapidly to keep pace with insatiable customer demand.

But asking developers to build then subsequently maintain platforms has placed significant strain on teams, Polan noted. This strain has reared its ugly head frequently, especially with regard to security and risk management. 

“The past couple of years, the previous trend was very much DevOps. So we wanted to encourage developers and engineers to build and run their own infrastructure and then we quickly became cognizant, both from a business standpoint but also the technical and developer standpoint, that the developers didn’t actually want to own and run their own infrastructure because there’s a lot that goes into that,” she said. 

“Whether it’s risk based, velocity based, or not being able to focus on the business problem.”

Miniman echoed Polan’s comments, noting that this increased strain has frequently been referenced as one of the key hindrances to developer teams in addition to the increased security considerations required when developing at pace. 

“DevSecOps is great, but we know one of the impediments to rolling things out faster often are security issues that you need to worry about or getting everyone involved,” he said. 

“You’re talking about golden paths and about giving people the guardrails, governance, and control. In certain pieces, we want to take that off the plates of the developers so they can focus on writing code, modernizing things and taking advantage of technologies. 

“So, if we can make things a little bit simpler and allow developers to focus on their primary role and the stuff they want to be doing, that makes things easier.”

Platform engineering in an era of heightened risk

Surging security threats in recent years have, in part, led to the growing popularity of platform engineering as a discipline, Polan insisted. By charging responsibility of governance, control, and risk management to developer teams, this can lead to overload and causes vital issues to be missed or overlooked. 

Platform engineering helps mitigate this problem by delegating an aspect of this responsibility. 

“I think that it [platform engineering] is becoming increasingly important, especially as we start looking toward the Internet of Things (IoT) starting to explode and edge computing, and having to have this modularity but also looking from the security and risk standpoint,” she said. “We need to be able to follow these different patterns.”

Polan cited the SolarWinds and Log4j incidents as a prime example of this lack of clarity about platform risks and transparency of oversight for individual platforms within an organization’s estate. 

“I think if we look at SolarWinds, for example, not being able to follow exactly where all of this infrastructure and deployments went affected where the potential threat breach was after the initial supply chain attack. 

“So just making sure that we as organizations are in control, but also garnering business value from this by setting guardrails in place and say ‘we want you to be able to do what you want, follow industry standards, use open source’, but also make sure we have adequate control over these things.”

Why DevSecOps is here to stay

All three roundtable participants agreed that platform engineering doesn’t necessarily spell the end for DevSecOps as a discipline. Instead, this forms another part of how organizations build, modernize and manage their platform environments. 

Miniman suggested that platform engineering represents an “outgrowth” of DevSecOps, adding that it’s “an extension of what we’re doing - it’s evolutionary, not necessarily revolutionary.”

DeSanto echoed his thoughts, noting that there is room for beneficial coexistence that will ultimately deliver benefits for developer teams and the broader organization. 

“I sometimes struggle with why something has to die for something else to exist,” he said. “And so, I don’t know what nerd references are right for the audiences, but maybe it’s the Jedi versus the Sith? That’s what it feels like to me.

“I think there’s a lot more overlap and value to that. In essence, platform engineering is making sure your DevOps or DevSecOps teams are more effective - and they’re more effective because they’re working with the platform team and vice versa. 

“One of the things I’ve seen is that the more you go toward infrastructure as code, that could be in the DevOps system, but it’s for the platform team. You end up in this situation where they’re actually related and one is not necessarily killing the other.

There are many things to do when starting a company. Find desk space, register the company, get a bank account, set up the website, and all the other tasks that require different hats to be worn. If the idiom were reality, hatters and milliners would be present at every startup accelerator. 

You quickly need a product or service to offer your customers. Without this, you won’t be in business for long, unless you already have other revenue streams or your investors or lenders are patient. Funnily enough, they never seem to be. 

You’ve probably already thought of an idea already; you think it’s going to change the world for the better, solve someone’s problem, or mitigate someone’s risk – possibly you hope it’ll do all three. So how are you going to get from A to B, where A is nothing but a headful of ideas, and B is a product in someone’s hand being used “in production”?  

We faced the same challenge when we started a company six years ago, armed only with a couple of laptops and a list of good contacts. You’ll need to work hard on networking throughout your journey – it’s hard to bring a product to life without a lot of help from others. 

Laying the groundwork

We wanted to disrupt existing practices in an area of healthcare research that aims to understand patient outcomes and experiences, outside of clinical trials, in almost any disease area. First, we needed to establish at least an outline “product-market fit”. We knew we had to ask ourselves – and answer – some very searching questions. These include: 

• Who are your customers?
• What problem(s) are you trying to solve for them?
• Do your customers even know what problem(s) they have?
• Will customers pay (and continue to pay) for that solution?
• What are they currently doing instead?
• If something already exists, how does it compare?
• How will you measure success?
• What does good look like?
• Are there any early KPIs/OKRs you can use to judge your progress?

We set out to validate our ideas by working with senior clinicians at The Royal Marsden Hospital in London, one of the world’s leading hospitals dedicated to cancer treatment and research. They helped us understand the contours of patient experience in oncology, many aspects of which extend across other disease areas too. We also ideated at length with industry experts at major pharmaceutical companies to gain insight into the types of evidential gaps they are trying to fill and where our solution could help with that.  

Finally, we checked the regulatory landscape to understand external constraints such as ethics, pharmacovigilance, data protection, and security requirements. Make sure you know who your key opinion leaders are and then ask as many questions as you can – most people are happy to have a chat over coffee, especially if you aim to improve the way things work in their industry. 

The building phase

With your assumptions challenged and your offer refined, you can move on to a highly creative and productive phase that will shape your product much further and quicker than you have so far. You need to start product iteration, which is a cycle of designing, testing, prototyping, and perfecting. Regular inspections and feedback are key to this process; as is listening. Remember the Lean methodology of “think big, act small, fail fast, learn quickly”. 

For this, we created the cheapest, quickest version of the product that we could and got feedback on it as early as possible. This way the cost of “failure” was very low, and we could easily pivot or simply remove functionality and try something new. You can find several ways to create minimally viable versions – wireframing tools help a lot here, but I’ve seen early product ideation done on paper.  

In our early user testing, we implemented clickable wireframes for mobile devices, which we chose as our initial target form factor. One version was a fully featured prototype that had the feel of a complete app but was built at a fraction of the cost. There are also now no-code or low-code solutions that go a stage further and provide basic functionality that further blurs the lines between prototype and minimum viable product (MVP). 

Asking for feedback

Who should you ask for feedback? It could be anyone: friends, family, and colleagues will all have useful thoughts. All feedback is a gift, and this is one that keeps giving. The best option is to ask your prospective end users.  

Don’t get too stuck on exactly who to ask. I guarantee the minute the rubber hits the road, you’ll start learning things you didn’t realize, or think were important yet. When we got feedback from people living with an autoimmune condition that caused muscle weakness, particularly in their eyelids, we soon realized it was hard for them to scan up and down a long screen of text or icons – we hadn’t expected that, but now it became obvious. 

We then organized phone calls, online surveys, and even in-person focus groups at venues in London, Paris, and Milan. There are many options where people can take part remotely, even remotely screen and voice-recording to get stream-of-consciousness feedback from users, but in-person testing is still important as the level of insight it provides is great.  

Choosing your priorities

When you have some feedback, organize it into areas of functionality; let’s call them “epics”, given we’re already considering an agile approach. At this point, you’ll be creating a backlog of product ideas. You can’t really expect your users to tell you which of them will provide the most value; it’s up to you to listen carefully. Find out what they’re doing now: can you imagine them switching from that to using your shiny, new application instead?  

As part of our discovery, we wanted to know if people track things about their disease already, such as medication, side effects, and treatments. If so, where do they already do that? We found quite a few people using spreadsheets to do this. If you give them your app, is there any friction in moving over to it – how likely is it to happen?  

You’re often not competing with the things you think you are, such as better features and functionality. Often you’re competing for people’s attention, the span of which is now frighteningly short. Dealing with patients suffering from acute or chronic diseases, we found users already have a lot going on in their lives and spend time managing their condition, treatments, and side effects – we usually couldn’t expect them to spend long using our app too. There are exceptions, such as a group we studied who needed regular blood transfusions which take the best part of a day in the hospital during which they had time to provide feedback. A different consideration here is hospitals are often places with restricted connectivity, which can limit the use of mobile devices. 

Finalizing your product roadmap

Finally, it’s time to review the backlog and the epics you identified. Try to arrange these into a product roadmap. There is much to cover in product management but this would be a good time to familiarise yourself with two key frameworks. 

First is the Kano Model, which helps us prioritize features on a product roadmap according to the degree to which they will satisfy users. I like this framework because it focuses on user delight while creating the biggest bang for your buck. It also explains why today’s “wow” feature soon becomes expected. 

The other model is the Design Council’s Double Diamond. This encourages you to take a two-stage approach in your thinking; initially divergent exploration, and then convergent focus. It does this across two areas called the problem space and the solution space. 

We’ve used this to great effect as a means of moderating conversations while planning our product roadmap. The idea is to prevent you or your stakeholders from jumping straight to the first solution they can think of. Taking the time to check if you really understand the nature of the problem and brainstorming for several competing solutions sounds like common sense but be prepared to explain the Double Diamond repeatedly on your journey to a successful product. 

As you can see, the process of designing, testing, prototyping, and perfecting is endless but you’ll recognize when you have something good enough.  

Our first oncology study app was quite basic, but we soon added more functionality. It’s also about removing features. Users had asked us for a feature that found people nearby with a similar stage of disease for support, but it wasn’t used as much as we had hoped. In the end, we withdrew it. Nothing in product management is an exact science and the data and feedback can only tell you so much. Don’t be precious about any of your ideas or assumptions as they will probably all be challenged at some point and your product will usually end up looking quite different when your journey from A arrives at B.

​​This automatically-generated transcript is taken from the IT Pro Podcast episode ‘Building recession-proof architecture'. We apologise for any errors.

Rory Bathgate

Hi, I'm Rory Bathgate.

Jane McCallion

And I'm Jane McCallion.

Rory

And you're listening to the IT Pro Podcast, where this week we're discussing recession-proofing.

Jane

As we enter what is set to be a long recession, businesses are looking to manage and reduce costs wherever possible – including in the IT department.

Rory

More than ever, companies are evaluating the benefits and drawbacks of serverless architecture, the best payment models for their business, and the risks associated with vendor lock-in amidst uncertain financial conditions.

Jane

Today, we're speaking to Tom Fairbairn, distinguished engineer at event broker Solace, to explore the financial risks associated with lock-in, and how businesses can use low-code, serverless solutions to upskill people.

Tom Fairbairn

Hi, Rory. Hi, Jane. Thanks for inviting me on. I'm excited to be talking to you guys about this.

Rory

Thanks for being here. So late last year, the Bank of England said we're officially in a recession, and they expect it to last for about two years. Many companies have been taking steps to prepare for this even before the official announcement was made. So what are some of the main ways that this is translating into decisions in IT departments right now?

Tom

Yeah, that's a really interesting question. One of the things I think we're going to see going forward is that IT and technology has been seen up to this recession as kind of the big growth area, especially during the pandemic, I think this may flip us into a situation in which our IT departments and our technology departments are going to face much bigger constraints, especially around financial constraints in a way that probably we've not been used to for, who knows two, five, ten years. It's going to be quite an adjustment for us all, I think.

Rory

So some of our listeners might be familiar with the term serverless, some might not. So just for those who aren't, could you go over sort of a basic definition of what serverless really means?

Tom

Sure, it's interesting. I looked up a few definitions, and there doesn't appear to be one single phrase you can use. But really how I think of serverless is an evolution of the cloud model, and a kind of, at the moment it's kind of the ultimate cloud model. And what serverless is, it's a cloud technology of an execution model for your business. So what that means is, instead of paying the cloud vendor for some computing resource, you're paying them to execute something you give them. Now generally what that is, and this is the power of serverless, is you can just write some business logic and give that to the cloud vendor and say "execute that". That's really powerful, because you're not worrying about operating systems and storage and stuff like that, the environment takes care of that for you. So if you think about it, in kind of owning a car versus hiring a car versus hiring a taxi, obviously your on premise data centre is owning your own car, your on demand compute, is hiring a car, I'm going to hire a car for a day. And what you do with it is up to you. Versus hiring a taxi. When you hire a taxi, you say where you want to start, where you want to go, and all you're paying for is that particular trip. And that particular trip in this case, is executing your business logic.

Rory

So it's a far more focused sort of narrow model than even a sort of elasticity model.

Tom

It is yeah, one of the key points of that is that you're only paying for your execution time, right? So like the taxi trip, you're paying just to get from A to B, you're not paying for the fact that you have a car available to you for a certain period of time. And that's a key point. One of the things about it is that it's elastic. So that's different from scalability, right? It's not just that the processing of the execution increases the more than demands you make of it, you only pay for the execution when it happens. So it scales down when you don't need to execute. So it's a great model for businesses with seasonal demand, for instance, so retail sector. You know, Happy New Year, everyone we've just come out of the Christmas period and I doubt the run up to that, hopefully, for most of us was busy, right? If your demand tails off for the rest of the year, it's a great model for being able to kind of elastically scale with that without you needing to worry about what scaling needs to happen.

Jane

So serverless solutions, and particularly those that are available on consumption based models, everybody will be familiar with from any kind of cloud type stuff. They'd already been increasing in popularity recently. Is this something that might appeal to companies more during hard financial times such as these or is it a business or a technology choice or a bit of both? I guess.

Tom

So it's like the cloud itself. Actually, it's a business model that has driven a technology change, just in terms of kind of adoption. I saw a survey from Datadog that was saying that 50% of organisations that are in the cloud, have already adopted serverless in one way or the other, right? And whether that's from your kind of your bottom, basic serverless, which is you know, your function as a service, something like AWS Lambda or Google Cloud Functions, all the way through to the more advanced things like app runners and stuff like that. So it's tempting to think of this as, like one of the things I always talk to people about is if your CIO, CTO, CFO comes to you and says, "we want to adopt serverless, as an operational cost reduction", you're in for a hard ride. Okay, so serverless, the benefits of serverless around cost aren't really around operational cost. The benefits of serverless, I think people should think of them are more around agility, time to market and moving quickly, right? If you've got an existing application landscape, and you want to reduce the operational cost of that, just moving that to serverless is not the right way to go in the same way that in existing legacy infrastructure, if you just lift and shift that to the cloud that's not probably not going to give you an immediate operational cost, operational saving rather, that's probably going to disappoint your stakeholders. Where serverless really, really helps out organisations is getting to market quickly. And whether that's through an entire application suite or adopting, coming to a new use case bringing in some new kind of digital transformation. So engagement with customers, for instance, it enables you to do that a lot quicker. Now the kicker is, what many organisations I've spoken to is, yes it gets you there quicker but there's a cost associated with that. So we talked about the idea of vendor lock-in. There's also often, funnily enough, an operational cost in that, you know, you're buying a lot of abstraction and you're buying a lot of risk reduction in your live queue. And that kind of doing this execution, in an environment that's provided for you on an on demand basis, generally on demand, things are going to be more expensive, right? Once you start scaling them up. So a good example is it's like owning a car, versus hiring a car for a day, which is kind of your elastic compute model, versus hiring a taxi for a particular job, which is kind of your, you know, your serverless model. If you just have that one trip to go and get some groceries because your cars in the garage. That's, you know, that's probably economically worthwhile. But if you're running the children to school every day, maybe hiring a taxi every time is not the way to go.

Jane

You'd be surprised. So if I go back to the beginning of your answer really, if there isn't a direct cost saving as such, it's slightly more abstract for want of a better word. How can you either as an IT professional make that case to line of business serverless is the way to go? Or conversely, explain to line of business if they come crashing in and say "we want to do this thing that we just read about that there isn't a direct cost saving, but that it's probably still worth doing it" and, you know, good on them for their enthusiasm.

Tom

Yes. Congratulations, Mr. CIO, you know, I'm glad to see you got the pulse on things. It's like everything, right, this comes down to what is the benefit for the business? What is the business use case? What is the business business plan for this? So on the subject of like, how do we get, how do we get an organisation that maybe isn't thinking about this to adopt this? Well, it comes down to simply okay, here's the use case, we want to improve our customer engagement, improve our omni-channel experience. We need to maybe integrate, you know, a couple of different channels. Simply the use case for serverless here is we will get this project done quicker. Because we're not going to have to stand up servers, we don't have to capacity plan, it's going to be elastically scaling, so we don't have to worry about quiet periods because it will scale down as well as scale up right? So we're we're going to get this project done in three months rather than six months.

Jane

So I guess in some ways, it's sort of a go to market type thing then really.

Tom

It's entirely around go to market. If you look at the really successful people who ever who have adopted serverless, not just as a tactical kind of glue logic thing, which is to my mind what I've seen is actually the majority of use cases so far, because adoption is, you know, kind of going through that maturity curve. If you look at the the organisations that have adopted serverless strategically, the reason they've picked it strategically is entirely around time to market. You know, opportunity cost, and not around operational cost. So it's easy to think, oh this is the problem we sometimes get when you're talking to people is they're all thinking about operating cost. They're not thinking about opportunity cost.

Jane

That's been the gospel of cloud, hasn't it? Move from CapEx to OpEx. And now we're moving from OpEx. Onwards, onwards We go. Yeah.

Rory

So it sounds like serverless is currently a balance between that speed and reaction time and the long term economic sustainability of such a model for business. Would you say that serverless is maybe a good way of reacting to uncertain economic conditions, but not something that businesses are looking to apply to absolutely everything in their architecture long term?

Tom

Yeah, that's a great summary. So it is a brilliant enabler of dealing with uncertainty, because it scales down very easily and the costs scale down very easily, almost without you trying. And in uncertain times, often, you need to react quickly as you say, to perhaps new opportunities in the market. Perhaps these new opportunities in the market are short lived. And so you need to get to them quickly. That's a great time to use a serverless approach to things.

Jane

So turning to another sort of more recent technology, or more recent way of doing things, let's have a talk about low code and no code. So once again, could you just explain this a little bit to our audience? Because I remember the first time I came across the idea of no code, much like the first time I came across the idea of serverless, I was like, "well... how do you do this? You're not writing any code, you're not getting the infrastructure?" Yes, low code, no code.

Tom

Yeah, so low code, no code solutions typically present the developer with an interface that enables them to implement their business logic without having to write specific code. So typically, what you have is some kind of graphical palette, where you will implement whatever function you need to, you need to implement it but in a graphical way. So it's more of a configuration rather than writing code. And the environment takes care of providing what needs to happen for that graphical shape, if you like, to run. So this is often really useful in things like integration spaces, for instance, where you need to take something that's happening in say, a third-party tool and feed that into some internal tool, for instance. Rather than writing a whole bunch of code it can be quite simple to, say, bring a connector shape onto your pallet to connect out to your third-party service, bring in some kind of transformation shape that takes that data and does — let's take an example — masking a credit card number field or a CVC field or something like that. And then output that to another connector shape, which is then going to feed it into, say, an analytics platform. Something like that.

Jane

So it's almost like a flowchart type thing.

Tom

That's probably the easiest way to think about some of the more common approaches to this. Yeah.

Jane

And who will be, I guess, using low code / no code? Have we put all our programmers in the bin, and everyone's had it? Or is it still sort of their, their domain?

Tom

Yeah, so who's using these platforms? So on the low code side, are we binning kind of coders developers? Probably not. This is more for organisations that have tactical needs, typically around things like integration. So you see, the iPaaS vendors, like Boomi for instance, will provide these kinds of platforms. So these are not companies generally, these are not use cases maybe that are seen as completely strategic to the business, in providing a functionality that nobody else can provide. It's more typically used, you know, in integration style of use cases. I mean, possibly a technology in this space is things like RPA, where you're looking at automating what a human would have done, as opposed to writing a completely new piece of code that does something that's either unique, or there is no tool in this space. Right? If you look at the serverless space, it's a mistake to think of serverless as a kind of low code. Actually, in terms of development serverless is pretty tricky. You know, developing for serverless is actually probably harder than developing on your own machine. It's difficult to debug, it's difficult to understand the performance implications. You know, the cloud vendors do a great job of providing that information. But you need to know what you're doing, right? In the serverless space, you have to think about how your serverless thing is going to be triggered. And that leads people on to this whole idea of event-driven architectures, right? Serverless is generally triggered by events, event driven is seeing a lot of adoption recently. But you know, for many organisations, it's very new. And it's a different way of thinking for developers. So generally, it's more advanced organisations, advanced developers that are adopting EDAs, which allow them to really get the most benefit out of serverless. Low code, on the other hand, that's more citizen that's aimed at the citizen developer. Right? So that's more aimed at organisations that don't have a lot of developers, maybe they would have contracted an SI, for instance, to build some code for them. And to, you know, maybe to operationalise it for them. Low code enables them, especially in kind of you know the iPaaS space, it enables them to give their business people who maybe understand data, you know, it gives them an environment that maybe they might find that easier to do.

Rory

So, on that point of the citizen developer, it's been suggested by some that low code could be a good route for training, for upskilling. And even for reskilling. I know we mentioned it briefly in the introduction, but do you think that this is a viable pathway for maybe training up a new cohort of developers, or even avoiding the need of hiring more developers all together?

Tom

So I think it's certainly a good approach to get people who wouldn't traditionally have seen themselves as developers into that, not developing space, but certainly the more IT space. So I see low code as potentially a way of integrating IT into business. I see low code as a way of integrating what were traditionally seen as IT functions more into the business, because you're bringing kind of people from the business into this space. As long as they understand the data and have a basic understanding of where data should flow from, what are important parts of the data, how data needs to be manipulated, they can probably use these no code platforms, right? So on the subject of upskilling, I don't think low code is a way of getting green employees into the IT department. But it is a way of extending businesses' reach into it, and IT's reach into the business.

Jane

So I guess is the benefit then of low code and no code, in this economic climate that we're entering into now or did at the end of last year, similar in some ways to serverless? That it's not about getting rid of people, it's not about replacing people, it is about speeding things up a bit, taking the load off your IT department and allowing them to do other stuff. Or indeed potentially not having to fork out so much for external contractors.

Tom

Yeah, so I think there's several aspects there. There's the idea that maybe you wouldn't be employing quite so many SIs to write code for you, because you can do low code yourself. Maybe the management aspects of it are simpler, because you're not having to commission an IT project to do this, maybe you can hand this over to perhaps there's some people in the business that are maybe less busy than they were. And so they can get more involved in this. I've got personal experience with this, actually, my brother is getting involved in kind of an IT-style project much to his disgust and my delight, for this kind of reason, right? So I think low code enables a lot more flexibility in the business, because as we talked about the kind of the skill aspects, it does extend out. Not to a huge degree, you're not going to have everybody in the business suddenly writing low code stuff, right? You do need, you know, a certain mathematical skill set that you would need to kind of interact with computers in this way anyway. But it does give a lot more flexibility. So IT people that are maybe really focusing on reducing the operational cost of a legacy system can maybe concentrate on that, while guiding their business colleagues using this low code environment to maybe move some new projects on forward.

Jane

So I guess, you know, from the point of view of these other employees, that might be a bit of an opportunity for them as well if they're a bit sort of technically minded but in another department. You know, I've seen this myself with people who I know moving from, let's say accounts actually into the IT department, because they have the aptitude and the willing for it even if they don't have the training. Is it a pathway, I guess, for those kinds of people?

Tom

Yeah, I'd say so. And it's interesting you say accounts, right? So immediately, I'm thinking, you know people in finance departments generally they're numerate, generally they are used to abstract thinking. So, you know, it's ideal for people like that. And as you say, I mean, personal experience here with my brother right? That's exactly the path he's following. He's in a finance department. He's able to draw up dashboards and understand what the data is that feeds that dashboard, right. And he's using a low code environment to do that. Yeah, another kind of cohort of people that will find this kind of concept appealing, are maybe those in the scientific or data domains, right? So anybody that's in economics, statistics, anybody in the science domains that aren't already using things like MATLAB to do a lot of scientific and statistical analysis, right? If they're needing to feed things into the business, they will probably find this low code kind of platform, useful too.

Rory

So it really has to be done maybe has more of a cross-silo operation within the same company, less than maybe an onboarding thing? I know that a lot of companies looking at hiring freezes right now, as we've been saying in kind of in a recession, looking at actually cutting spending. But that maybe they can assess moving some people around within the company into positions that are more more beneficial, as long as they've already got that kind of core skill set that we've discussed.

Tom

Yeah, I wouldn't say this has to be cross silo. But I would say that it's a good opportunity for that. A danger is when you're dealing with more regulated entities, organisations that have a lot of privacy concerns. There is obviously some danger here about how this data is used in the awareness of privacy concerns and regulatory concerns. So we talked about the idea that this might be, you know, attractive to people in the scientific domain. You can imagine in a pharmaceutical company, there is a risk there for instance that you don't want to leak, leaking clinical data or patient information. Right? So while it is attractive, as with everything, there are risks to it too.

Rory

So we've talked about serverless architecture, we've talked a bit about some of the different options that businesses are having to assess. And in the intro, we briefly discussed vendor lock-in. In the current economic climate, would you say that it's better for businesses to embrace a diverse range of architectures or really to sort of concentrate on one and put all their efforts into it? While keeping in mind this, maybe this worry about vendor lock-in?

Tom

Yeah, it's as with all these answers, it depends right? If you're attacking the time to market challenge, let's say you're a startup. And you you're coming up with a completely new feature, you need to write a lot of your own code. That's an ideal use case for a serverless architecture. And that's going to build on you're probably, because this sounds like a kind of greenfield thing, you're probably going to build that on top of an event-driven architecture right? Now, in my experience, what happens is you grab a bunch of market and then the, you know, the organisation expands, and what often happens is operational costs, in this particular case, cloud costs of running this serverless architecture. They start to explode, right, because you're moving from a very bursty and very on-demand model into a business that's maturing, the demand is becoming more, you know, more plateaued, hopefully growing but it's going to become more steady. And that's the point typically, at which CFOs, start getting itchy and we need to start architecting re-architecting a little, maybe we'll start to back off some of our more core functions that are running more at the time. Maybe we need to re-architect them, refactor them to be less serverless and more kind of static services sitting there servicing requests. So like a micro service or something like that running on elastic compute. But that's fine. That's part of the inevitable maturation of any architecture, right? And we should be doing this, we should be constantly looking at our architecture and deciding what is the best approach going forward. Now that's for, you know, an organisation that has a deep technical challenge that they're trying to solve that potentially hasn't been solved by anybody else. And so they're writing big chunks of code that just happens to be running in a service architecture, because that is the best way of dealing with the business model. If you look at more mature organisations and organisations that perhaps aren't as technically led, so you look at your typical manufacturing shop, or consumer goods manufacturer, or logistics providers. People like those, obviously, they have big technical needs, but their technical needs that are probably better served by third-party applications. And they're the probably, it's going to be more the low code approach. Now, with serverless, there is at the moment, a large degree of vendor lock-in. There are standards around this, but as far as I can see they're not being widely deployed or adhered to. On the low code side of things, you do have a less degree of locking, because basically you're picking a low code solution. If that one works for you, great. If you feel that there's a problem with that, you can always adopt another low-code solution and run them side to side. And that's, I'm not gonna say it's easy to do. But I'd say it's easier to do than deploy necessarily to serverless solutions. There is a third way in between this. I think we're seeing in the market a lot more maturity around cloud models, and the move to hybrid cloud, and multi cloud, where you see that one cloud vendor provides a particularly good service service and another cloud provider perhaps provides a particularly good data analytics kind of capability. The idea of being able to operate in that multi-cloud model is becoming more, it's gaining popularity. The key there is how you communicate between your on-premise cloud and multi-cloud environment, especially in an event driven architecture, right? And there are solutions that enable you to do this, helps to mitigate the vendor lock-in problem. It doesn't deal with it completely, but it certainly gives you a lot more options.

Jane

So another sort of buzzword, trend, however you want to think about it for the past couple of years has been digital transformation. Is serverless architecture, or indeed low code / no code, a long-term cure I guess, for firms who are struggling with digital transformation? Or is it a false hope?

Tom

Interesting question. I mean, the first thing I'd say is, if a firm is struggling with digital transformation, serverless is not the way to go. Not yet. Right? You need to be, with respect to digital transformation, you need to be quite mature, have a good developer team and a good architecture team to really make the use of serverless strategically. You may find there's some tactical uses, a bit of glue here or there that you might use serverless for, but what would I would advise that kind of organisation is you probably want to be looking at a low code solution. Because it will be a lot easier for you, there's less architectural things you need to think about, and it gives you more opportunity to just concentrate on fulfilling one particular use case at a time. Now, is it a magic bullet that enables you to do digital transformation? No. But then again, I don't think anything really is. Digital transformation, a bit like people think of cloud as a technology where actually it's more of a business model, digital transformation I think is more of a cultural thing. It's about how the business operates than it is a technology. And we need to think about these technologies as enabling digital transformation, rather than being the digital transformation.

Jane

Yeah.

Rory

So as with kind of the range of things that we've discussed throughout the episode, it's more of a business strategy, business culture that each individual company is going to have to very carefully assess, rather than just stick it on as a kind of a sticking plaster for the recession and hope that things just 'get more digital' and improve in that way.

Tom

Yes, exactly. I mean perhaps, I don't know, I'm kind of speculating here. But perhaps the recession is an excellent opportunity for companies to really focus on what digital transformation is and what it can do for the business. So one of the problems with any, you know, initiative is that it's going to cost money to do. Digital transformation is going to cost money to do. Now what it can do, is it can unlock value in your customers and suppliers, and it can produce operational efficiencies. Maybe you know, because times have been good, maybe you've let the operational efficiency slide while you try to grab market. Maybe this is good opportunity to look at your operational costs and not look at, you know, low-code digital transformation or service as an opportunity to reduce your IT costs. But maybe it's a good opportunity to say we need to transform these parts of the business in a way that enables us to cut out manual processing, in a way that enables us to engage with customers without having to build up huge customer support teams.

Jane

So I suppose it's one of those things with the business, it depends on their strategy for dealing with the economic climate, you know are you going to batten down the hatches hold on, and open the doors again in two years time, or whenever we get out of this. Or are you going to try and invest your way out of a recession?

Tom

Exactly. Right. And if... digital transformation is disruptive, and it's going to disrupt your business, in that you know, you're implementing a new way of dealing with your employees, your customers, your suppliers, your business processes, and everything. So perhaps this is a golden opportunity to hopefully take some of that cash you've built up during the good times, to make yourself more lean and agile during the recession. I mean, this reminds me actually, it's probably a bit off topic. But I remember seeing an advert on TV years ago about TV advertising. I've noticed the TV is now advertising for TV advertising again. But it had a really nice metaphor, and that was a kind of basketball bouncing up and down. And the hand that was bouncing the basketball up and down was TV advertising. In this case, it could be it and digital transformation, right? If you stop investing, yeah, you don't have to put so much effort into bouncing the ball. But the ball is going to slowly reduce the size of its bouncing. And if you think about that, in terms of your business that's a good metaphor.

Rory

Well Tom, thank you so much for being on the show.

Tom

Thank you very much. It's been really great to talk to you guys, really interesting discussion.

Jane

As always, you can find links to all of the topics we've spoken about today in the show notes, and even more on our website, it pro.co.uk

Rory

You can also follow us on social media, as well as subscribe to our daily newsletter. Don't forget to subscribe to the IT Pro podcast wherever you find podcasts. And if you're enjoying the show, why not tell a friend or colleague about us.

Jane

We'll be back next week with more from the world of it. But until then, goodbye goodbye.

As we enter what is set to be a long recession, businesses are looking to manage and reduce costs, and IT departments will be forced to budget where possible.

More than ever, IT leaders are evaluating the benefits and drawbacks of serverless architecture, the best payment models for their business, and the risks associated with vendor lock-in amidst uncertain financial conditions.

This week, we spoke to Tom Fairbairn, distinguished engineer at event broker Solace, to explore when serverless and low-code solutions might be right for a business, and how the recession could be an opportunity for meaningful digital transformation.

Highlights

“Where serverless really, really helps out organisations is getting to market quickly. And whether that's through an entire application suite or adopting, coming to a new use case bringing in some new kind of digital transformation.”

“If you look at the serverless space, it's a mistake to think of serverless as a kind of low code. Actually, in terms of development serverless is pretty tricky. You know, developing for serverless is actually probably harder than developing on your own machine. It's difficult to debug, it's difficult to understand the performance implications. You know, the cloud vendors do a great job of providing that information.”

“Perhaps the recession is an excellent opportunity for companies to really focus on what digital transformation is and what it can do for the business. So one of the problems with any, you know, initiative is that it's going to cost money to do. Digital transformation is going to cost money to do. Now what it can do, is it can unlock value in your customers and suppliers, and it can produce operational efficiencies.”

Read the full transcript here.

Footnotes

• What is serverless architecture?
• Low-code vs no-code: What’s the difference and how can these tools change your business?
• Why low-code and no-code can be a route through recession
• What is agile development?
• UK's digital skills shortage reaches "all-time high"
• Tech skills in the UK lag behind most of Europe
• Report: One in four UK businesses hiring overseas to tackle talent shortage
• Cost of living crisis driving salary concerns for C-suites
• Datadog's 2022 Container Report Finds Organizations Expanding Container Adoption With Improved Ability To Scale And Manage Complex Environments

Subscribe

• Subscribe to The IT Pro Podcast on Apple Podcasts
• Subscribe to The IT Pro Podcast on Google Podcasts
• Subscribe to The IT Pro Podcast on Spotify
• Subscribe to the IT Pro newsletter
• Subscribe to IT Pro 20/20

Accenture and Atlassian have partnered to help organisations learn how to be more agile, after having collaborated together on over 100 projects.

The pair will also help companies get more value from tech investments, improve customer and employee experience, embrace change, and introduce new enterprise agility services.

Accenture and Atlassian are hoping to support clients in every step of the enterprise agility journey, from speeding time to market through product-centric development and migration to the cloud, to adopting lean management principles that connect strategy to execution.

The two companies have already collaborated globally on over 100 engagements for various clients across industries. One of these is Belong, a division belonging to Telstra, Australia’s largest telco. The organisations helped Belong to implement Atlassian’s cloud-based agile work management software, Jira and Jira Align, at all levels and across all teams, reshaping how the team tackles cross-functional collaboration to be more productive, innovative, and agile.

“As organisations continue to move to the cloud in droves and make substantial investments in technology to transform how they do business, new capabilities are needed for agile enterprise management,” said Greg Douglass, senior managing director and global lead for Technology Strategy & Advisory at Accenture.

“Through our strategic collaboration with Atlassian, we are bringing comprehensive enterprise agility solutions to our clients, which are critical not only for the technology organisation, but for modernising the way all teams will work in the future.”

The two companies will also establish an Accenture Atlassian Center of Excellence, which will provide coaching, education, agile practice disciplines, and platform enablement to clients around the world.

“Accenture has a proven ability to drive transformative value by combining technology and process excellence for the world’s largest organisations,” said Kevin Egan, head of Enterprise Sales at Atlassian. “In combination with Atlassian’s platform, designed to unlock the full potential of every individual within every team, we can achieve true and lasting digital transformation for our joint clients that is never truly done but continuously innovated on.”

Accenture has formed a number of partnerships recently, and teamed up with the US multinational Mars earlier this month to transform and modernise its global manufacturing operations. Accenture is set to help the confectionery giant to expand its use of artificial intelligence, cloud, edge technology, and digital twins.

Google Cloud has announced the launch of Software Delivery Shield (SDS) which aims to fortify businesses' resistance to software supply chain attacks that have proliferated in the last 12 months.

Announcing the new product at its Google Cloud Next conference, SDS is a fully managed software supply chain solution that was designed to address five key areas of supply chain security: application development, continuous integration (CI) and continuous delivery (CD), supply of software, policies, and production environments.

The new product aims to equip developers and security teams with the tools to create cloud applications free from supply chain vulnerabilities which has become a mounting concern across the industry

It will also work alongside other Google Cloud services like Google Kubernetes Engine (GKE), Cloud Build, Cloud Deploy, and more, bringing with it a score of features that promote collaboration between IT, DevOps, and security teams.

Providing protection for software at the development stage, SDS includes a new service called Cloud Workstations (preview) which gives developers managed development environments on Google Cloud's platform.

These environments will be browser-based and fully customisable, with options for collaborative configuration across IT admins and security teams that aim to promote better overall app development security, according to Google Cloud.

Cloud Workstations won't store code locally and come with features such as private ingress/egress, identity access management (IAM) policies, and forced image updates to proactively prevent common security issues in development.

Google Cloud's family of integrated developer environment (IDE) plugins, Cloud Code, will also support a new feature called Source Protect which offers developers real-time feedback within their IDE, identifying vulnerable dependencies and license reporting.

Many of the software supply chain-related issues from the previous few years have been rooted in vulnerable dependencies which can be difficult to check manually. GitHub's Dependabot, for example, has been available since April 2021 and offers similar dependency-scanning capabilities to developers with the aim of securing the open source software space.

Software supply chain security is an area of increasing concern for tech leaders. A recent survey of C-suite executives identified that software supply chain attacks are a ‘chief concern’, with just 32% of respondents indicating that their supply chain was ‘very secure’.

Google Cloud said one of the primary routes to attack open source software is through the compromise of CI/CD pipelines and has strengthened both its CI and CD platforms, Cloud Build and Cloud Deploy, as a result.

New security feature integrations will be added to both platforms such as deep IAM controls, isolated environments, and approval gates to give DevOps teams greater governance over app builds. The cloud giant said these are "key parts" of the overall SDS product.

Protecting the runtime environments is also as important has protecting the development pipeline, Google Cloud said, which is why it also added new security features to the GKE and Cloud Run containerised app runtime platforms.

New capabilities for GKE, currently in preview, will now offer a more comprehensive view of the security posture of clusters and workloads. Teams can now access detailed reports, automatically assigned vulnerability severity ratings, and insights into operating system flaws and workload configurations.

All the new reporting available to GKE can also be fed into Cloud Logging allowing for security event information to plugged directly into a business' a security information and event management (SIEM) system.

SDS also introduces a policy engine for the strictest environments, Binary Authorisation, to establish a chain of trust across the entire software supply chain.

It's what Google Cloud is calling a deploy-time security control that only trusted container images can be deployed on GKE or Cloud Run - images must be signed by trusted authorities during the development process in order for them to be deployable.

SDS can be used alongside Google Cloud’s Assured Open Source Software service, announced in May, which ensures that open source software used within an environment has been vetted by Google.

The service now covers over 250 packages, written in both Java and Python, all of which have been scanned and tested for vulnerabilities by Google. In this way, Assured Open Source Software and SDS can both speed up and improve the security of cloud deployment, through the collection and distribution of verified software.

Faster response with Chronicle Security Operations

A separate announcement made at Google Cloud Next was the introduction of Chronicle security operations, a cloud-based software suite that aims to allow teams to detect, investigate, and respond to threats more quickly.

Chronicle Security Operations will bring together SIEM technology with the security orchestration, automation, and response (SOAR) products from its earlier Siemplify acquisition, as well as its Google Cloud's own threat intelligence and Mandiant's incident management tools, and bundle it all under new Chronicle branding.

“With Chronicle, we no longer have to make the tough decisions around which data we can afford to log and how little retention we can live with,” said Mike Orosz, chief information security officer at infrastructure management provider Vertiv.

“Insufficient security event monitoring is a thing of the past, and there is no better time than now to align to best practices.”

The branding for Siemplify's SOAR and SIEM tech will now be Chronicle SOAR and Chronicle SIEM respectively, and host of new features have reached the product family, all in preview.

The suite seeks to provide teams with cloud-scale security data using Google’s hyperscale infrastructure and petabytes of contextual information, up-to-date threat intelligence, and automated responses to common security threats including phishing and malware.

With Chronicle Security Operations, Google Cloud also aims to provide security teams with a number of quality-of-life features and data aggregation tools to improve threat insight.

These include a single display that combines information on chosen entities from multiple data sources, including Google Cloud Threat Intelligence, integrated alert management between Chronicle SIEM and SOAR, and pre-built responses to cloud-based alerts to speed up threat resolution.

Google has called on its employees to improve efficiency and focus during the current economic uncertainty with a new internal effort called a "simplicity sprint".

The tech giant told staff at a meeting on Wednesday 27 July that it isn't performing as it should be with its current headcount, according to internal documents seen by CNBC.

Essentially, the "Simplicity Sprint" initiative is about crowdsourcing ideas for quicker product development from within the company. The term sprint is synonymous with software development for accelerating or collectively pushing towards a common goal. For Googlers, Pichai will open the floor up for idea sharing via an internal survey that asks if management can reach out with follow-up questions.

Some of the questions included in the survey, seen by CNBC, suggest Google is seeking information about areas that could be reduced and streamlined. "What would help you work with greater clarity and efficiency to serve our users and customers? Where should we remove speed bumps to get to better results faster? How do we eliminate waste and stay entrepreneurial and focused as we grow?" is an example of one of the questions.

There are growing concerns among Google employees about cuts and redundancies, according to the documents. The firm is one of several tech companies that has announced plans to slow recruitment with Google even said to be reviewing its headcount in recent weeks as the economic downturn begins to shrink revenues.

"I wanted to give some additional context following our earnings results, and ask for your help as well," CEO Sundar Pichai said, according to CNBC. "It's clear we are facing a challenging macro environment with more uncertainty ahead. There are real concerns that our productivity as a whole is not where it needs to be for the headcount we have."

It's reported that Pichai then asked employees to help create a culture more mission-focused, that priorities the company's products and customers.

"We should think about how we can minimise distractions and really raise the bar on both product excellence and productivity," Pichai said.

Business data/insights reside in databases, making them integral to business operations. As a result, IDC estimates that at least 75% of business applications interact with databases. With cloud-based application modernisation becoming the core strategy, databases also need to become cloud-ready and be flexible/cost effective, agile, and scalable to create and manage database operations.

Cloud is the infrastructure engine of digital business. But cloud adoption is moving beyond vanilla IaaS strategies.

The combined Public Cloud IaaS and PaaS market is forecast to have revenues of $400 billion in 2025 with a 28.8% CAGR for 2021-2025. Unstructured data analytics/data management and media streaming are forecast to be the fastest growing segments with CAGRs of 41.9% and 41.2%, respectively.

Deploying and scaling databases at the speed of applications and ensuring continuous security, patching, cloning, and data reduction capabilities is complementary to application modernisation on PaaS.

Provided by

More than 500 million digital services and apps will be developed and deployed by 2023, according to an IDC prediction. The number of new software programmers and developers to build these apps, however, is expected to be woefully short. Businesses will not be able to hire enough individuals to meet this gap and the demand is likely to remain unmet.

In such a scenario, IT teams need to find an efficient way to build applications quickly, which is where low-code and no-code development capabilities come into the picture. Low-code and no-code platforms allow IT teams to rapidly build and deploy applications with the help of a drag-and-drop interface and visual design tools. Such platforms have been widely touted as viable alternatives to traditional development methodologies, but what are the differences between low-code and no-code systems, and what are the benefits?

What is low-code and no-code?

According to Gartner, low-code development is a form of rapid application development, or high-productivity development, which gives users the option to use scripting or predefined code. Such tools use a variety of approaches to automate development and also render the process more abstract. They include code generation, drag-and-drop editors, component assembly and model-driven and meta-driven development.

Low-code platforms are targeted at professional developers, but they don’t actually require coding expertise; a technical business user or a junior developer can be highly productive when using a low-code development platform.

No-code tools, on the other hand, are meant specifically for non-professional developers. The name implies no programming language is used and little-to-no experience is needed, and it’s very much targeted at non-developers or citizen developers.

What are the key differences between low-code and no-code development?

Low-code development platforms are meant for users with some level of coding knowledge. Professional programmers and developers can use low-code to deliver applications quickly, and focus their efforts on more complex tasks rather than tie themselves up with mundane programming tasks. In such a scenario, even non-IT professionals with some programming knowledge can use low-code development platforms to develop apps.

No-code development platforms, meanwhile, target non-technical business users who have little or in some cases zero programming skills. Such users can use no-code development tools to quickly build and deploy business apps.

No-code is typically used for creating tactical apps that handle simple functions, while low-code tools, though also fit for such use cases, are also used to create apps for processes critical to a business's core systems, such as digital transformation initiatives.

To generalise, a no-code platform can be thought of as a special type of low-code platform, where the visual components are used to address a specific line of business, industry-specific function, or support a company's corporate branding. A low-code platform, though, may need the help of in-house developers to make changes in the back-end code so any new app aligns with other software used by the business.

Examples of low-code and no-code platforms

Mendix is a low-code development platform that provides an integrated development environment (IDE) tailored for all developers. Business users get to use Mendix Studio, a WYSIWYG IDE, while professional developers can use Studio Pro, a low-code IDE. It has a React Native framework that can create mobile apps for both Android and iOS.

Zurich insurance, for example, had a DevOps team staffed partially with business analysts, i.e. developers having a keen awareness of business needs, but very little or no coding expertise. Using the Mendix low-code development platform, the firm was able to realise millions of pounds of business value annually from its apps.

There are other key examples of low-code platforms including those from Appian, which is fitted with native deployment tools and integrations with third-party systems, and Microsoft Power Apps, which helps users build custom business apps. Other providers include Salesforce Lightning, Zoho Creator, Quickbase and OutSystems.

The Betty Blocks development platform, meanwhile, is a no-code system that can build complex applications for the enterprise without the need for any programming. People can use the User Interface Builder to drag and drop any front-end, ranging from dynamic web apps to customer portals. They can also combine it with JavaScript frameworks like Vue.js and Angular for full flexibility and freedom. Data processing and management can also be done easily with the help of visual modelling.

Betty Blocks is being used by enterprises in various industry sectors, with the city of Zaanstad saving €2.4 million in four years by replacing their legacy software system with Betty Blocks.

There are a litany of options available though if your business aims to pursue no-code development, including Bubble, which is geared towards building web apps, as well as Glide, which is designed to create apps from spreadsheets. Voiceflow, too, helps businesses design and launch virtual assistants. Other notable no-code platforms include Google’s AppSheet, Clappia, Kintone and Creatio.

What are the benefits vs traditional development?

Developers using traditional application development methodology use conventional methods for creating robust applications. On the other hand, low-code and no-code development environments allow business users, citizen developers, and others to create apps quickly with the help of drag-and-drop interfaces.

Although there are measurable benefits of building apps using traditional development methods, with experienced personnel, some businesses might find they’re more likely to appreciate a handful of the benefits of low-code and no-code. These include overall costs, the frequency of encountering bugs in the end product, and the pace of development.

Cost

Traditional software is expensive to build and can cost a business hundreds of thousands of dollars to roll out. On the other hand, low-code or no-code development is far more cost-effective if harnessed properly. The reason is that with such platforms, a business usually pays for accessing a specified service rather than bear the cost of end-to-end development.

Quality of applications

Due to the complex nature and high technicalities involved, traditional application development methods end up producing many bugs and security flaws, more often than not. Although the apps tend to be far more complex and feature-rich, users might find it difficult to use applications developed through traditional methods.

Applications developed with the help of low-code or no-code platforms usually don’t have many bugs and normally run smoother than traditionally developed applications.

Agility

Traditionally developed apps are developed on platforms with complicated configurations. As a result, the apps and the development processes tend to be less agile. Furthermore, due to complex coding, a developer typically takes a lot of time to learn and also operate such platforms accurately.

On the other hand, low-code and no-code platforms are easier to operate and use due to drag-and-drop features. Furthermore, developers don't need to write any special code, and overall development, as well as maintenance, is much more simple.

What's the future of low-code and no-code?

Citizen developers and the use of low-code or no-code development platforms have expanded exponentially in the last few years. Such development methods offer a meaningful way for businesses to develop digital solutions quicker than traditional development methods, and without relying on a seemingly shrinking pool of skilled programmers.

While web developers aren’t likely to become obsolete, the growth of the use of low-code and no-code development methods has led to a rise in businesses being able to create and design apps, web apps, and integration workflows. This is mainly due to the benefits of businesses using such tools being able to boost digital transformation efforts, reduce costs, and achieve increased agility and greater productivity levels.

For IT decision makers, the case for cloud adoption has been well established, but in a business case they need to clearly and concisely communicate those benefits to their organisation.

This whitepaper explores Amazon Web Services’ Cloud Value Framework, which breaks down the business value of the cloud into four key dimensions: cost savings, staff productivity, operational resilience and business agility.

Download to learn how to improve the ways in which you communicate cloud value to your organisation.

Provided by

Malaysia's digital travel and lifestyle platform AirAsia has signed a five-year partnership with Google Cloud to boost its Super App and create a cloud centre of excellence.

Airasia operates in Malaysia, Indonesia, Singapore, Thailand, and the Philippines, offering flight and hotel bookings, e-commerce, food and parcel delivery, ride-hailing, and financial and health services.

Launched in 2020, it currently has 51 million users and is one of three unicorns headquartered in Malaysia.

The deall will see the companies establish a Cloud Centre of Excellence (CCoE) made up of Google Cloud technologists and AirAsia employees. It aims to embed an agile culture across AirAsia’s business units, accelerating product development and upskilling talent as a result. It will also deploy Google Cloud’s Site Reliability Engineering and MLOps best practices.

Through Google Cloud’s infrastructure, the CCoE hopes to help AirAsia’s Super App, which provides customers with flight, hotel, and food delivery information, to run seamlessly as more features are added, especially if accessed by a high volume of users at any given time.

The CCoE’s data scientists will also use the tech giant’s analytics, machine learning, and AI technologies to get real-time insights into customers’ opinions and emerging trends. This includes an understanding of the travel sector’s recovery trajectory and consumption patterns around food delivery and ride hailing. Through this, the companies aim to serve users hyper-personalised recommendations.

Lastly, the CCoE will develop a suite of software development kits (SDKs) that ecosystem partners and external developers can use to create new features and services for AirAsia. This includes software components for biometric identification, chatbots, and e-wallets. Google is also set to expand its developer community in Southeast Asia through AirAsia’s network.

"With Google's help, our ecosystem will not only be transactional, but be about building community, and enriching that community – not just the customers but partners like restaurants, airlines, hotels and drivers," said Tony Fernandes, CEO of Capital A, formerly known as AirAsia Group. "I'm excited to reveal how Airasia and all of Capital A's assets will transform Asean and deliver value, not only in transactions but in making Asean a smaller place. What we are doing is not evolutionary, but revolutionary. I'm going to enjoy the ride with Google."

This isn’t the only new partnership Google Cloud has been involved in this week. Yesterday, Boeing signed three new cloud agreements with AWS, Google Cloud, and Microsoft, as it aims to get rid of outdated legacy systems. The aerospace company said that its partnerships expand and grow on existing relationships with each cloud provider, creating a single foundation for the company’s approach to cloud computing in the years ahead.

Mizuho Financial Group has signed a deal with Google Cloud in a bid to help it undergo digital transformation and regain its customers' trust after it experienced serious IT errors last year.

The Japanese bank was hit by eight system failures between February and September last year, temporarily suspending transactions for about 80% of its ATMs in one example, according to the Japan Times. In December, it experienced its ninth system failure, where it failed to process 300 money transfer requests.

This occurred despite the bank pledging to prevent similar failures after financial authorities ordered it to improve its operations in November. As a result, the company’s chief, chairman, and three other executives resigned from their positions.

Mizuho's partnership with Google Cloud, announced on Wednesday, aims to accelerate the company's digital transformation and retail innovation strategy. The bank is set to build a new digital marketing platform on Google Cloud that will integrate with Google Analytics. It will use Google Cloud’s data analytics technology to generate real-time insights about customers’ preferences and anticipate their needs, and use its artificial intelligence (AI) to deliver more personalised experiences for customers.

The bank will also create new and advanced digital financial services offerings, like banking as a service, through the Google Cloud. It also plans to create new digital products and services that appeal to customers at scale.

Mizuho also aims to rebuild trust among its customers, chiefly by leveraging Google Cloud’s security engineering expertise and development processes to modernise its systems and make its digital infrastructure and system development process more agile and secure.

As part of this, the deal will look at transforming Mizuho’s corporate culture. The bank hopes to adopt Google Cloud’s approach to product development and problem solving into its corporate culture and become a destination for top talent.

“To deliver on the needs of our customers in this new era, we're working with Google to enhance our operational capabilities, innovate across our online and mobile channels, and transform our corporate culture,” said Masahiro Kihara, president & group CEO of Mizuho. “Through this partnership, we will become a stronger and more resilient financial group which our customers can trust.”

The bank will hope that this new partnership will stabilise its IT systems, after spending $3.6 billion on development in 2019, according to Reuters.

It reported a large-scale system breakdown on 28 February last year, which affected over 4,318 ATMs out of 5,395 nationwide, resulting in thousands of bank cards and paybooks becoming stuck inside machines. The bank had previously experienced large-scale system glitches in 2002 and 2011.

This automatically-generated transcript is taken from the IT Pro Podcast episode ‘Stamping out scope creep'. To listen to the full episode, click here. We apologise for any errors.

Hugh Armitage

Hi, I'm Hugh Armitage.

Adam Shepherd

And I'm Adam Shepherd.

Hugh

And you're listening to the IT Pro Podcast where this week we're channelling Thom Yorke and talking about creep.

Adam

There are any number of potential pitfalls associated with running a live development project within an organisation. But one of the hazards that frequently ends up causing problems for dev teams is the dreaded spectre of scope creep.

Hugh

As projects progress, it's easy for their remit to gradually morph and expand, eventually growing to the point of becoming unwieldy and impossible to manage. Controlling and limiting scope creep is a finely honed craft, and one that the best project managers have perfected, because without keeping a close eye on it, unrestrained creep can have major negative impacts on an organization's development strategy.

Adam

Joining us this week to talk about the dangers of scope creep, as well as some of the strategies for preventing it is Jess Cregg, developer advocate at LaunchDarkly. Jess, welcome back to the show.

Jess Cregg

Thanks so much for having me.

Adam

So, Jess, you've worked on a number of projects. To start us off with, can you give us some examples of scope creep in action?

Jess

Yeah, absolutely. So scope creep. It's, it's, it's a very human problem. And I was chatting to a lot of people about it recently, I found one person put it really well. They said that scope creep often happens when you confuse your project team, with your product team's aims, it can often happen when you're designing for the problems that you're going to have in the future, as opposed to the problems that you're facing right now. So one of my favourite instances was I was chatting to a speaker that I met at a conference recently, and he was talking about how he was once working on a team where they were creating a UI and a database for changing the country value in their product, a value that could have been tweaked with three lines of code and was probably going to be changed once every three years. So that's thousands of pounds for something that gets used once. It's maddening.

Adam

And that seems like it should be, as you said, a fairly simple problem to address and one that really doesn't need that much engineering, you'd think.

Jess

Absolutely. But it's, it's one of those problems that's hardwired into us, we experience the pain once of something going horribly wrong. And we know the worst case consequences. So we try to avoid those in a very helpful human evolutionary science. Scope creep can come really, it can really easily infiltrate your team. And one of the ways that, one of the one of my favourite ways of preventing it from happening is to keep your teams really small. Everyone's talked about the, everyone speaks about the the two pizza rule. Have you heard about that?

Adam

Yes. It's one of my favourite rules.

Hugh

I'll have two pizzas, please.

Jess

So the two pizza rule is a rule that was made really famous at Amazon. It's another FAANG concept that's taken the entire industry by storm, essentially, it's the idea that your team should only be big enough so that they can be fed by two large pizzas.

Adam

Yeah, it's a good rule, but slightly flawed in that I know many workplaces where that would result in a development team of one and a half people.

Hugh

I was gonna say two, but one and a half. Yes.

Jess

I mean, it's not only great for keeping team delivery orders down, but for preventing teams from growing. But when I was recently reading the DevOps Handbook they're talking about one of the main reasons why this is implemented is, is due to, we have a hard limit of the number of relationships that we can form with people.

Adam

This isn't something I'm familiar with.

Jess

So it's the phenomenon that as humans, usually we can form quality relationships with about 150 people.

Hugh

It's like the policing thing, how you can control a small town is fine, and then you get more people and then you get chaos.

Jess

Exactly. And when those sort of chaotic relationships start to breed, we can kind of start to misinterpret where the boundaries lie between our commitments to a project. When we're faced with a problem, we think more heads equals faster solution, right? But that can only be true when when work is completely interchangeable, when you can just throw a bunch of people onto a project and they can get on with it without without any interaction. My favourite analogy for this is, is I don't know whether on not you used to play Age of Empires?

Adam

Oh, yeah.

Jess

But whenever you used to throw just as many villagers as possible on a build project, they could just get on with it, and it was absolutely fine. But it's just not really the case with software, unfortunately. You need to be able to speak between each other, you need handoffs for different subtasks, and it's actually been proven that the more people that you throw on a software team, the later the project becomes, so you not only end up compounding things in terms of time, but then that puts everyone under pressure and they're pressured to deliver more over their agreed aims.

Hugh

It sounds a bit like a sort of personnel problem, like we're hardwired to devolve to scope creep, if we're not guarding against that.

Jess

Definitely, I mean, we want to, we don't really want to meet the aims, we want to be able to like exceed what we're going for in our team.

Adam

So one of the one of the best kind of illustrations of the principles behind scope creep, for me, is, when somebody leaves a business, often in the interim, before a replacement is hired, one of their colleagues will end up assuming some or all of their responsibilities in the kind of in the interim. And in a lot of cases, that just becomes the norm, you know, the hiring process goes on, maybe it's a couple of months, or, you know, even longer before a replacement for that person who's left can be hired. And over the course of that period, those kind of temporarily assumed responsibilities, just become part of that person's day job. And that is, I think, a really good illustration of how easy it is for scope creep to creep in, if you like, to, basically, anything, and it's especially the case in software projects, I think, particularly when you have large numbers of business stakeholders who aren't actively involved in the development process, and don't necessarily understand that, you know, when they say like, oh, maybe we could add such and such, you know, capability, or, you know, maybe it would be good if it could also cover, you know, this business area, or geographic or whatever, isn't just a case of tweaking a couple of variables.

Jess

That's the version of scope creep, I've become the most, that I've had the most personal experience of, and it's really easy for it to happen, because as soon as it's just one more thing, things can spiral out of control. And the just one more thing rule doesn't usually end up eating into the main sort of time that you frame for your project, for the thing that you're developing, it falls into the sort of, into the the time you've allocated to either cut down on technical debt, or the time you've allocated to perfect your overall process, you know, that 20% time that we talk about in DevOps. I've seen a lot of, a lot of companies recently, they've been organising their teams differently to stop scope creep. And I think this is kind of a bit of a nod to Conway's Law, which is the idea that - have you heard of Conway's Law?

Adam

I have not.

Jess

Okay. It's this idea that we produce, the things that we produce are indelibly impacted by the way that we're organised. Anyway, it was studied by Professor Conway. And he found that he tasked two teams to produce two different compilers, and five people working on one project, I think, a COBOL project and three people working on an ALGOL project. And it turns out the finished state compilers each had three or five stages, depending on how many people were working on that product. So it shows that, we see this all the time in products that we use, usually, the way that we're organised if you have, if you have security experts on your team, you usually work in a different way. And it's can be, it's I mean, it's often to our detriment, because we often end up working with the people that we have, you know, the skills crisis, you're kind of, you have as many people as you can, you can get your hands on. But by kind of building teams really specifically and organising people into different squads and giving squads real specific remits on how, which work they're allowed to factor into their daily processes, you kind of are able to sort of clamp down on scope creep a little bit.

Hugh

What is it about scope creep, that makes it damaging?

Jess

It's really damaging for a few problems, it can really, it infiltrates the way that your, your order of processes, so it can circumvent your established way of doing things, which means that it just interrupts your team from being autonomous. If you if people know where their information is coming from, then they know where to direct their efforts. And as soon as things change, it throws... if this is different, why, is this potentially different? You start to distrust the entire system. There's like four main models that people can use to like measure organisational health, and one of them is deployment frequency and you only really, you're only really going to deploy if you're really like confident in what you're deploying, if you're confident in your product. So it shakes your confidence in what you're doing, but also shakes your confidence in the overall way of doing things. So there's the idea of a microservices architecture, it only works because you're designing for things that can fit into your head, that's what the author of Team Topologies said, you've got to design for software that can fit inside your head. So as soon as things start to become complicated by one or two degrees, quickly gets amplified, it also gets amplified because it's, you're communicating a different thing through many different people. And it's got the, you've got the added risk factor in there, right?

Adam

Yeah, absolutely. I think that one about designing software that can fit in people's heads, I think is super important. Because when you are dealing with particularly kind of business systems, as soon as you start needing complex flowcharts, and diagrams, to explain kind of how your app works, or how your software works, and what it does, it becomes much much easier for bits to fall through the cracks. And to end up with this Kafkaesque nightmare of code where, you know, there's 18 different modules, no one's entirely sure which specific functionalities each one governs. And then kind of that's, that's the stage where, you know, when one element breaks, the entire thing has to be forensically broken down and troubleshot to find out, you know, what's wrong, and how to fix it, which - and again, it goes back to the question of technical debt, I think - the larger you allow scope creep to become the greater risk it introduces of unfeasible technical debt build up.

Jess

But we're talking about scope creep like it's, I mean, it is, it is a bad thing in a lot of instances, but you kind of brought up the the idea of incidents there and scope creep can be really powerful from an incidents perspective. I mean, if we're looking at game days and chaos engineering, the idea of creating incident days where things go wrong in a way that's outside of your scope to correct, is a fascinating way of finding out just how things fit together. So I think that's like a really good way that scope creep can be used to, to kind of strengthen your organisation.

Hugh

That's a really interesting confluence, because scope creep coming out of a place of people being worried about making errors and you know, we've got to account for everything and then, but the positive side of it being creating errors and creating chaos that could be beneficial, rather than just negative.

Adam

Are there any other benefits to kind of permitting a certain amount of scope creep within your kind of development environment or project management environment?

Jess

I think that the main one seems to be stress testing your organisation, the main thing seems to be that if you permit a little bit of scope creep, scope creep, you're able to see whether or not people can, your engineers can rise to the challenge a little bit. But the only issue is, it's it's such a good idea on paper, but it's just often really, it's often carried out in bad faith. We're not in the situation yet where by rule, organisations tend to vouch for their people. And when that is the standard, then sure, we can use scope creep to stretch ourselves and to challenge ourselves. If you permit it within a set period of time, I mean, for instance, like hackathons are all scope creep, they're like a scope creep challenge to see how far you can take the bounds of something.

Adam

So we've talked about some of the ways that you can, if you like, deliberately introduce certain elements of scope creep, or a certain level of scope creep into your organisation, things like hackathons and chaos engineering, and all that kind of stuff. What are some of the main ways that scope creep comes into projects unintentionally, do you think?

Jess

So okay, so scope creep always comes into projects unintended and unintentionally when organisations are either heavily weighted towards the revenue side of the org and that happens all the time, right, because salespeople are speaking to customers all of the time, so they know exactly what they want, but also salespeople are wired in a way that they work in a very explicit manner, which is excellent, but also in very immediate terms. So hearing about a potential feature can sometimes become the difference between a deal happening and a deal not happening. And then sudden, suddenly you're, you're running towards a product with really disparate aims and no main strategy, that's often when you start to drift away from the idea, or the idea of your customer profile, right? You start designing for this person that doesn't exist, it's like the Doctor Who problem; you're demographically kind of designing something for an audience that doesn't quite... I mean, is, do people actually use that feature? And is that just this small organisation over here? That's why positioning the product team and making sure that the product and product management team is able to be essentially a bit of a fortress against these customer success aims is, is really, really, really important.

Adam

Yeah, and one of the growing categories that we've seen over the last several years really growing in prominence, is that product owner, product manager role, which is essentially kind of geared entirely around around that, around nailing down, right, what is it that we need this particular product within our organisation to do? What aims is it meeting? What, you know, what is the roadmap for developing it? And where do we need to essentially cut off development and say, No, that is actually not necessary for this, or not feasible.

Jess

Yes. One of the, one of the really encouraging things we're seeing at the moment is that a lot of - I've seen this loads on Twitter - it's a lot of senior engineers are taking, their like developing this sort of sense, this sort of this reflex to understand Wait a second, is this, does, do do we actually need to build this? I remember seeing someone recently talk about how being a senior engineer is essentially having endless conversations about why you shouldn't make something.

Adam

Yeah, exactly. It's it's the whole, you know, you were too busy thinking about whether or not you could, you didn't stop to think about whether or not you should, from famous DevOps pioneer, Dr. Ian Malcolm. Like, yeah, I think, yeah, I think that's a really, really shrewd observation, a lot of a lot of being a senior technologist in general, and a senior developer in particular, is knowing when not to do something.

Jess

And it can often be really counterintuitive, right? Because if you are, if you're an engineer on a team, and you really want to step up into into, you really want to develop your career, you often need to find your niche and able to, to be able to kind of carve out your unique path, you know, you want to be the change management person, you want to be the, you want to be the person that understands feature flagging. For you to be able to do that you need to be able to carve yourself out as a specialist. But for the general overall team to thrive, you need to still be able to be like a generalist, essentially, like you need to be able to build a team where you can hedge your bets on their knowledge, and you can bet, okay, I believe that this team will be able to encounter this area of development that they haven't had any experience in because of these sort of things that they've experienced in the past. So it's very easy for you to take scope creep and see it as an opportunity to develop not only your career, but the kind of the, the reputation of your team. But that's that's I mean, that's, that's ego speaking. That's ego speaking, and that's only going to be successful for the short term. It's not a sustainable way forward.

Adam

Yeah, everyone wants to, to stand out and to be the, you know, the team or the staff member that goes Yep, we can, we can make that happen. We can deliver on these things that that you want to do. And also it's you know, in a lot of businesses, particularly ones where the development team aren't kind of, aren't necessarily the central pillar of the business, it can be hard to say, to say no to business stakeholders who go right we want such and such functionality or you know, we want this project to deal with X, Y and Z. It takes a lot of soft skills, ability and confidence to be able to push back on that and go, Well, no, actually, this isn't going to be a feasible project or this isn't going to deliver on what you want it to do for these reasons, and to have that be, you know, received and understood and acted on by, by business stakeholders. It's a big ask.

Hugh

That's a universal problem really, isn't it, just not in dev work but in all sorts of, you know, every areas of business, it's a problem we deal with.

Jess

Yeah, it really is. I mean, it's it's with with, you know, within the within the context of a development team, you can, you can end up wasting thousands of pounds, hours, it can become an expensive headache, but we're taught to say yes, as a way of falling in with the, the people that we want to see be seen as allies to. And not to blow this up too much. I mean, I've become a bit obsessed with, with with tech failures recently, and I find it amazing how these things flow , like Juicero or even Theranos.

Adam

Beautiful.

Jess

It's incredible stuff. But people say yes, when they really should say no, and you don't really want to be seen as a downer, you don't really want to be seen as a blocker. But in essence, you need to be able to all unite around the same idea. And that sometimes requires you saying no, right?

Adam

Yeah, I mean, it's it's organisational peer pressure.

Hugh

I think it's still quite hard, though, because it's not like we all work in institutions where that isn't frowned upon. And, you know, you have to, you have to have higher ups who are willing to hear no. And I mean, and whether that's true or not, there's there's often a feeling that you can't say No. And, you know, you need to combat that.

Adam

Yeah, I think a lot of that, as well feeds back into, you know, company culture and company environment, and how, how much are you listening to your project teams and your dev teams, and, you know, just your kind of junior staff in general, you know, because if you're, if as if, as a business, your leadership are not willing to actually sit down and listen, properly listen, to what your junior staff members are telling you, then yeah, that's gonna be a problem.

Jess

These sorts of bad decisions, they thrive in silence, they, the the best people on our team are usually the ones that really want to avoid context switching, they usually are the most adverse to communication and communication could sometimes be the most intense, like time intensive training activity that we need to, that they need to engage in. So I remember reading a study recently was talking about how collective intelligence in teams, you know, the ability to be able to do something well, on a collective scale, it thrives in teams that often spend fewer words communicating with each other, the longer they spend with each other, the more they seem to understand their nonverbal cues, and whether or not that's virtually or in person, they start to understand their innate cues around something. So this is why observability has become such an important part of preventing things going wrong.

Adam

So let's quickly address the differences between scope creep and feature creep. Because both are brought up fairly often in, you know, the development world. But there there is a distinction there that's that's worth noting. So what's what's the kind of the key difference for you?

Jess

Scope creep can happen to anything; scope creep can happen to a product, a project, it can happen, it's that additional few pieces of functionality, whereas feature creep, feature creep happens when a feature diverts from its original intended purpose. It becomes, it becomes a, like, an improv game; a 'yes and' situation.

Adam

That's a marvellous way of looking at it.

Hugh

I mean, in this case, is feature creep a sort of a species of scope creep?

Jess

Yeah, scope creep is more the, is the overarching term. And feature creep is, is the additional overload og features again and again, in a sort of a continuous stream. So it's...

Adam

It's like a subset of scope creep. I think it's the the phenomenon that non developers are probably most likely to be familiar with, in in like a software context. You know, we're all familiar with social media platforms are the ones that springs to mind first for me in terms of feature creep. We're all familiar with, you know, companies like Facebook and Twitter, bolting seemingly endless additional capabilities onto their core platforms, most of which, you know, no one really asked for. And it just, it seems in a lot of cases like adding new functionality for the sake of having new functionality rather than for what it actually offers in terms of benefits.

Hugh

Twitter stories and LinkedIn stories.

Adam

Exactly. LinkedIn stories - that much demanded and widely used feature.

Hugh

I'm sat at my desk, I love working, it's great.

Adam

Definitely isn't going to end up being quietly dropped in 18 months time.

Hugh

That's interesting, because feature creep does, as you say, kind of like that's quite easy to conceive. But scope creep is kind of, the more you think about it, the more vast it feels, and the more different ways it can come in. And different ways that it can cause chaos.

Jess

Feature creep is usually influenced directly by commercial pressures. So I don't know whether or not you've noticed this, whether or not, I think it's more of a sort of a thing if you're, if you use an iPhone, but your notifications are increasingly being used as sales, collateral or marketing, marketing collateral, and that's not necessarily what they're for. And you can kind of see that this has been led by the top down, even when you could jump into the settings feature, you can see sometimes a try this feature for x pounds per month. And that's it's that's the that's where you kind of see the commercial pressures of elsewhere in the business being leaked down into the into places where we shouldn't frankly be touching, and the highest person's opinion is getting weight over the others.

Adam

Yeah, absolutely.

Hugh

I mean, talking about saying no, try saying no to, to the higher ups about monetizing things.

Jess

Exactly. Because if you say no to monetizing, you're saying no to the success of the product overall, aren't you? Well, I mean, don't you want a job, I mean, this kind of... But that's, this is the thing. So some things need to stay local, you know, if we, if we argue for everything on a global scale, then it's really easy to find yourselves quickly going from discussing the use of a certain tech stack to whether or not we should be paid x pounds per month or what or whatever it's it's, we kind of need to keep things at a certain level to be able to argue for their categorical importance or unimportance.

Adam

So we've touched on this a little bit already. But let's talk a little bit about some of the methods and strategies that organisations can use to prevent scope creep from infiltrating their projects.

Jess

Sure, so scope creep, it's, it's a people problem, it's an organisational problem. So we need to look at how we're, how decisions are being made, and standardise that across the business. And I know that sounds like such an eye rolling exercise for everyone involved. But every single year, whenever the state of DevOps report is completed, it seems to find uniformly that when orders of interactions are defined, people become more satisfied with the outcome of their decisions. That's not to disrupt the org chart or the hierarchy of the people that you're working with. It's just so the expectations are set up so that people know how to have their voices heard. And people have a direct route to saying, to raising concerns about something, about about saying actually, I think this is going to, this is going to have us drift away from the main thing that we're focusing on. Another really important part is including feedback loops, and closing and making sure they're really fast. One of the merits of of working with production data is that you know what you're, you know that you're grounded in fact, whether we're talking from a sort of testing perspective, or, or just in terms of testing our, testing our theories, there is there is no production like production, we can't stage our decisions, and we can't stage our environments. So we need to keep things grounded in reality, and that means that we get tied to the same version of the truth. So testing in production is a great way to stop scope creep from happening because it prevents us from going down and into these imaginary realities where people will buy this feature that you haven't made yet, that doesn't necessarily exist and that might not even save the company keeping teams small, we've spoken about that. But the squad model I think is really interesting, because the squad model unites a small number of people like say eight people around one problem. And it forces you because because these teams are quite small and because the the the point of those teams is so specific, it causes you to create many more of those teams. And look at enablement teams way faster than you otherwise would. You stop having these big silos of teams and think, oh, gosh, we've now got this many engineers, we need to be able to support the engineers, let's have a supporting engineering team. And it's easy, it gets you away from having those layers, and it starts, it gets you to think about how to support the output a lot sooner. One, one thing that - it's really early on, but it's a project that I'm working with, with a couple of people - is around the idea of making it compulsory for team members to switch roles. A while ago, we saw that DoorDash was really, it was really insistent that everyone throughout their company, no matter whether or not they were on a super high pay packet or working in product, or if they had to have some sort of customer experience hours in in their contract. And I think grounding people in the actual work that you're, the actual kind of aim of the businesses is really important. But it's, it's about what form that takes, right? I mean, is it enough to just talk about your problems, is it enough to just to just have monthly meetings where the technical support team feed back the problems that that that that your users are having? Or do you actually need to be... how much do you need to consume your own cat food, dog foods, animal foods metaphor, for your activities to be productive?

Adam

I think that's a wonderful kind of idea. We were discussing a very similar thing with Ocado's James Donkin a couple of episodes ago about how when Ocado first started, basically, everyone did a couple of shifts in the warehouse doing the, you know, picking and packing and loading orders and everything. And I think that that is really vital, not just for, you know, customer service businesses, but for every, every organisation should have some mechanism for, as you say, grounding, you know, the workforce at large in what the organisation does, and what the priorities of of the organization's should be. When looking at, you know, expansion and, you know, new features and tightening up processes, you know, what are, what are the actual priorities, effectively?

Jess

I recently recently spoke to Patrick Debois, he works over at Snyk, and he's just a really interesting person, it was talking about how the majority of our problems come up when we tried to cram 100% utility into our teams, right? We try to make everything the peak of efficiency, and to meet all of our aims with all of our hours. And that's just not how we work, we need to break down - while we don't need to, we shouldn't be working in silos - we need to break down our time a little bit to make sure that we're sectioning off certain hours for being able to allow feedback to happen and to understand whether or not we're actually meeting the problem that we're trying to solve.

Hugh

Well, sadly, we're out of time for this week's episode. But thanks once again to LaunchDarkly's Jess Cregg for joining us.

Jess

Thanks so much for having me.

Hugh

Pleasure.

Adam

Always a pleasure to have you on the show, Jess.

Hugh

That was really interesting.

Adam

You can find links to everything we've spoken about today in the show notes and even more on our website, itpro.co.uk.

Hugh

Don't forget to subscribe to our social media and YouTube channel for more great content and leave the podcast a rating and a review.

Adam

We'll be back next week with more analysis from the world of IT. But until then, goodbye.

Hugh

When you’re working on a large-scale software development project - especially as part of a sizable enterprise business - it’s easy for the project’s original goals to become muddied over time. As the goalposts shift further and further from their starting point, timelines can slip and the efficiency of the project can be undermined.

This kind of scope creep can be seriously damaging for organisations, but thankfully the software engineering community has been hard at work for years on strategies to help limit its spread within organisations. Returning to the podcast this week to discuss some of these strategies - as well as how scope creep can sometimes be beneficial - is Jess Cregg, developer advocate at LaunchDarkly.

Highlights

“It's really easy for it to happen, because as soon as it's just one more thing, things can spiral out of control. And the ‘just one more thing’ rule doesn't usually end up eating into the main time that you frame for your project, for the thing that you're developing; it falls into the time you've allocated to either cut down on technical debt, or the time you've allocated to perfect your overall process - that 20% time that we talk about in DevOps.”

“Scope creep can be really powerful from an incidents perspective. I mean, if we're looking at game days and chaos engineering, the idea of creating incident days where things go wrong in a way that's outside of your scope to correct, is a fascinating way of finding out just how things fit together. So I think that's a really good way that scope creep can be used to strengthen your organisation.”

“One of the really encouraging things we're seeing at the moment - and I've seen this loads on Twitter - is a lot of senior engineers are developing this sort of sense, this sort of reflex, to understand, ‘Wait a second, do we actually need to build this? I remember seeing someone recently talk about how being a senior engineer is essentially having endless conversations about why you shouldn't make something.”

Read the full transcript here.

Footnotes

• What is a microservices architecture?
• How microservices help Perlego deliver textbook availability
• What is DevOps?
• The driving forces behind DevOps today and how to integrate it into your business
• How to become a DevOps engineer
• IT Pro Panel: Defining DevOps
• Seven skills you need to become a front-end developer
• How to become a developer: A beginner’s guide
• 7 UX skills every designer should have
• Synk to move into cloud security with Fugue acquisition
• The IT Pro Podcast: Picking a programming language
• The IT Pro Podcast: Building the supermarket of the future

Subscribe

• Subscribe to The IT Pro Podcast on Apple Podcasts
• Subscribe to The IT Pro Podcast on Google Podcasts
• Subscribe to The IT Pro Podcast on Spotify
• Subscribe to the IT Pro newsletter
• Subscribe to IT Pro 20/20

Provided by

What makes a company agile?

IDC’s Robert Parker surveyed nearly 2,000 execs to uncover the core tenets of agility, finding low-code application development can contribute faster execution. In fact, 53% of highly agile companies had an excellent rating for time to market, leaving non-agile companies in the dust.

In today's accelerating digital economy, business agility has become an imperative, and in order to assess your organisation's own agile capabilities, you need to consider five dimensions:

• Leadership
• Structure
• Process
• Portfolio
• and Technology Architecture

Download this guide for best practices and go on to achieve superior financial results.

Software developers can work well alone, if they're in control of all their software assets and tests. Things get trickier, however, when they have to work as part of a team on a fast-moving project with lots of releases. A group of developers can contribute their code to the same source repository, like Git, but they then have to run all the necessary tests to ensure things are working smoothly. Assuming the tests pass, they must build those source files into executable binaries, and then deploy them. That's a daunting task that takes a lot of time and organisation on larger software projects.

This is what Jenkins is for. It's an open-source tool that co-ordinates those stages into a pipeline. This makes it a useful tool for DevOps, a development and deployment approach that automates the various stages of building software, creating an efficient conveyor belt system. Teams that get DevOps right with tools like Jenkins can move from version roll-outs every few months to every few days (or even hours), confident that all their tests have been passed.

Jenkins used to be called Hudson, but its development team renamed it after Oracle forked the project and claimed the original name. It's free, and runs on operating systems including Windows, Mac, and Linux, and it can also run as a Docker image.

You can get Jenkins as a downloadable from the Jenkins.io website, but you'll need to run the Java runtime environment to support it. Alternatively, you can install it as a Docker container by following the instructions on the official Jenkins site, which is what we'll do here. Docker takes a little extra work to set up, but the advantage here is twofold: it solves some dependency problems you might run into with Java, and it also enables you to easily recreate your Jenkins install on any server by copying your Docker file and the Docker run command to run it, which we put into a shell script for increased convenience. Jenkins' Docker instructions also install a souped-up user interface called Blue Ocean. If you don't use the Jenkins Docker instructions you can also install Blue Ocean separately as a plugin.

First, we must create a Python program for Jenkins to work with. We created a simple file called test-myapp.py, stored on our Linux system in /home/$USER/python/myapp. It includes a basic test using the Python PyTest utility:

Create a Github repository for it using git init. Commit the file to your repo using git add ., and then git commit -m "first commit".

Now it's time to start Jenkins using the docker run command in the Jenkins teams' Docker instructions. Once Jenkins is running, you can access it at localhost:8080. It will initially show you a screen with a directory path to a file containing your secure first-time access password. Copy the contents of the file to get logged into the administration screen, and from there it will set up the basic plugins you need to work with the software. Then, you can create a new user account for yourself.

If you're not already in the Blue Ocean interface, click on that option in the left sidebar. It will ask you to create a project. Call it myapp and then select Git as the project type in the Blue Ocean interface.

Blue Ocean will now ask you to create a pipeline. Click yes. We're going to write this pipeline ourselves in a 'Jenkinsfile', which we'll store in our myapp folder.

A Jenkinsfile is a test file describing your pipeline. It contains instructions for the stages of each build. It looks like this:

Each stage reflects a step in the build pipeline and we can have as many as we like. Let's flesh out this template.

Python programs don't need building and deploying in the same way that, say, C++ programs do, because they're interpreted. Nevertheless, Jenkins is useful in other ways. We can test our code automatically, and we can also check its formatting to ensure that it's easy for other developers to read.

To do this, we need to compile the Python program into bytecode, which is an intermediate stage that happens when you run Python programs. We'll call that our build stage. Here's the Jenkinsfile for that step:

The agent is the external program that runs this stage. We don't define a global one, but we do define one for the individual stage. In this case, because we're using Docker, it's a lightweight Alpine container with a Python implementation.

For our step, we run a shell command that compiles our python file.

Save this in your project folder as 'Jenkinsfile' and then commit it using git add . and git commit -m "add Jenkinsfile".

Back in the UI, ignore Blue Ocean's prompt to create a pipeline. Once it spots the Jenkinsfile in your repo, it'll build it from that automatically. Go into the Jenkins dashboard by clicking the exit icon next to the Logout option on the top right, or clicking the Jenkins name on the top left of the screen, to get to the main Jenkins screen. Look for your new project in the dashboard and on the left, select Scan Multibranch Pipeline Now.

Wait for a few seconds and Jenkins will scan your Git repo and run the build. Go back into the Blue Ocean interface, and all being well you'll see a sunny icon underneath the HEALTH entry that shows the build succeeded. Click on myapp, then Branch indexing, and it'll give you a picture of your pipeline and a detailed log.

Now we will add a test stage. Update your code to look like this:

We're using another Docker container to run a simple PyTest (which we included in the code of our myapp.py file). Save this file and update your repo with another git add a . and git commit -m "add test stage to Jenkinsfile". Then, scan the multibranch pipeline as before. When you drop into Blue Ocean, you'll hopefully see success once again. Note that Docker stores everything it runs in its own volume, along with the results. Although you can work some command line magic to access those files directly, you don't need to; Jenkins shows you those assets in its UI. Click on the latest stage to open the build details, and find the entry that says py.test –verbose –junit-xml test-results/results.xml testmyapp.py. Clicking on that shows you the results of your test:

Everything passed! Now we're going to bring it home with the final stage in our demo pipeline: checking the code formatting. There are specific rules for formatting Python code as outlined in the language's PEP-8 specifications. We'll update our Jenkins file to use a tool called PyLint that will check our code. Here's the full Jenkinsfile for all three stages of our pipeline:

Follow the same steps as before: save the file, commit it to your Git repo so that Jenkins sees it, and then rescan the multi-branch pipeline. Then go into Blue Ocean and look at the result. Oh no!

The pipeline stage failed! That's because our code is badly formatted, and PyLint tells us why. We'll update our test_myapp.py file to make the code compliant:

Now, save, commit to your repo, and rescan. Blue Ocean shows that we fixed it (note that in our demo it took us a couple of runs at the Python code to get the formatting right).

You could run all these steps manually yourself, but the beauty of Jenkins is that it automates them all for faster development. That makes the tool invaluable for developers working on a fast cadence as part of a team, but even a single freelance dev, or a hobbyist working on open-source projects, can use this to refine their practice.

Much effort has been put into trying to divine the secrets of what makes tech titans like Netflix, Amazon and Spotify so successful. There are many factors in their meteoric rise, but one common thread is the fact that they all use DevOps methodologies to build their software and services. They’re not alone; DevOps has become a pervasive trend within the software development world, and a huge number of organisations across a range of sectors have adopted the practice.

DevOps can mean different things to different people, however, and one of the most persistent debates within the community is how to actually define a DevOps structure; what essential elements have to be in place for it to qualify? In this month’s IT Pro Panel discussion, we asked some of our expert panellists who have undertaken this journey for themselves what makes DevOps so compelling, and how they’ve navigated some of its more challenging aspects.

Speed freaks

DevOps has many advantages, but the most commonly cited benefit is development velocity. IT leaders want to be able to release more code at shorter intervals, speeding up time to value for the business and addressing changes in a more timely manner. The practice is built on the concept of continuous integration and continuous delivery (CI/CD), and it’s this focus on pushing code as frequently as possible that drives much of the thinking in its implementation.

“Three years ago, we did around one release every two weeks, and it took us about two to three days to get to production,” explains Yolt CTO Roderick Simons. “It was too painful, took too much time and we had to improve.”

“As a first step,” Simons says, “we increased the frequency of delivery from once every two weeks to once every week. If something hurts, you have to do it more often.”

This allowed the team to get faster feedback on changes, and from there, they mapped out their development process in order to identify major bottlenecks. Now, he says, Yolt is able to do daily code releases. To support this, Yolt uses a highly-automated CI/CD pipeline, which is something that SmartDebit CIO Gavin Scruby and TempCover CTO Marc Pell have also been working on.

Scruby, Simons and Pell are all using automation tools to speed up their code releases, but some of our other panellists say they’re concentrating on alternative elements beyond speed of delivery. For instance, design house Studio Graphene has a number of internal checks and controls to ensure that code is stable before being pushed out to clients, which founder and CEO Ritam Gandhi notes must be balanced with agility and speed.

For Craig York, CTO of Milton Keynes University Hospitals Foundation Trust, making sure that all new code is stable and fit for purpose is also paramount. Even outside of the ongoing COVID crisis, healthcare IT systems operate with a razor-thin margin for error, and York’s team focuses on maintaining the availability of systems as much as possible.

The hospital’s IT estate includes systems from many external suppliers, so every time one of them releases an update, the MKUH DevOps teams are responsible for the customisation and integration work necessary to ensure it doesn’t impact the delivery of clinical care. This often involves working as “testing and development partners”, in York’s words, as well as liaising with non-IT teams across the business.

“User engagement from our organisation has been critical for us over the past few years,” he explains, “with a clinical consultant and nursing staff now embedded within IT. Each week, our change/release board includes these staff and managers from across development groups and operational IT to ensure that, as a group, we’re ready and confident of all releases.”

The DevOps journey

It’s a common mantra within the community that establishing a DevOps culture is an ongoing process, and for most of our panellists, the finish line isn’t yet in sight. Newcastle Building Society, for example, is still in the relatively early stages of establishing its DevOps capabilities. The company has set up a pilot programme, putting its mobile app product in the hands of a ‘model team’ to act as a testbed for DevOps practises and tooling, which CIO Manila McLean hopes to roll out across the rest of the development organisation.

“This team is fully DevOps,” McLean says; “multi-skilled, and accountable for product ownership, user experience, development, testing, release, et cetera. The team is accountable for the holistic product including the operational performance and future roadmap, so they need to balance between business as usual and strategic developments.”

She plans to expand this strategy over the course of this year, but notes it’s not simply a case of spinning up more DevOps teams. In order to replicate this model team’s structure across other products, buy-in will need to be obtained from the rest of the business first. While McLean is positive about the progress made on this front so far, she admits that it’s going to take time.

“It's really important to take the business on the journey, so we can't rush this.”

This journey doesn’t stop once product teams are successfully in place, however, as Pell can attest. While he’s already established CI/CD pipelines and automated elements of TempCover’s infrastructure, he says he’s closer to the middle of the process than the end, and that there’s still more to do to automate further aspects of its infrastructure to allow for more frequent releases.

“Similar to Marc's response, we are somewhere in the middle,” Gandhi adds. “It's a mindset and skill that we really started to focus on a few years ago and one of the nice things about constantly working on new blank canvas projects is that we have been able to adapt and try different approaches, before adopting them holistically across our business.”

For those panellists who have successfully productised their engineering organisations, the focus now is on maximising the efficiency of their internal tools and pipelines, tuning processes and deploying further automation where possible. As Scruby notes, improvements in one area of a DevOps structure often naturally lead into another.

“Once you start with automating testing a bit to improve quality, it links within similar tools and skills to more automated deployment, then more automated integration, and then a second cycle with more efficient tooling once people have become familiar.”

Tooled up

This feeds into what is arguably one of the most important aspects of DevOps: An organisation’s choice of tools. The library of tools that DevOps professionals have access to is vast, much of it open source, and the variety of options available allows teams to customise their toolchains based on the specific details of their requirements.

The most popular software tends to be built to assist with automation and testing, largely because it’s so fundamental to the DevOps process. There was a wide spread of favourites among our panellists in this regard, including JUnit, NUnit, Cucumber, Jenkins and Selenium, demonstrating the sheer breadth of choices in this category.

Almost all of our panellists, however, were united in their use of cloud infrastructure for DevOps. In addition to a pair of on-site computer rooms, York makes heavy use of Microsoft Azure and its PaaS capabilities, which his devs tell him make it easier to automate their processes. McLean, meanwhile, uses the YAML pipelines offered by Azure DevOps, as well Terraform for automated infrastructure builds and Packer for creating machine images.

Pell also uses pipelines in Azure DevOps for the actual automation aspect of TempCover’s releasing, although he notes that the company does still have a few legacy on-premise CI/CD systems that it’s yet to port over. This will be a priority for 2021, he says, as it represents a large proportion of the company’s remaining on-prem estate and will prove to be an efficiency gain and cost saving once the migration is complete.

SmartDebit is the main outlier in this case; the company is mostly based on-premise, and uses Nutanix’s private cloud technology (which Scruby describes as “expensive but less effort than OpenStack”) to deliver its applications. This naturally has an impact on which tools are most appropriate, but SmartDebit’s situation isn’t all that unusual.

“We used to run on a private cloud but transitioned to the [public] cloud a year and a half ago,” explains Simons. “We primarily started with IaaS but are moving higher in the stack, with some managed services on the data side and data storage. Similar to Manila, we use Packer for images and use Terraform for all our infrastructure-as-code. In our DevOps journey, this was actually a key enabler; giving all teams their own ‘team environment’. To keep the costs down, we scale it up in the morning and destroy it at night.”

It’s easy to see, then, that cloud technologies like PaaS and IaaS are a natural fit for DevOps toolchains, but there was a little less consensus around the necessity of container technology. Kubernetes, in particular, was identified as a framework with significant potential for enabling DevOps, but it’s not without its drawbacks.

For McLean, Kubernetes allows her organisation to develop, deploy and scale their applications faster, with greater predictability, but others noted that the value proposition isn’t as straightforward as something like cloud infrastructure. Gandhi points out that cloud platform tools like AWS ECS can often be used to accomplish the same goal, while Scruby adds that additional systems such as general virtualisation, load balancing and distributed storage may already be delivering many of the benefits of containerisation.

“There isn't the same step in advantage from virtual machines to containers as there was from tin to virtual machines,” Scruby explains. “VMs are, in essence, just flabby containers, so can do most of the job. Changing a whole infrastructure to something that provides an incremental advantage is a hard sell to a board, especially when public clouds often have their own serverless technologies to promote as well.”

“Exactly,” Gandhi adds; “I think migrating a large scale existing infrastructure is much harder compared to when you're kicking off a new blank canvas build. Hence, I completely agree that it just takes time and if the incremental benefit isn't in line with the effort, it's difficult to prioritise.”

Priority transmission

This highlights one of the principal challenges identified by our panellists in relation to DevOps, which is the issue of prioritisation. For many of them, it’s been difficult to balance working on elements of their DevOps platform – such as CI/CD pipelines or automation tooling – with keeping the lights on and performing normal business tasks.

Part of Yolt’s strategy for addressing this problem was to set up what Simons calls a “platform team”; an internal group made up of members of various existing teams, with a remit to work on creating self-service tools for other dev teams within the organisation. In addition to the company’s CI/CD pipeline, this team is also responsible for maintaining elements such as the end-to-end testing suite, performance testing and code security tools.

“We share the same challenge of prioritising resource between normal business development and DevOps,” Pell says. “I'm a firm believer in ownership of concerns and therefore completely agree with the principle of a platform team; coincidentally, we're hiring a team with exactly that name right now!”

This approach has been somewhat challenging for Scruby, however; like Yolt, SmartDebit is beholden to a number of regulatory and compliance requirements, which he says can make breaking down the barriers between teams difficult. He also cites the impact that a DevOps model can have on processes outside of the IT organisation.

“There’s a knock-on effect outside of product. For example, the ancillary teams covering customer engagement, marketing collateral generation and support who can struggle to keep customers up-to-date with change, especially as some customers are institutions that may not have strong employee training or engagement in place. There's also a little-mentioned aspect of accounting practice that’s impacted by DevOps improvements – how to account for system capitalisation and R&D costs. Accountants are still used to big product releases for the balance sheet and struggle if something does not fit neatly into ‘widget factory’ type models.”

Squad tactics

All of our panellists agree that DevOps is as much about people as it is about technology, so the structure of teams is key for a successful implementation. The general consensus is that groups of between four and ten individuals represent the ideal format for a DevOps team, although while Gandhi encourages these squads to be fully cross-functional, Pell takes a slightly different tack.

“We've followed a similar squad structure which features natural specialisms, but with an understanding that cross-functional requirements exist,” he says. “This covers all of our software development across the entire business, so everything from an entirely new project to a feature iteration or bug fix.”

McLean also runs this kind of structure, with a scrum team of around ten people currently acting as the model team in charge of the development and operation of Newcastle Building Society’s mobile app, supported by a platform engineering team. Scruby, meanwhile, says that SmartDebit is too small to support a separate platform engineering team, but the company is extending and empowering its existing scrum team to do more. For compliance reasons, however, it does maintain a separate Ops team responsible for reviewing and controlling its automation pipeline.

“For us, a feature team consists of all the roles needed to deliver the end-to-end journey,” says Simons. “We promote the ‘t-shaped’ or full-stack principles, and all engineers have access to the code base and can make merge requests if needed. We don't have special QA roles; the team is responsible for testing themselves.”

This is a common theme; giving DevOps teams the autonomy and responsibility to run their respective products was cited by the majority of our panellists as an essential part of building a solid DevOps foundation. McLean and Pell also advise that technical and non-technical staff must work extremely closely, and must all be equally responsible for successful project delivery.

On the other side of the coin, McLean echoes Scruby’s point that DevOps professionals need to consider what their code actually does in practice, not just how well it runs. This, Scruby says, can be a hard mindset for some developers to change.

“I completely agree,” McLean adds; “A DevOps team is like running your own business, you need to care about every facet of your product.”

“I'd say the most important factor is an eagerness to embrace a DevOps ‘mindset’ and step outside your comfort zone,” Gandhi says. “Without it, the cultural shift within the team won't really happen and one will just adopt the status quo. One of the pitfalls is to think that setting up the team means your responsibilities are complete. It's an ongoing journey that will continue. There are systems and processes that need to be embedded within the delivery ecosystem - which is a gradual process.”

To apply to join the IT Pro Panel, please click here to enter your details. Please note that we are not accepting applications from technology vendors at this time.

NATO has awarded Viasat and CDW a two-year technical innovation contract for Command, Control and Communication (C3) support for Programme LELANTOS.

This agile experimentation initiative will provide NATO's Allied Rapid Reaction Corps (ARRC) headquarters, located in Gloucester, UK, with “superior decision-making, cross-domain integration, and fast manoeuvre/survivability in adversarial conflicts.”

"This is a defining moment,” said Lt. Col. Steve Short, lead, ARRC Programme. “We have looked at the whole ecosystem of the deployed working environment and see opportunities to harness best-of-breed technologies that will deliver real advantage to our Commanders and their staff in the field.

Equally important is the development approach we are taking with our industry partners, Viasat and CDW, which is innovating both technical and commercial processes."

Under Programme LELANTOS, the two companies will work with ARRC, the Ministry of Defence and industry partners to apply agile development processes to test new capabilities in the C3 domain. The collaboration will support the experimentation of secure wireless military networks involving enhanced battlefield command and control capabilities.

“Programme LELANTOS provides an opportunity to challenge the status quo and drive agile C3 innovation across NATO and its allies,” said Steve Beeching, managing director at Viasat UK.

“This partnership will leverage Viasat's deep breadth of defence experience and innovative technology approaches, alongside CDW's proven track record for delivering digital transformation solutions across multiple industries to bring ARRC rapidly-deployable, highly-secure and scalable land formation capabilities, which are critically-important in a rapidly evolving battlespace.”

Nick Garland, director of defence at CDW, added: "It is an honour to be collaborating with Viasat UK, as we lead technological change across multiple domains that will see real-time benefit to a strategic headquarters through agile C3 experimentation.

“We believe Programme LELANTOS will deliver technological transformation aligned to the core objectives of both the MoD and ARRC."

CDW is a multi-brand technology solutions provider to business, government, education and health care customers in the United Kingdom, United States and Canada. The company has an international presence in private and public sector IT, and provides products and services in more than 130 countries.

Viasat is a global communications company providing internet, satellite, mobile and related services to consumers, businesses, governments and militaries. The company is developing a global communications network to “power high-quality, secure, affordable, fast connections to impact people's lives anywhere they are - on the ground, in the air or at sea.”

DevOps has become a very popular methodology among digital-first companies, but it can be tricky to get right - especially if your thinking is still based on older models of software development. Join Puppet field CTO Nigel Kersten as he takes us through the process of moving away from multiple cross-functional DevOps teams towards a truly unified, platform-centric approach.

Nigel Kersten

Field CTO, Puppet

Nigel Kersten, Field CTO at Puppet, leads a team of engineers to build greater knowledge around how customers around the world are using Puppet to ensure the company is building pragmatic products for current and future customers. In addition, his team focuses on building out service strategies and finding new opportunities and geographies to expand Puppet’s commercial footprint. Nigel brings a worldwide perspective to the challenges some of the biggest companies on the plant have with infrastructure automation, continuous deployment, compliance and DevOps.

Nigel works with Puppet’s largest customers on the cultural and organizational changes necessary for large scale DevOps implementations. His deep expertise and research into DevOps give him invaluable experience and understanding of what it takes to make DevOps work at scale. Nigel has served in a range of executive roles at Puppet across product and engineering over the last 9 years, and came to Puppet from the Google SRE organization, where he was responsible for one of the largest Puppet deployments in the world.

By now, it shouldn’t come as much of a surprise to anyone that data is an important tool for businesses. Proper collection and analysis of data can reap huge benefits such as greater efficiency, higher turnover and increased agility. But what happens when the data tells you something that you would prefer not to hear? Just how ruthless should a business be when it comes to analysis of its processes and performance?

Some organisations favour a more subtle, empathetic approach, using data to add context to the decision-making process. Others focus their use of data on purely mathematical questions, like how much stock they’ll need for the next quarter. There is another, more mercenary school of thought, however; one which holds that data should be used to govern all aspects of a business, even the more creative-led elements.

One example of this is Voodoo, a French mobile games publisher that specialises in the ‘hyper-casual’ free-to-play market. Dominated almost exclusively by pick-up-and-play mobile games such as Flappy Bird, the market is incredibly competitive, with titles becoming overnight sensations and then fading into obscurity just as quickly.

To try and keep up with these rapid cycles, Voodoo uses a ‘LaunchOps’ system to manage and track the releases of new games from its studio partners. This system allows Voodoo to optimise the content and release of new titles for maximum impact, says Voodoo’s LaunchOps product manager, Antonin Chapelon, and data is a crucial element of its strategy.

“Data is a vital part of Voodoo’s development design, and tells us what works and what doesn’t in a project,” he says. “The goal behind our usage of extensive data is to give technical and artistic guidance to our partner studios, who would otherwise have some difficulty in effectively developing a title for a worldwide audience. But this data isn’t just used on a case by case basis, it provides a reference point for all of our ongoing projects. We share the key information and lessons on good practice we have validated with our data collection throughout over 150 game launches.”

When Voodoo starts working with a studio on a new game, it establishes a range of KPIs for the title based on its main strengths and the elements that Voodoo wants to focus on. This is then followed by using a broad range of data points to build a decision tree, which is used to determine the next steps in the development process. Due to the extremely short turnaround times required by hyper-casual development, this data needs to be collected and ingested in an extremely rapid manner in order for it to be useful.

According to Chapelon, this doesn’t leave much time for things like validation of A/B testing, but Voodoo’s games have large numbers of players, all of whom provide valuable analytics data. This data can be fed into Voodoo’s models and analysed in order to tune the development of a particular element or feature.

“So for example, say in an early game build, we decide that we need to focus on two different angles to improve a specific aspect in the game,” explains Chapelon. “This can be something as simple as a certain texture colour, or visual feedback on an action that just isn’t quite there yet.”

“Our next step is that we track all the data possible to help us make a decision on what is working and what isn’t, which allows us to focus our efforts on the most efficient course of action. Once we have made our choice, we use data again to track if we have made the right amendments, reviewing user information such as retention and play time.”

So far, all of this is fairly standard practice for any data-led organisation. Where Voodoo’s situation becomes somewhat different, however, is that sometimes its data analysis can result in unpopular decisions about the nature of a game’s development. The hyper-casual market is extremely competitive, and if the data shows that a game isn’t likely to be successful enough to meet its target, then Voodoo has no qualms about shutting it down in order to focus on something with greater prospects.

“Based on our findings, some projects require major overhauls and others, unfortunately, do not make it through our testing process,” Chapelon says. “It can be tough to put a hold on a project when good work has gone into it, but it’s important to ensure we focus all of our efforts on games that are showing the most promising results.”

Perhaps understandably, the focus is on revenue generation, and Voodoo’s internal data tools include prediction models to assess which features of a game are more likely to be profitable than others. ‘Win-rate’ – or the number of users that beat an area of level of a game – is also an important metric for Voodoo, as this is key to continued player engagement.

“Almost every new feature we push live has the goal of adding depth to the user's progression, which is why most of our updates come with a good uplift of our KPIs,” Chapelon says. “There are of course circumstances where we find ourselves validating a feature that could very well end up unpopular. But, by using data to gain greater insight from our testing, we are always able to make sure that it incorporates correctly within the user's flow.”

This may seem like an uncomfortably mercenary attitude for a company to take, particularly regarding creative projects like games. One imagines unpleasant conversations when developers are told that their recently-released game isn’t going to make the grade – but Chapelon assures IT Pro this attitude, painful though it is, is essential to Voodoo’s continued success.

“This is an extremely important part of our process, as we are able to teach developers where they should focus their efforts to ensure they are able to create their next hit confidently and efficiently,” Chapelon says.

“[Data] has allowed us to work with developers to turn their games into chart topping hits, all thanks to preliminary data that helps us to pinpoint important statistics about the game. Using data means we can ensure that our games are exactly what our audience wants. Data helps us to predict the potential of our prototypes. With it, we can put the right resources in the right place; this is the great strength of having a LaunchOps [programme].

The distributed workplace is here to stay and the disruption caused by the coronavirus pandemic will have a lasting impact. These are two of the key messages from the Day One keynote of the annual Appian World conference.

Taking to a much smaller stage than the one he would have been expecting just a few months ago, Appian CEO Matt Calkins acknowledged the unusual nature of the virtual conference, saying: “2020 [has been] a strange year .... Appian World was postponed, and you’re probably watching this from home and I’m speaking to an empty theatre.”

He noted that even when the world does slowly start to come back to normal, the world of business we return to won’t be the same as the one we left.

“We’re going to come back to a slightly changed and different business environment than we left. And it will be different in a number of ways, but one important way is it is that we will have come to a new understanding as to the importance of change.

“We’ve seen how essential it can be for a business to be able to change itself rapidly. We know that the fate of the enterprise can depend on an organisation’s ability to change.”

Calkins’ words were underlined up by Darin Cline, EVP of Operations at Bank of the West – a subsidiary of BNP Paribas – during a panel session.

Asked by moderator Marc Wilson, co-founder and SVP of partnerships at Appian, how his organisation has responded to the continually changing environment, he said: “We’ve had to be more agile than ever before. I think you sometimes see the meme about ‘what led your tech transformation’ and I think this has moved us closer to [being an agile enterprise] moreso, more powerfully than anything we’ve ever seen in the past.”

“To us this has been a human crisis,” he added. “Our customers need to have solutions that make things easier, because everything is harder right now and internally we’ve had to help our people because we were not a work from home organisation and now, like everyone else in the industry, 88%-90% work from home.

"And so all of our tuning for those first six weeks was to get together daily and listen to the voice and listen to our customers and what they’re going through, what are their hot-spots that we need to come together to deliver.”

Fellow panel member Darren Blake, COO of Bexley Health Neighbourhood Care, added: “From a digital point of view, we developed from nothing over the course of five days an application to track our patients out of hospital into the community and be able to assess the level of care that they needed … so we did that with Appian in five days, we worked over a weekend, we worked into the night and it was the most agile piece of work I’ve ever completed in my lifetime.”

He added that for his organisation, both remote working and the need to adapt to rapidly changing needs via agile development are here to stay.

“We’re planning on probably a two-year cycle of moving through this. There will be a second wave, a third wave and a fourth wave with different contexts to [them] and it’s only the point at which you get an immunisation that that will seal it for people to be able to return to more normality. So we’ve got a phased approach that we’re going to go through here.

“We’re fully remote now, we’ve got digital technology in all of our general practice surgeries and patient contact is primarily through video consultation. That will not go away now – that digital revolution that happened in a matter of weeks was something that was challenging for us in terms of getting this done, so out through the next few phases of this we will leverage the good that has come out of this in terms of the way we work.

Agility: it's something that every organisation claims to value, and is often touted as one of the cornerstones of modern business success. And yet, it’s also something that many businesses seem to struggle with, as agile initiatives wither and die while employees fall back into old ways.

So why is it so hard to get agile to stick within larger organisations, and how can tech leaders set up agile projects for success and long-term adoption? In this month’s IT Pro Panel discussion, we spoke to some of our expert panellists to find out how they’ve tackled this delicate issue.

Defining agility

One thing that all of our panellists agreed upon was that building a truly agile business is about far more than using agile methodologies, drawing a distinction between agile development and business agility. They also highlighted that the true measure of agility is not the speed at which an organisation moves, but the speed at which it reacts.

“Agile for me is a cultural mindset,” says Moonpig CTO Peter Donlon. “I think it's too easy for leaders to see agile as a framework or terminology that's magically going to transform that organisation. The reality is, it's a different way of thinking that can feel uncomfortable to some at first. Within Moonpig, I see it as our ability to respond quickly to change in the market, our willingness to take risks and our ability to learn from those risks.”

According to TRX CTO Dan Jacobs, many organisations try to use pre-rolled agile frameworks as a shortcut to true agility, but they’re often implemented without the spirit of agile.

“Being agile means being able to respond quickly to change,” he says; “To change direction, radically if needed, in a short period of time. I see an agile methodology adopted in many places, but no agility - agile without being agile. The process can get in the way.”

Similarly, Just Eat CISO Kevin Fielder notes that using agile development without adopting the mindset more broadly does not equate to agility. “You can even be agile in your dev, but still slow,” he says. “With my security hat on, I also think you can be the reverse - you can be an agile security team delivering at pace and able to pivot, even in a non-agile organisation.”

Data is essential, says Jacobs, and you have to be willing to make course corrections based on what works and what doesn’t. A further consensus was that without firm and wholehearted buy-in from senior management, any agile efforts are doomed to failure.

“I completely agree with Dan and Kevin,” Studio Graphene co-founder Ritam Gandhi says. “Agile needs to be embedded within an organisation’s DNA for it to work and this has to come from the leadership team.”

How do we achieve agile?

Gandhi adds that an essential part of the agile mindset is allowing for failure and creating a culture in which failures are not punished, but used as an opportunity to learn and adapt. Using a hypothesis testing approach is also key, through which organisations can validate their assumptions and ensure they’re taking the most appropriate action.

“I think you can solve a lot of the blame and fear of failure problem through transparency and sharing early,” Donlon muses. “I think the more people are brought on the journey through the formation of a hypothesis, through the testing, to the outcome, the easier it is to avoid. The ‘ta-da’ moment isn't as good as it sounds!”

“Part of the challenge with agile that I've seen is that sometimes we set too many rules,” says Gandhi. “Every scenario is different. It's important to determine a length of time for each test cycle, but it should depend on the specific scenario. The question to ask is, how long will it take us to test this hypothesis to achieve a degree of certainty which is sufficient. It may be two weeks, it may be two months – it all depends on what you are testing.”

Jacobs is a firm advocate of ‘minimum viable product’ testing, which enables companies to test new ideas without going through a full development cycle. He says organisations should be aiming for shorter testing cycles, but the important thing is to ensure that cycles are long enough to provide meaningful results.

“I think the key thing there is ‘meaningful’,” Donlon offers. “I'm personally a big proponent of A/B testing where possible, and then the length of time required comes down to the ability to get to a statistically significant outcome. I think the key thing is then being pragmatic on your definition of statistical significance.”

“The key is to not be too prescriptive,” Gandhi explains. “It's impossible to define everything, otherwise you get stuck in the same bureaucratic red tape. Hence, it's something that has to be encouraged. The leadership has to encourage their mid-level management to take risks and if they believe in a hypothesis to test it (without always worrying about being disproven or failing).”

Unwrapping the red tape

All of these points are relatively easy to implement within a small organisation or team, but according to our panellists, they’re much harder to apply within a larger company.

“I think it's a really hard problem to solve in larger and especially older or more established organisations,” explains Donlon. “There are some semi-successful cases of businesses spinning off their ‘digital’ businesses into separate entities to enable agility without transforming the whole organisation, but eventually, you'll hit some process or governance wall.”

Jacob notes that areas like R&D and product development are generally the departments that most need to be agile. Focusing on making these divisions more agile, he says, can be a good way to introduce agility to a larger corporation more broadly. Donlon, however, argues that this approach comes with its own risks.

“The challenge comes in whether you can isolate those areas enough to avoid the agility being broken at some stage in the development cycle. Having lived this myself in a past life, you can be as agile as you want, but if you then hit the governance machine (usually towards the end of the process), it breaks it quite easily!”

“I totally agree,” Jacobs adds. “This is why it needs buy-in and embracing from the top, to create space for R&D to be properly agile. ‘Governance’ is often a code word for bureaucracy – and people wanting to get their sticky paws on things.”

In Donlon’s experience, financial cycles, processes and governance can make agility taxing in larger organisations. These cycles are often years if not decades old, he says, and breaking them can be difficult, requiring close collaboration between technology leaders and upper management.

“That's where it's been slow progress in older organisations where technology is only just making it to the board room table,” he says. “For a long time, I think agile was seen as an IT thing, but I definitely think that's changing. For teams to really embrace agility, I feel the organisation has to as well.”

One reason larger businesses struggle with agile is that it clashes directly with their need for predictability and their desire to plan as much as possible in advance. The nature of agile thinking means that, thanks to the rapid changes in strategy and approach that come with it, forward planning becomes a lot more difficult.

“With agile, you can have deadlines,” explains Jacobs, “but you can't predict what will be delivered on those deadlines. With a waterfall approach, the projects never work anyway - and they’re usually late and over budget. Agile gives you more control.”

“Exactly,” Gandhi adds, “it's a false prophecy. Everyone thinks waterfall gives more control, when in reality the lack of flexibility actually makes things not go according to plan. Yes, big companies need to set goals, targets, budgets et cetera. But how you get there needs to have an inherent amount of flexibility built-in.”

Planning for success

Another approach is to set up a mini-organisation within the business that can act in an autonomous, agile way – similar to how Centrica set up Hive as a self-contained company within British Gas. This approach is becoming more common as a way to both cut through red tape and minimise the wider company’s exposure to potential fallout from failure, Gandhi notes.

Fundamentally, getting the right approach to ensure sustainable adoption of agile is a tricky task, and the answer will look different for every business. For Studio Graphene, for example, the challenge has been balancing a desire for agile practises with clients’ demands for more traditional approaches.

“We are a services business and it's been a challenge because clients want ‘waterfall’ commitments,” Gandhi says. “We've essentially embedded a hybrid solution. We operate as an agile organisation internally, but we provide waterfall commercials to clients. This satisfies both needs. I still think we have a long way to go before we have embraced agile fully, but I would hope to think we have made more progress than many of our other counterparts and equally made more mistakes in our journey than others would have made too.”

For Moonpig, meanwhile, the process has involved giving teams more autonomy to choose their own processes, trusting that they will have enough maturity to do it properly.

“We've been on quite a journey at Moonpig,” Donlon explains. “As a very tech-focussed online-only business, it's probably been easier for us to improve our agility than maybe it is for some others. I think the learning along the way has been while the agile mindset needs to be throughout the whole organisation, agile methodologies and frameworks do not. We've had the entire company work in a squad-type model, which was maybe a step too far as it was enforcing a way of working on everyone. We took a step back and have found something that works well for all teams.”

“From a methodology point of view, we let our teams decide what works for them. Predominantly, our teams choose Kanban but we do use scrum as well. I think the key thing is being pragmatic about the use of frameworks and seeing them for what they are. As soon as you start being dogmatic, the system breaks and scrum can start to look a lot like waterfall!”

One thing that all our panellists agree upon is the fact that, whatever your approach, agile needs to be something that the whole business gets behind.

As Donlon says, “the minute you make it a ‘tech thing’ is the minute it’s doomed to fail."

If you're a senior IT decision-maker and you'd like to apply to be part of the IT Pro Panel, please email panel@itpro.co.uk.

The role of a project manager is to manage business projects and make sure every objective is met on time and within the agreed budget. There are a number of frameworks that help guide the planning and strategy around delivering such projects but PRINCE2 is one of the most flexible.

Fully compatible with agile delivery, PRINCE2 is also one of the more versatile frameworks out there too, with the ability to be applied to projects in a range of industries and tailored to each project’s individual criteria.

Seven key principles, sometimes referred to as ‘obligations’, define a PRINCE2 project and every one must be applied at all stages throughout. Without sticking to the seven core principles of PRINCE2, you don’t have a true PRINCE2 project.

Public sector projects run by the UK government back in the 1990s are thought to have initially championed the methodology first owned and created by the Central Computer and Telecommunications Agency (CCTA). Since its acquisition in 1996, the PRINCE2 framework has been owned and developed further by AXELOS which also created, and still maintains, the methodology’s official documentation.

According to AXELOS, PRINCE2 is hugely popular worldwide with more than 1 million certified practitioners across more than 150 countries. It’s available to learn in 18 different languages, can be used in a number of roles and once you pass the course, PRINCE2 practitioners receive a globally recognised certification to use their skills abroad, should they wish to take their skills into their company’s global offices.

PRINCE2 Principles

Copyright 2017 AXELOS Limited. All rights reserved.

PRINCE2 is classified as a ‘principles-driven’ project management methodology, built around the idea of seven principles interacting with seven themes and seven processes. These principles are designed to be universal in their application, able to be shaped to fit any project regardless of scope, application, language, or culture.

The seven principles are:

• Business justification: the weighing of risks against potential benefits
• Learning from experiences: building upon mistakes from previous projects
• Defined roles and responsibilities: a clear organisational hierarchy - team managers that handle the day-to-day tasks, who report to project managers, who in turn report to an overriding executive board
• Managed by stages: provide control points for the board to review progress to assess justification. Authority for managing the stages is delegated to the project manager
• Managing by exception: the idea that authority should be delegated using 'tolerances' - threshold ranges for cost, time, quality, scope and benefits of a project
• Focus on products: agreement on the amount and quality of the product being delivered should form the basis of any project and its reports
• Tailor to the environment: PRINCE2 is generic by default, and needs to be tailored depending on risk, complexity, project size and importance

These form the fundamental philosophies that project managers should consider when approaching a task. However, the overall framework relies on the interaction between these principles, and what are considered 'themes' and 'processes', both of which are also split into seven elements.

PRINCE2 Themes

PRINCE2 practitioners can view the seven core themes in a similar way to the seven principles, in that each must be considered at every stage of the project. The themes represent the considerations project managers must make when choosing the optimum strategy of the project to achieve its goals most efficiently.

In fact, they help coordinate any priorities identified in the planning process as well as make sure that the team members involved all have the same idea in mind, especially when it comes to keeping deadlines and staying within budget. Simply put, themes should be interpreted as the most basic commandments for the project’s best practice.

The main themes worth considering when planning out a project are issues such as:

• Costs and benefits, which is a classic business case pitch
• Organisation (the maintenance of a clear hierarchy of reporting)
• Product quality (a focus on the product itself)
• Plans (regular reporting between team members)
• Risk (a clearly-defined plan for risk management)
• Change (a comprehensible structure for facilitating amendments to the products)
• Progress (the continued communication about the progress of the project).

PRINCE2 Processes

While we compared the above-mentioned themes to commandments that should be strictly followed, the processes function as descriptors to the particular activities and stages which comprise a project. What's more, they should also name specific persons who will be responsible for making sure that they are handled correctly and in a timely manner. These include the following:

• Starting up a project: initial information is filtered to the project executive board, so they can filter out good ideas from the bad
• Directing a project: regular points within a project for the board to decide whether it has continued business justification
• Initiating a project: this sets out the firm foundations for the project; it's when the project plan and the detailed business case are written, as well as strategies for managing risks, communication, and quality
• Controlling a stage: performed regularly by the project manager, it involves dictating the various responsibilities, including allocating work to teams, managing risks, and ensuring the subsequent stages remain within tolerances
• Managing product delivery: during this stage, all the specialist products are designed, built and checked to ensure they conform to the requirements
• Managing a stage boundary: performed by the project manager at the end of each stage, this prepares the information required by the board to help them decide whether to continue
• Closing a project: involves gaining agreement from the users and operation teams that the products are satisfactory. It is also an opportunity for the project managers to report on the achievements of the project and any follow-up actions required

The Principles, Themes, and Processes combine to form a highly structured roadmap that a project is able to follow in order to meet its goals, as well as ensuring a high level of visibility and frequent opportunities to assess progress and make changes.

PRINCE2 Certification

Since its conception almost three decades ago, PRINCE2 has grown to be one of the most widely-used project management methodologies in the world, making it an established and acclaimed certification with a multitude of benefits for its holders. Its flexibility and universal application has made it an attractive tool for companies undergoing change, particularly through businesses transformation. If you're looking to go into project management, a PRINCE2 certification looks great on a CV. But depending on the type of project you're looking to manage, you might be better served by an agile methodology. Check out our article here for how to generate buy-in for an agile approach.

AXELOS operates its own exam-based certification programme available through its website. However, there are hundreds of accredited course providers throughout the UK, offering opportunities to learn either online or through weekend classes - just be sure to look out for the AXELOS accreditation badge.

First coined 15 years ago, DevOps refers to a collection of tools and practices in the software industry that teams adopt to make the software development cycle more efficient and shorter. It is the perfect marriage of software development (Dev) and IT operations (Ops) – and allows development and operations teams to work in unison. This methodology is based on the principles of shared ownership, rapid feedback and the continuous pursuit of workflow automation. 

The core DevOps principles, according to GitLab, are automating the software development life cycle, collaboration and communication, continuous improvement and minimization of waste, and focus on user needs with short feedback loops. "By adopting these principles, organizations can improve code quality, achieve a faster time to market, and engage in better application planning," according to the platform.

Since DevOps was conceptualized, businesses around the world have adopted this methodology in their droves as they migrate away from traditional structured waterfall development methods. The majority of organizations out there use DevOps in at least some way, depending on the source you find, with figures varying from 74% to 83% depending on the source and the research conducted. But the most important organization that matters is your own – and whether adopting DevOps could make a genuine difference to outcomes and productivity levels. 

The origins of DevOps

The traditional waterfall methods of development were falling away in the late 2000s to make way for agile methodologies, with speed over the quality of output more of a priority. Meanwhile, the rise of cloud computing saw platform as a service (PaaS) and infrastructure as a service (IaaS) products maturing into propositions that businesses could adopt. Then continuous integration (CI) began to emerge, according to Steve Mezak, the founder and CEO of Accelerance, a software development outsourcing company.

Concerns grew in the software and development industries that the siloed nature of work could not continue; the process of creating the software could no longer be segmented from deploying and maintaining the software when live. This, according to Atlassian's principal solutions engineer Ian Buchanan "was a fatal level of dysfunction in the industry." 

"They [IT operations and software development communities] railed against the traditional software development model, which called for those who write code to be organizationally and functionally apart from those who deploy and support that code," he wrote in a blog post. "Surely there’s a better way, they said."

The first conference named DevOps Days was held in Belgium in 2009, founded by project manager and agile practitioner Patrick Debois, and the first annual "State of DevOps" report was published at Puppet Labs – a pioneering company in the DevOps industry – by Alanna Brown, vice president of growth at Remote and formerly the senior director for marketing at Puppet.

How does DevOps work?

DevOps methodologies favor releasing small and iterative updates to software rather than massive updates every six to 12 months throughout the same period. This, in turn, feeds into a strategy of bug fixes and new features that allow customers to quickly respond to shifting trends or customer feedback. Should there be new market opportunities or changing demands and behaviors, adopting DevOps principles positions an organization to respond more productively and efficiently. 

Adopting DevOps significantly reduces the waiting time and ensures businesses can be more responsive. If one element needs changing following customer feedback, for example, it can be tweaked quickly and tested immediately – without needing to jump through hoops for approval. There are different rationales as to why DevOps is so popular and effective. Some say it's down to how teams are tasked to collaborate much better while others suggest it's the specific tools and workflows that streamline operations. However the best implementations tend to involve a combination of both. The flexibility of DevOps also makes it arguably better than other project management techniques for software development.

What are the benefits and drawbacks of DevOps?

There have been years of research into the benefits of DevOps for organizations, with the consensus falling on the fact that DevOps practitioners increase the frequency of releases and improve test automation practices. Research published in 2016 suggested as much. The key benefits include greater encouragement of collaboration between departments – which, in turn, improves communication and welfare – as well as continuous releases paving the way for more experimentation and rapid feedback collection.  

It's the frequency of releases that means you can deliver more features to users or customers in a timely manner, while less time is required for testing and quality assurance (QA) than in traditional siloed models. Moving at pace also encourages greater business agility, which lets you respond quickly to market shifts and be better prepared to make quick decisions about the state of the product more confidently. Iterating faster also means you can be ahead of the curve. Even failure, in this model, can be seen as a success. Moving faster means you're more likely to recover from a setback.

Challenges of adopting DevOps, meanwhile, according to the 2016 paper, include the communication structures that prohibit cross-departmental collaboration – and needing to address the culture shift in the organization. Not all industries are also suitable for DevOps, while ambiguity about what DevOps actually means may make adoption that much harder for teams going through the process. Because DevOps requires fairly massive structural changes, it can be difficult for legacy organizations lumbered with inefficiencies to achieve the transition. The concept of reorganizing at a single stroke can be a daunting prospect. The cultural shift can be difficult to manage as well. Workplace attitudes are often deeply ingrained, and it can be hard to convince previously distant teams to collaborate if they don't want to do so. DevOps can also be a tricky concept to sell to the board.

What are some common DevOps tools out there?

While there are no hard and fast rules for which software should be included in a DevOps toolchain, the same options tend to crop up in most DevOps environments. There's a huge variety out there, but these are some of the biggest names in the DevOps business. 

Docker: Docker is the technology that has been spearheading the uptake of containerization within the enterprise. The beauty of containers is that while they're similar to virtual machines, the fact that one OS image can host multiple containers makes it much lighter on resources than spinning up multiple VMs. The reason Docker, in particular, has proved so popular with the container crowd is that it's made it a whole lot easier and safer to create, manage and scale container-based applications, which can save IT departments huge amounts of time and money. Its flexible and scalable nature has also made it popular within the DevOps community.

Jenkins: Jenkins is an open-source tool used for continuous integration of code. It includes support for a huge number of plugins allowing it to work with a massive amount of systems and projects. This is one of the reasons it has become so invaluable to many DevOps toolchains, along with its ability to trigger builds based on a variety of factors. Like many of the tools used in this sector, it's heavily focused on automating as much of the software development process as possible.

Puppet: Used as a configuration management utility, Puppet is central to many DevOps teams. It's excellent for spinning up new servers and virtual machines and features its own language that is used to define the specific system resources needed. Another commonly used alternative to Puppet is Chef, and while they're widely considered to be neck-and-neck in terms of popularity and sophistication, many argue that Puppet is best suited to operational tasks where Chef favors uses that lean towards development.

When it comes to software development, there are two dominate methods: the traditional waterfall approach, and agile development. 

The latter is often referred to as simply 'agile', and is largely favored because it does exactly what its name suggests – it provides agility and a greater degree of flexibility for its users.

It focuses on flexible, iterative development, incorporating ideas like collaboration and cross-functional timing, and continuous assessment of the product being created. It's often faster and more accurate than the traditional waterfall method as it enables problems to be fixed quickly during development.

Products created under an agile structure generally get to market quicker, as each element can be tested and built simultaneously. It is particularly popular with organizations that use multiple teams to create software.

What is agile development?

The concept of agile development is underlined by a set of principles set out in 2001's Agile Manifesto. Its ultimate goal is satisfying customers through continuous delivery, but the philosophy is about taking restrictions away for developers.

With a waterfall approach, errors often force teams to go back to the start of the process as each element has to be completed before moving on. This causes delays and can be extremely frustrating, particularly if the problem is deep into the build. 

But with agile, changes can be made at any time in the process and usually fixed before combining all the elements together. Another benefit is that it's also easier to add or remove elements if unforeseen issues or planning tweaks arise during the production.

Agile principles

The Agile Manifesto, set out in 2001, outlines the key principles underpinning the concept:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan
• That is, while there is value in the items on the right, agile values the items on the left more.

In short, agile favors speed of delivery, testing, and continuous feedback. Its central principles support this. They are:

• Early delivery, creating shippable software in two-week 'sprints'
• Responding to changing requirements
• Measuring progress with working software as the metric
• Collaboration between business people and developers through face-to-face conversations
• Simplification - not making software overly complex
• Small, self-organising teams that regularly reflect on best practices

Why is it important?

When it comes to projects, everyone has at least one nightmare story. This could be exceeding budgets, missing deadlines, or even disappointing your most loyal customers - the building of services and products can be a huge obstacle course. 

Being able to fix issues on the go or make a quick change can be a real life-saver, especially in the IT sector. And in some cases, a project-saver too.

A key component to agile development is customer involvement as it allows customers to get an insight into the building process by giving them the opportunity to provide real-time feedback. 

Agile methods evolve and adapt to suit, in contrast to linear operations, where a simple mistake can mean having to throw away all the work and start from scratch. 

If a customer makes a request for something to be done in a different way halfway through the process, for example, a team utilizing an agile approach will be better equipped to deliver a solution to the request.

What types of agile methodologies exist?

Like many expressions from the tech world, ‘agile’ doesn’t fully define the approach. This is why organisations use a variety of terms to describe the development. 

‘Scrum” is one of the most popular terminologies, a word which has expanded the idea beyond the IT sector and into management teams. A ‘ScrumMaster’ manages and leads the workload of a ‘Scrum’, but provides the team with a collective responsibility of delivering the end goal.

Extreme Programming (XP) is another model which is gaining popularity, which focuses on the project’s technical side, instead of the business objectives. 

In contrast to month-long Scrum sprints, XP sets two weeks as the maximum amount of time a project should take. However, when it comes to changing goals mid-sprint, even that deadline is more flexible.

Agile advocates who don't subscribe to Scrum or XP will tend to favour Lean Software Development, a methodology coined in 2003 and based on seven principles. 

This generally favors a back-to-basics approach that cuts out anything but the necessary features to deliver a product as quickly as possible while meeting customers needs and allowing the business-aligned developers to take charge, rather than managers.

Agile v DevOps

Alongside agile, DevOps is another IT methodology that has grown in popularity within the enterprise community. 

However, there remains some confusion about the relationship between the two, with some organizations treating them as separate disciplines and some arguing that they're basically the same.

In reality, both camps are right; DevOps and agile share a lot of similarities, but they're also different in some key ways and are often suited to different tasks. 

Both methodologies focus on rapid iteration, regular user feedback and a high degree of flexibility, all in the service of delivering more functionality to end-users over a shorter period.

However, while agile can often focus predominantly on processes and workflows, DevOps generally puts more emphasis on tools and infrastructure. 

DevOps makes heavy use of continuous delivery, continuous integration and container tools, which allows for the quick revisions and small teams favored by agile methodology.

In many ways, DevOps can be thought of as an extension of agile. Often, a company will employ agile practices in software development, only to revert to non-agile methodologies as soon as the product is finished. DevOps simply extends the attitudes behind agile to cover the product's whole lifecycle.

Are there downsides to agile development?

Like any business practice, there are positives and negatives to consider here. While agile development remains highly popular among developers globally, there have been signs that agile is waning in popularity at larger enterprises. 

The ‘State of Agile’ survey from software development company Digital.ai found that agile is struggling to make an impact at larger enterprises. Smaller organizations continue to view agile as a powerful productivity and organizational framework with "obvious benefits".

However, medium-sized and larger companies have grown less satisfied with what agile can offer and are more likely to pick a software development strategy that uses a number of different frameworks.

There are a number of contributing factors to this waning appeal at larger enterprises, the study found. This method places significant pressure on developers, many of whom are dealing with surging levels of burnout.

Siloed teams and poor leadership also has a serious impact on this, the study noted, underlining the importance of having a robust plan in place.

One of the most frustrating feelings for a developer is to have worked hard on a particular piece of code, only to submit it and then have to dodge curses and thrown objects from your teammates as it breaks the build. Continuous integration (CI) was devised as a way to avoid this problem, allowing programmers to work on code in isolation without risking the integrity of the core build every time a new piece of code is introduced.

This is achieved by having a project's codebase stored in a single repository, which the whole development team submits to. When a developer wants to work on the code, a copy is created so they can work in isolation, with modifications uploaded to the repository as 'commits'.

Automated systems then check these commits at the point of upload, validating them for errors and ensuring that the modifications do not break the sanitised codebase stored in the repository. Any commits containing errors or found to create conflicts in the main repository will be rejected. If a commit passes, the main repository will be updated to reflect the changes.

Another tenant of CI is robust version control, meaning any repository should allow for versions to be rolled back to a stable state if unexpected conflicts arise. Perhaps the best example of this in action is the GitHub repository.

Developer teams operating under CI often update their repositories dozens, if not hundreds, of times each day. This means workflows generally produce much faster deployments with fewer errors. Often the alternative is to wait weeks to integrate code, during which time duplications can occur or a change in strategy.

Verify, find, fix

Given that the repository creates a copy as part of the development process, each developer can work in isolation without disrupting the rest of the team – a major advantage of this type of approach. If any errors are spotted during this time, the updated copy can simply be added back into the main repository.

However, CI relies on developers adding commits on a regular basis. Given each developer is working on a copy of the repository, failing to integrate commits frequently can result in their work falling drastically out of sync with other developers, compromising the integrity of the main repository.

Developers first share their own code into a central repository. From there the committed code triggers an automated quality control script, grabbing the latest code from the repository to build, test and validate the code sequence - if clean, it gets added to the main 'branch' or 'trunk' (the sum of all the contributions).

With the single repository being used by a whole team, developers can continue to work in isolation, although their contributions are constantly checked and tested. Everyone else in the team can see what is being added, taken out and changed, thanks to timestamps.

Because CI has a specific set of rules, such as: check-in frequently, don't check-in broken or untested code, don't check-in when the build is broken and don't go home until after checking in the system builds, the process takes care of itself, meaning teams need little management oversight.

The operation makes it so developers can not only add code at any time but also submit 'pull requests', grabbing bits of code to fix and tweak in isolation of the main branch.

Building an automated shared repository of this kind can be quite challenging and time-consuming, to begin with, but once established it can drastically improve the quality and efficiency of code development.

CI/CD

Continuous integration goes hand-in-hand with continuous delivery, to the point where the two are generally referred to as a pair under the acronym 'CI/CD'. Continuous delivery is similar to continuous integration in a number of ways; it's highly automated and is designed to make development workflows faster and safer.

While CI focuses on automating the creation and maintenance of a central code repository, CD is based on sharing that repository with a wider team and then deploying to production. In continuous delivery models, once code has been successfully committed, tested and validated as part of a build, that build is then automatically shared with the operations team who are responsible for deploying it to production.

This helps speed up the overall pace of updates and releases by removing the need for developers to manually share code with operations and creates a faster and more seamless pipeline between the two functions. For this reason, it's usually a key component of DevOps workflows, and it's rare to see an organisation that has adopted continuous integration without also using continuous delivery.

However, while it is possible to use CI without CD, because continuous delivery relies on the code being automatically tested and validated before it's shared, having continuous integration is prerequisite.

Confusingly, while 'CD' generally refers to continuous delivery, it can also mean continuous deployment, which forms the final phase of the CI/CD pipeline. Once code has been tested, validated and integrated into a code repository, this function automatically deploys it to the application's production environment. This requires no manual checks or approval by operations staff, so relies on confidence that any errors or issues will be picked up as part of the previous automated testing and validation stages.

While this can greatly increase the speed at which updates and new features can be delivered to users, it does involve an element of risk, as any problems which do slip the net of automated testing could end up in live, production versions of the application. However, the flip-side is that it also makes patches and fixes similarly quick to roll out.

CI/CD tools

The ever-growing popularity of the CI/CD methodology has contributed to the development of an array of tools capable of supporting it. A significant number of these have been originally developed as internal tools for enterprises and subsequently released to the community due to the high demand. The majority of these are readily available without any cost in the form of open source software.

While many tools are focused either on continuous integration or continuous delivery, there are some which were created as an attempt to strike a balance between the two. However, many of the most commonly-used tools are interoperable with one another, making it easier for users to construct the perfect toolchain using the ones they're most comfortable with.

One of the oldest and best-known CI/CD tools is Jenkins, which is known for its high level of automation, minimal maintenance, and a GUI for easier operation. Due to high popularity, there are many integrations and plugins created to even further increase its flexibility.

GitLab also provides CI/CD functionality within its offering and has the advantage of providing users with an entire development cycle within one application - from build to deployment. Despite being an open source software, GitLab is mostly targeted towards enterprises and offers increased support and functionality based on a flexible subscription model.

Travis CI was developed around a decade ago and, despite what its name might suggest, handles CI as well as CD functionality. However, it was constructed specifically around Github, meaning that if you are planning to use an alternative code repository, such as Bitbucket or the aforementioned GitLab, you might have to start searching for another tool.

Lastly, there’s CodeShip. This is another CI/CD tool from CloudBees, the main company responsible for maintaining Jenkins. However, unlike Jenkins, CodeShip focuses on simplicity and user-friendliness, facilitating pipeline construction with as little management and maintenance as possible

Polarising arguments have been at the epicentre of different industries since the dawn of time: Picasso or Matisse, Mozart or Beethoven, Mario or Sonic, Star Wars or Star Trek we could go on. The software development community is no different with people gravitating towards either the agile or waterfall approaches of app building, but can there be only one winner?

The two approaches differ substantially in their methods and project goals and these are reflected in their respective ages. Waterfall is largely considered to be the more legacy approach, opting for a linear approach that focuses on creating an iron-clad plan to map out the project in stages, giving developers a clear view of a project's roadmap and how they should progress towards completing tasks in the correct order.

Agile, on the other hand, takes the task of development and focuses on getting the project done quickly with an emphasis on nailing the core goals of the project in no particular order. It's the younger sibling of waterfall and has seen a growing adoption amongst teams throughout the industry. It typically sees teams developing applications faster and then upgrading them in quick stages.

Although youth seems to be overtaking the aged, developers in both camps will fervently argue their corner and for good reason. Despite agile's uptick, waterfall still serves a strong purpose for teams with specific needs that favour one specific development methodology. So, read on to discover why the approaches to software development are so divisive.

Agile vs waterfall: Working methods

Agile

Agile development is concerned with getting the software developed quickly and to a recognisable state to where the customer wants it to be. The customer tests the software and then it is refined according to feedback, in a later development stage.

At the beginning of the development cycle, the project is outlined according to customer needs (features), known as user stories. Developers then develop "iterations" which are built to address these needs, in order of importance.

Teams work in sprints to finish each iteration within a set time span, usually measured in weeks. Each sprint aims is to create working software the user can trial, then advocate changes so it more suits their needs.

But the end of the project doesn't mean all features are built. Agile projects can run out of time, meaning developers must either agree with the customer to do without some features that remain incomplete or extend the project (and pay more money) to deliver them.

Waterfall

A waterfall development project follows a traditional, linear approach. The first stage is scoping out users' requirements, which can be an extensive process. Because developers won't produce working software for customers to try out mid-project, they want to understand up front exactly what users want the software to look like.

Then, the developers will design the software, giving themselves a detailed template to work from. By working from a plan, the idea is that the actual development process, which comes next, will be smoother and hit fewer obstacles.

Once they've built the system, developers enter the testing stage - this is where they make sure the software actually works. This sees developers debug the system, sometimes involving users who try it out. After that, the developers go into maintenance mode, installing, supporting and maintaining the software on behalf of the customer.

Agile vs waterfall: Pros and cons

Agile pros

The greatest benefit to Agile development is that it involves the customer or end user at all stages of development, and seeks their views to feedback into the project.

This, as a result, will lead to an application that is far more likely to fit the brief, or at the very least the initial picture, the customer provided. Developing under an agile methodology also means if the customer changes their mind about an element of the software, or requests any last-minute additions, it doesn't present too much of a challenge to incorporate these. Developing under waterfall, meanwhile, means this could be extremely tricky.

This approach also delivers software more quickly to customers, making it ideal for projects in which speed is of the essence.

Another advantage in agile is the concept of continuous improvement, in which lessons learned in one iteration inform work on the next iteration.

Agile cons

However, agile projects are intensive for customers. Their involvement is required throughout, so the customer must be able to spare stakeholders, who must be enthusiastic and engaged in the project. Agile projects can fall apart if the customer isn't interested in working closely with developers.

Agile's other disadvantage is that, because of the strict sprint deadlines, sometimes the overall project can end incomplete. Customers must either just accept the reduced scope of the project, or pay more to get the developers to finish everything they had planned to complete.

Waterfall pros

By focusing on quality over speed, waterfall is well-suited to less urgent projects with customers who know exactly what they want their software to do. The extensive testing period can result in fewer bugs when the project is complete.

Waterfall's strict linear process means a project is likely to be finished on time and within budget, and with regular milestones, it is easier for both developers and customers to track its progress.

Waterfall cons

In waterfall development, once the course is set there is no deviating from it. Unlike agile, waterfall doesn't involve customers every step of the way but scopes out their requirements only at the start of a project.

This means the waterfall approach works best when customers have clear ideas about what their software needs to do. If users have only a vague idea, or the project takes a long time to deliver, the finished software often won't need users' requirements, which may change over the course of the project.

Similarly, once a waterfall project hits the testing phase, it's very hard to make changes to the software. Customers will have to pay more money and wait longer if they want to make changes this late.

To avoid this, waterfall developers sometimes build in customer feedback points between the stages outlined above, to adjust the project as they go.

Image: Shutterstock

BMW has been locked in a process of reshaping its IT strategy for just over a year following its decision to abandon the Waterfall model of software development, in favour of becoming '100% agile'.

Agile is fast-becoming the most popular model of software development, revolutionising the way companies and organisations build applications. Agile differs from the traditional waterfall method in that products are finished and pushed out as quickly as possible, supported by frequent minor changes rather than large dispersed patches.

The car giant says that it has learned above all else there's no silver bullet to tackling this transition, and that each organisation must find their own way based on their existing infrastructure and the configuration of their teams.

Changes in the wider market and increasing demand from customers for new features to be delivered more frequently forced the company to change its approach to IT, according to BMW's vice president of IT delivery Ralf Waltram.

As Waltram puts it: "We want to get flexibility in delivering software, trying new things out fast, to learn very fast, and gain speed".

You can go your own way

There were a plethora of models from which to choose, as well as books and YouTube tutorials at his disposal. He even turned to the likes of Microsoft and Spotify to attempt to mirror their own approaches to agile. However, he soon realised that BMW couldn't rely on a tried-and-tested method; instead, his company needed a bespoke methodology of its own.

"The success of our agile transformation is highly dependent on the right adoption of the agile models," says Waltram. "There are so many different models out there. Yes, you can buy a book, or you hire a good advisor from outside, but we believe that you need to find your own way on how to become Agile.

"Your team is different than my team, and your legacy infrastructure is different, the technology you're using is different, so really the key takeaway from my side is find your own way and learn by yourself."

Breaking the waterfall deadlock

The company's own transformation involved changing the way it approached four key areas - process, structure, technology and culture.

Process, for instance, involved researching the best way to approach adopting agile development, while attempts to overhauling the development structure were driven by a need to overcome the inherent inefficiencies of BMW's agile and non-agile teams working in tandem.

The incompatibility of the two team types frequently resulted in delays to ongoing projects, which BMW have since tried to rectify, in-part, by implementing a full DevOps organisation structure from last year, as part of its ongoing process to become fully agile.

On how technology played a part in BMW's transition, Waltram said the key was investing in the right architecture and innovation, including cloud services. He said that BMW was currently focused on delivering an Agile Toolchain, a coding workplace, a microservice-based architecture, and having everything cloud-enabled.

Building the toolchain

The Agile Toolchain, BMW's backbone for software development, involves a defined process - organised somewhat like a flowchart - which the company believes its employees will be able to easily follow.

It begins with planning, before shifting focus to the design, build, test, deploy, release, operate and monitor stages - into which the company hopes all projects and users will eventually work under.

Last June, when BMW began transforming its approach to software development, 8,000 users were on its Toolchain. Today, some 20,000 are following the model, which is also supporting more than 3,000 sub-projects, and 450,000 items.

Finally, promoting qualities such as transparency, openness and trust, were essential in altering the company's culture to embrace Agile development. This shift in attitude took place at the very highest level of the company, with execs affording a healthier degree of freedom and autonomy to its teams, as well as bringing all of BMW's coding in-house.

The proof is in the pudding

This adoption has led to improvements in the way BMW handles the development of products, and has helped significantly reduce the time to market - perhaps best demonstrated in the company's new approach to interior design.

Historically, Waltram's small in-house R&D team would work using physical mockups of car interiors only twice a year. Under the new approach, his team is now using software to visualise mockups in a 3D game engine, allowing for new integrations every week.

Waltram explains that where it would normally take four months to come up with initial concept designs, tests under an agile approach were producing viable complete products within the same time frame.

"[In a Waterfall world] it's not possible in four months - definitely not. And this is so exciting, if you can see the way the new process is working, and we have our first results here," he says.

"But to be honest to you, for sure... you will also encounter in agile projects or products that you have problems; you have to tackle problems. The good thing is you find them more faster, in a more iterative way."

Scaling Mount Everest

Everest, BMW's SAP system that handles the logistics and production processes for prototype hardware, was a project blighted with inefficiencies and bugs. In bringing several teams together, he managed to resolve a series of the biggest problems in three months - quickening Everest's release cycle from twice a year to three times a year, plus sprint releases every three weeks, while recording a 72% reduction in bugs and incidents.

But its IT chief was keen to stress, despite the progress made, the company still had a great distance to cover and that its journey would never end.

"It's a journey. We're not done here; we're not even getting started. We did pilots, we are currently doing the ramp-up, we are collecting our Agile working model together, we will go into scaling into all of our products, and it will never be over, this transformation."

Image: Shutterstock

Whether your business runs formal projects such as product launches or updates, software rollouts as part of a digital transformation strategy, or you just need to keep track of everyday resource and time-limited tasks, project management tools should be a vital part of your business's IT set up.

They vary in feature set, from basic task tracking to fully-fledged resource, time and cost management platforms, with collaboration, accountability and every single detail your teams need access to built in.

Like their features, project management tools can vary significantly in price, with those offering basic functionality often free or with free trials, allowing you to try for a limited time before you buy. But with so many to choose from, how can you find the tools you need to power your business and make projects run as smoothly as possible?

We've rounded up the top project management tools to help you choose one right for your business before you commit.

LiquidPlanner

LiquidPlanner is a project management tool that's built upon schedules to ensure projects run smoothly. It allows businesses to plan, track, schedule, collaborate and manage resources across multiple projects, so everyone involved knows what's expected and when.

LiquidPlanner also includes key insights on resource availability alongside other data-driven observations, allowing for in-depth project management. There are three plans available, suited to different sizes of organisation and each is billed annually. Each supports a different number of active tasks and users, with the small business/standard plan offering the fewest (250 tasks and five users).

Alongside core project management and scheduling tools, LiquidPlanner also offers free storage, plus integrations with Box, Dropbox, and Google Drive, allowing for collaboration on files and folders.

Platforms: Android, iOS, web

Zoho Project Management

Zoho Project Management is definitely one of the best project management tools out there, with features you'd expect to experience on a subscription-based platform. However, one of the big caveats is that although you can have unlimited users in your team, you'll have to be careful about what you share, because the free version only includes 10MB storage and upgrading to unlimited will set you back $25/month.

Platforms: Android, iOS, web

Trello

With its drag and drop card-based interface, Trello is super-easy to use. It also boasts a huge range of integrations with third-party applications such as Google Drive, so you can't really go wrong. Everything is customisable, allowing you to add columns according to the individual project, and colour-coding task cards however you like. You can also easily assign tasks by tagging users on a card and setting deadlines. The more people you invite to your board, the more credits you get, which will allow you to upgrade to gold with higher storage limits for free.

Platforms: Web (mobile and desktop versions available via third parties)

Asana

First developed by Facebook engineers for internal use, this cloud-based tool is designed to help users manage the progress of various projects. Essentially an alternative to Trello, it allows users to create tasks, assign users to them, set due dates and add comments.

Its strength lies in its minimalist approach to project management, offering a more streamlined experience compared to other suites, while still offering similar levels of functionality. Asana currently has integrations with Gmail and Google Drive, Dropbox and Slack, as well as a host of other third-party applications.

Asana is available as a free tool for smaller teams, with support for up to 15 users, although this only grants partial access to its range of features.

A full-fat version is available starting at $9.99 per month, which opens up all of its features without a restriction on the user count. For more advanced tools, there's also an enterprise plan that adds branding options and enhanced security and data protection services.

Platforms: Android, iOS, web

Wrike

If you only have a small team, Wrike offers its full version for free, which makes it a top choice for businesses just starting out. Like Trello, it integrates with a whole host of third-party apps such as Dropbox, so you can easily link your files with project documents. Collaboration is at the centre of this free project management tool, including a private messaging function, so you can scrap to-and-fro emails and just use Wrike to communicate for everything project-related.

Platforms: Android, iOS, Web

Freedcamp

Freedcamp takes a more traditional approach to project management, helping keep all the admin in one place, as well as tracking project progress. The free version is pretty basic, but you can add modules on to help cover your entire business process, including CRM, meaning Freedcamp can scale as your business grows.

Like Trello, it uses a Kanban board-style UI, allowing you to drag and drop tasks as they progress through the project. Task lists are present to keep your teams organised, and it can even create Gantt charts to help you present your progress to stakeholders.

Platforms: iOS, Web

Bitrix24

Bitrix24 is more than just a project management tool - it's a collaboration platform for your business. In each project group, you'll find all of the files related to that specific project, while all tasks can be assigned or delegated to colleagues at any step of the process, with a Kanban board view option if you prefer.

You can track how much time an employee has spent on each task, with task counters available to keep track. Projects can also be shared online, so clients and those outside the organisation can track progress and contribute too. Bitrix is free for up to 12 users with a storage limit of 5GB, but that's enough for most small businesses.

Platforms: Android, iOS, Web

Producteev

Producteev, as the name suggests, seeks to make teams more collaborative and therefore productive, helping businesses make decisions on the spot to guarantee continuity.

You can set priorities for tasks, colour code them and add custom descriptions according to where they fit in your overall plan. You can also include deadlines to ensure they're completed on time. Each task can comprise subtasks too, with a tick box available for each, so teams can keep track of where they are on a wider task that may comprise several parts.

Every task can be transformed into an email to quickly assign it to someone, and of course, you can upload files via Box, Dropbox or from your device.

Platforms: Android, iOS, Web

Main image credit: Trello

Earlier this month Microsoft bought GitHub for a mighty $7.5 billion. Just ahead of the acquisition, rumours were floating around that Google was the one interested in snapping up GitHub, even though an offer never materialised.

Now though, it appears that Google was indeed eyeing up GitHub and one company executive is slightly miffed that the search giant missed out. As reported by Bloomberg, speaking at a Fortune Magazine event, Google's head of cloud Diane Greene admitted that "I wouldn't have minded buying them, but it's okay".

While not the salacious grovelling details you'd probably have liked to hear from a Google executive over missing out on the deal, any acknowledgement of it is interesting enough. Greene also expressed concerns about Microsoft taking control of GitHub, echoing those of many in the development community. "I really hope Microsoft can keep them totally neutral", she said, though Microsoft CEO Satya Nadella has stressed his commitment to "developer freedom, openness and innovation".

Developers reacted to the news that Microsoft was to own the site by pronouncing the site's death, with many upping sticks for rivals like GitLab and BitBucket.

But many also stayed, and under Nadella Microsoft has embraced open source software like longtime operating system rival Linux and the CEO has vowed that GitHub will"retain its developer-first ethos".

It's not the first time Google and Microsoft have reportedly been interested in the same target. Before Microsoft bought LinkedIn back in 2016, four suitors were involved in a furious bidding war for the social network platform. One of those was Microsoft, but another was Google, according to Re/code, which named another as Facebook.

Picture: The Googleplex/Credit: Trey Ratcliff on Flickr

Agile methodologies have become a popular approach to building and running digital services for most organisations looking to undergo a digital transformation.

Even for the UK government this is now a mandatory process for improving digital services. In 2007 it was compelled to change from DirectGov to Government Digital Services (GDS) after a damning report from UK Digital Champion Martha Lane Fox.

Her report called for a "revolution" in the way Directgov dealt with the public online, adding that the government "should take advantage of the more open, agile and cheaper digital technologies to deliver simpler and more effective digital services to users."

Since the report was published, the government has embraced agile methodologies by creating a digital service team who have since set about modernising online services throughout central and local government.

Essex County Council was one of those authorities keen to revamp its online services, having found their old website ill-suited to the demands of a increasingly connected public. Through the use of agile methodologies, the council set about creating a new blogging platform in just two weeks, a tool the council could use to communicate information more effectively.

The council turned to public sector digital specialists Dxw for help on the project - a company that has previously worked alongside the Cabinet Office on its new assurance tool for approving government spending, and the Small Business Commissioner service, a online advice platform provided by the Department for Business, Energy and Industrial Strategy.

Agile working

The first post on the new blogging platform, written by ECC's service delivery team manager Nicholas Ward, paid tribute to the new method of service delivery.

"It was exhilarating making something so tangible in just two weeks. That said, trying to keep pace with a truly digital organisation like Dxw really highlighted how slow some of our internal processes and systems are and how we tend to accept this as the norm," he wrote.

Two-thirds of CEOs have digital transformation at the centre of their corporate strategies, but many IT organisations struggle with managing the complexities involved. Learn what to do about it in this whitepaper.

Download now

An agile project is where research, planning, design, development and testing all happen in parallel. Feedback from users comes regularly, so a thick skin and an adaptable mindset are required. It allows for continuous improvements to be made throughout the process with the new system only going live when it has demonstrated the service works and meets the user's needs.

"This kind of sprint-based approach uses less documentation and provides more value, which are the principles of what we are doing," Ward tells IT Pro. "We understand the problem that we are trying to solve and we do our user research to make sure what we are doing is right."

"It involves early prototyping and testing and continuing testing as we go through the process of building. It's very focused on you as the user of the website achieving your goal."

Borrow, don't build

The guiding principle for Ward was "not to reinvent the wheel" and use a system that already exists and has proven to be successful. The team decided to use the incredibly popular platform WordPress due to its ease of use, although the design and coding was based on a publicly available template already deployed on GOV.UK blogs, sites that use the same style and functionality needed by ECC.

It was this process of recycling tools that have already been tried and tested elsewhere in government that helped cut down on any extra programming that would have otherwise been needed.

"As well as the obvious speed advantages, using a codebase borrowed from centralgovernment illustrates a broader principle: borrow, don't build," says Ward. "Our problems are not unique, there are people all across central and local government facing very similar challenges."

User stories

Dxw and Ward worked with the Essex team to develop a series of user stories, used to explain the functionality of a feature from an end-user perspective.

These stories describe the type of user, including what they need from a specific application feature, to help create a simplified, easy to implement solution to the problem. For example: "As a user, I need to be able to navigate between different blog posts so that I can read other posts that I'm interested in".

Dxw deployed a copy of blog.gov.uk and the two teams began to work on the user stories, allof which went onto a Trello board. Each story worked on by the Dxw team had specifically defined acceptance criteria, so the whole team knew at what stage each task was to be completed by.

As the stories moved from the product backlog through each of the columns on the board, it was the responsibility of both the deadicated Dxw delivery manager and the ECC product owner to provide oversight and approve stories as they were developed.

Working in the open

The delivery service team looked at central government's recent technology led work and tried to implement similar collaborative practices into Essex's digital services.

"We learned lessons from central government and its ways of working that we have found tremendously useful and we wanted to do the same," says Ward. "It's a tool to help us to reach out and build relationships with other local authorities who are doing similar work. And, potentially, we could form partnerships that help us and help them."

Local and central governments now share tools, templates and patterns as part of this new agile method and ECC has lots to gain from making this part of its own culture. The service design team is now piloting the blog and will roll it out to other teams while also documenting the process as part of its commitment to working in the open.

The idea is that sharing the work as you go and taking on the feedback you can quickly get a sense of whether the project is going in the right direction. It's the opposite of building something in isolation, where there is no feedback until its final reveal as a fully formed product.

"Everyone is at a different stage, but I think it is a growing community," says Ward. "We share and talk with a number of different partners in other local authorities, some of them come out of comms teams, some from IT teams and some from digital teams.

"It's a methodology that crosses over quite a few different departments in traditional local authorities. Everyone has a part to play."

Setting the standard

ECC's blog platform project demonstrated what can be achieved in a short space of time with a well organised effort and a culture of openness and collaboration.

"We are at the start of the journey," Ward says. "Things like the blog are a good example of something we can do that is achievable and shows the principles of how we want to work and this is something that we can just get out and deliver very quickly."

Image: Shutterstock

Like many enterprise technology companies, Dell has put much of its focus over the last few years on the concept of digital transformation, whereby companies use techniques like data analysis, DevOps methodologies and agile practices to become more productive and innovative.

Dell has made a big deal out of the importance of digital transformation, calling it the future of business, and essentially telling attendees to this year's Dell Technologies World conference in Las Vegas that they either need to digitally transform or face extinction.

It's tempting to dismiss all of this hype as little more than a shallow marketing ploy designed to move more products, but Dell is practising what it preaches, too. It may be a tech company, but as a 34-year-old international enterprise, it faces many of the same challenges that its customers are, and it's using digital transformation and DevOps to solve them.

One of the company's oft-repeated mantras is the maxim 'people, process, technology' - the three areas businesses need to address if they want to enact change within their organisations.

For Dell, this idea is the cornerstone of successful business transformation, and according to Bask Iyer, CIO of Dell Technologies and VMware, it's something that the company keeps coming back to. Iyer is responsible for Dell's internal IT, so it's his responsibility to ensure that all the idealistic notions about agility and digital transformation that Dell's leadership team espouse actually function in practice.

"Even in Dell, we still have to go back and say ' people, process, technology' - and culture. And I had to sell it to my executive teams and so on," Iyer tells IT Pro. "Of course, they get it very quickly because they're tech folks, but you can't take it for granted. I had to present it at the quarterly business reviews, and all these folks questioned me, challenged me - like they would, like normal businesspeople - and then supported me after that."

People

DevOps is one of the methodologies and techniques most commonly employed as part of a digital transformation strategy, so it's no surprise that Dell is also making full use of it within its own organisation. However, Iyer told IT Pro that the company had to make some changes to how it thinks and talks about DevOps in order to fully implement it.

"If you just say 'agile, DevOps, CI/CD'," says Iyer. "It's very geeky and confusing for the business. These are not business terms. So, similar to what GE did with SixSigma and what Toyota did with the Toyota Way, we've come up with Dell Digital. That's the business language."

Iyer believes it's critical to involve the business in digital transformation projects, or risk those projects failing.

"You need end-users to sit with you in your war rooms and do pair programming with you. Just IT being efficient isn't going to create digital transformation, so we had to transform the methodology within the company. Agile for us is not just an IT practice - users have to want to come there."

Bringing line-of-business employees into the IT conversation is key, then, but Iyer also points out that to truly benefit from DevOps, these staff must stop assuming that they can simply punt technical projects over to IT and wash their hands of them.

Process

For this reason, Dell's DevOps strategy is largely prescriptive. According to Iyer, in order to facilitate that kind of close collaboration in an organisation with around 145,000 employees, you need to lay down some defined guidelines and ground rules. This, he says, is a large part of the 'process' element of the 'people, process, technology' approach.

Digital transformation and DevOps are most successful when they're driven or supported by executives and business leaders at the top levels of an organisation. Executive buy-in gives staff the freedom to experiment with these methodologies at their own pace, rather than feeling pressured to deliver results and justify the project as quickly as possible.

Thankfully, Iyer says, this isn't a problem at Dell. He claims founder and CEO Michael Dell is keen to embrace new technologies and ways of working, and has absolutely no problem engaging with the company's agile processes personally.

Pair-programming with Michael Dell

In fact, Michael Dell himself ended up collaborating directly with one of the company's rank-and-file engineers in order to get a project done. Iyer relays a story around one Thanksgiving weekend, when Dell texted him to ask if his team could put together a map showing the company's global orders.

After a very brief discussion, Iyer gave the project to some of Dell's engineers in India and Austin. "These guys start developing it," he says. "No requirements, no specs, no funding discussion, nothing - they just start coding."

The following Monday, Iyer received an email with a link to a functional, production version of the tool that Michael Dell had asked for, which he duly forwarded to the CEO. Then, Iyer said, his boss did something unusual.

"He just went to the actual programmer - directly, he didn't copy me [into the email] - and said 'what is this, and can you make this change'. The guy did it, and he freaks out - he's getting an email from Michael Dell," Iyer laughs.

The programmer, he explains, was messaging his colleagues, panicking over what he should say to Dell and continually copying Iyer into his responses to him. "Finally, I tell him 'dude, please don't copy me. You can always update me in a one-on-one. You're doing pair programming with Michael; that's what's happening.'"

Understandably, this story quickly spread throughout Dell, but Iyer believes it illustrates a broader point about the company's culture, and how the business can contribute to IT projects. "If he has time to do that, you [someone from the business side] have time to get in the dojo and explain your requirements to me," he states.

Picture of Michael Dell, credit: Oracle PR/Hartmann Studios (under Creative Commons license)

Digital transformation is now a vital part of most organisations’ business strategy. For many, embracing new technology is crucial to stay competitive and meet the challenges of an ever more competitive and complex world economy. Businesses must be agile and flexible, with an ability to scale applications up or down in line with customer demands.

Many digital transformation strategies for organisations now include the utilisation of cloud and new application architectures to boost profitability, while minimising vulnerabilities and reducing the likelihood of having to do extensive and costly software updates.

Applications have a vital part to play in this strategy. As organisations embrace the cloud, they are adding to the complexity of their IT infrastructure, which can prove challenging in terms of managing and maintaining visibility of multiple workloads across a diverse system of platforms and applications.

Microservice applications

Microservice applications are one way that these challenges can be met. Microservice applications are agile, configured to enable new features to be added without disrupting applications in production, and are designed to automatically spin up new instances in response to increasing user demand.

This drastically changes the way that applications are deployed and managed, bringing a new level of complexity to network infrastructures as traditional applications are not going away. Applications will continue to be designed for both microservices and traditional 3-tier implementations, and organisations will need to maintain traditional IT practices while developing agile practices for newer DevOps microservice applications.

To bridge the gap between this new microservice application delivery and traditional application environments, some businesses are adopting hybrid application delivery infrastructures.

But supporting a hybrid application delivery infrastructure places unique demands on the business' Application Delivery Controllers (ADCs) due to the diverse qualities of both models.

WIth microservices application delivery, ADCs are placed in a container and are deployed per microservice, whereas in legacy applications, they are in front of the application servers. There are also variations in the quantity of ADCs deployed; thousands may be required to support traffic with microservices applications, compared to just a few for traditional 3-tier applications.

These differing structural characteristics create a large management challenge that must be overcome to support both traditional and microservices applications in parallel. There are three strategies that can help businesses at different stages of digital transformation overcome some of these challenges:

Conventional Hardware - Most ADCs carry a fixed licensed capacity. Traditionally, when businesses needed to increase capacity, they either bought new appliances or upgraded the appliances they already owned. This buy it when you need it' strategy makes short-term sense for companies that prefer to manage hardware on-premise, and may not be ready to move to cloud. In the long run however, as business demands increase, recurring purchases can be expensive and disruptive, and can eventually lead to appliance sprawl' which is costly to support.

Consolidation with multi-tenancy - Many organisations are replacing multiple point solution appliances with a scale in' approach. Rather than continue to buy more and more devices, they choose to go in the opposite direction, consolidating existing load balancers, VPNs and firewalls onto a multi-tenant platform that can scale up as needed over time without the set-up and deployment requirements of complex upgrades.

Pooled capacity licensing - As applications transition to the cloud and the data centre evolves towards a software-defined model, the requirements for ADCs are rapidly evolving. A software-first ADC decouples software from hardware, freeing IT departments to allocate instance capacity flexibly across the network, in hardware or software either on-premise or in the cloud.

Low-code

Recently other methods of application delivery have gained market traction. Software has been developed to enhance existing DevOps processes, aligning the departments more closer than ever before, with an approach named low-code standing out from the pack.

Low-code is a visual approach to application delivery that’s specifically designed to ease digital transformation. It’s especially useful to SMBs, who typically have tighter budgets and fewer developers in-house.

Low-code allows users full autonomy over the application development process. The concept relies on constant visibility. As the interface is usable even to those without coding experience (reducing the amount of hand-coding necessary to produce applications), those who conjure the initial concept can then build the application, ensuring the finished product closely resembles the idea.

In this way, the development process is democratised. Employees outside typical IT teams are empowered to have continuous input into the application delivery process, encouraging collaboration and feeding into agile-working initiatives.

The real value in low-code is that it increases an organisation’s agility. It’s streamlined method enables ideas to come to market quickly, allowing even SMBs with fewer resources to compete with large enterprises with handfuls of developers at their disposal. Organisations can do more with less.

The business technology landscape is changing so fast that IT will have been completely eradicated in less than 15 years, CA's chief technology officer has claimed.

Speaking at CA World 2017, CTO Otto Berkes told attendees that the accelerating pace of change within the tech sector is fundamentally changing the nature of how businesses consume technology and tools.

"I believe that by 2030, what we've come to know as 'IT' today will be gone," he claimed, "or will be virtually unrecognizable."

Businesses are currently in the midst of a struggle between entrenched ideals and culture, and rapidly-evolving technology and workflows, he said. Increasing end-user engagement and the democratisation of access to tools has flipped the process of IT decision-making from a prescriptive, management-led approach to a 'bottoms-up' model driven by workers themselves.

Berkes also pointed out that businesses need to be willing to take chances on offering new products and services, saying that today's fast-moving markets necessitate a certain level of risk-taking.

"The new world won't wait for a complete set of requirements to start a long waterfall execution process. Today, you need to start with what you believe about the future, and then iterate with your customers to continuously shape and re-shape what you deliver."

However, he also warned that businesses should try and avoid getting too far ahead of themselves. While he advised attendees to build products to meet future demand, that demand can't be too far in the future.

"This is where a lot of companies get into trouble," Berkes said. "They either take a leap of faith and tackle something that nobody even needs yet, or they build something for today that becomes almost immediately obsolete, or is simply an attempt to catch up to the competition."

"Start with what you know and what you believe will happen, and iterate quickly. Making this fundamental change is hard for everybody, but we can make it easier for ourselves."

CA Technologies has today announced more than 20 new and updated products across different business verticals, expanding the company's portfolio in areas like DevOps, automation and testing.

Speaking at CA World 2017, CA president and chief product officer Ayman Sayed took to the stage to tell conference attendees about the new capabilities.

"I am particularly happy and pleased, and very excited, about our ability to make the products easier, more intuitive to use, and simpler. This speeds up time-to-value, the time it takes you to get a POC going," he told IT Pro. "We've invested heavily in try-and-buy, in quick time-to-value, and if you look at the number of enhancements we've offered, several of them are in this category."

The new products and features build on the strategic acquisitions made by the company over the last year, including updates to products from Veracode, Blazemeter and Automic.

DevOps has been a major focus of CA's most recent slate of updates, with three new and four updated products. This includes new testing and monitoring tools like API testing for Blazemeter, a new SaaS-based customer experience and application performance management console and new tools for testing code security as part of an IDE for building a DevSecOps workflow.

The company also showed off its award-winning new 'Microgateway' product, allowing businesses to rapidly deploy and manage microservices to modernise existing legacy systems and add additional functionalities.

Machine learning and data-driven analytics are a common thread amongst CA's new products. The company is betting big on the value of these technologies to power what it calls the 'modern software factory', a highly agile, software-based business model that can rapidly adapt to changing market conditions.

"Every business can be a modern software factory and achieve better business outcomes faster with the right intelligence," Sayed said. "Today our portfolio is comprised of the tools and technologies that inject analytics and machine learning into our customers' existing technology investments to compete and win in today's fast-moving marketplace."

CA Technologies CEO Mike Gregoire today kicked off the company's annual conference in Las Vegas by urging businesses to empower their employees to be more creative and to take greater risks.

Gregoire talked about the importance of being able to pivot your business model, should the need arise. He pointed to the example of Leadville, Colorado - a dying mining town that was revitalised when it rebranded itself as a destination for extreme sports enthusiasts.

Organisations that don't have the inclination or capacity to adapt to change are at risk of being left behind be more agile companies, he said, citing the examples of Netscape and Eudora as companies who didn't move fast enough, and were left behind as a result.

"It's not good enough to be good, or even great," Gregoire warned. "If I know one thing about the tech industry, it's this: There is no such thing as too big to fail."

"With digital business change being a top CIO priority, existing IT infrastructure and applications, as well as business processes, must accommodate the 'new' quickly and rapidly," agreed analyst firm Gartner. "If legacy systems fail to support, integrate or enable these new digital initiatives, digital business success will be impaired, or even fail."

"Playing it safe can actually be the most risky decision you can take," he said. "I believe change in your companies will be driven by you, the technologists - the ones who aren't afraid to take the risk."

The best way for companies to fuel this change is to unleash their employees' creativity, according to Gregoire. Gregoire explained how Eurosport, Netflix and Citi are using CA's tools to pivot in various ways, exploring interesting new use-cases and applications. Highlighting some of CA's more prominent and impressive customers has become a running theme at CA World, and it's something that Gregoire took evident pleasure in.

"Software is the creative medium of our age," he said. "In fact, writing software and digitising processes is legally considered a form of art."

In addition to public companies, Gregoire also took the opportunity to show off CA's government customers. The Singapore government, Montreal's transit authority and the FBI are all using CA's tools in order to become more agile and efficient.

Gregoire's keynote reinforced some of the biggest themes of CA's recent strategy - a focus on agility, transformation and rapid change. The software revolution is going to let businesses explore new opportunities and keep themselves ahead of the curve, Gregoire said; but they have to be ready to take some chances along the way.

Banks and financial services companies are leaving themselves at risk of being hacked thanks to poorly-written code, according to new research.

Software analysis firm CAST reviewed 278 million lines of code from more than 1,380 applications developer using Java EE and .NET, and discovered more than 1.3 million vulnerabilities caused by errors and sloppy code hygiene.

Financial services companies, IT consultants and telcos were found to be most guilty of this, with the highest number of common weakness enumerations (CWEs) per thousand lines of code.

"We found that overall, organisations are taking application security quite seriously. However, there are clear outliers to this broad finding that put companies and their customers at significant risk," said CAST's senior vice president and chief scientist Bill Curtis. "Without a clear understanding of existing application security vulnerabilities, organisations are not addressing some of the biggest software risks that pose a threat to their business."

Interestingly, the report found that outsourcing had little measurable impact on code quality, with significant differences in the CWE rate of apps developed in-house compared to those outsourced to other firms.

Similarly, there were little statistical differences between onshore and offshore-developed apps. Application size also did not appear to affect the amount of weaknesses present. The biggest indicator of risk appeared to be age, with applications between five and 10 years old presenting the greatest potential for flaws.

The report also had harsh words about Microsoft's .NET programming language, warning that .NET applications had more vulnerabilities on average than Java apps, though it didn't provide numbers. Microsoft's .NET apps developed with the waterfall software evelopment method had the worst scores overall.

However, CAST cautioned that continuous deployment can prove risky, too. Java apps with six or more annual releases had the largest number of CWE vulnerabilities, which could prove a problem for companies that have adopted an agile, DevOps-centric development model.

Picture: Bigstock

British businesses waste an estimated 37 billion a year on agile projects that don't achieve their objectives, a report by IT consultancy 6point6 has found.

The company questioned 300 CIOs about how they're using agile project management methodologies and their successes and failures.

Although the majority (95%) of CIOs have worked in a scaled agile environment with multiple teams working on multiple projects at the same time, there are significant failures that mean the projects don't have the expected outcomes.

Just under one third (32%) of projects are failing because the teams are too geographically dispersed, while 34% have failed because the teams didn't plan before getting started or didn't plan sufficiently as the project developed.

Longevity of support was also a problem highlighted by CIOs. Some 44% of respondents said projects failed because not enough (or in some cases, none at all) documentation was produced to offer ongoing service support of the software created in the project.

"Agile IT in the UK is facing a hidden crisis - 12% of agile projects are failing completely," said Chris Porter, CTO and co-founder of digital delivery specialist 6point6.

"CIOs tell us they expect to undertake six agile projects next year, one in eight of which will fail completely. Given there are about 6,000 CIOs in the UK and that the average agile IT projects costs approximately 8 million, that represents a huge amount of waste. The truth is that, despite the hype, agile development doesn't always work in practice."

The report also revealed that more than half of CIOs think the agile methodology is now discredited, while three quarters aren't prepared to defend it as a way of completing projects anymore. Additionally, half of CIOS think agile processes are just an IT fad rather than a beneficial way of ensuring a project runs smoothly and meets business objectives.

"This is a conservative estimate," Porter added. "We've only looked at agile IT projects that fail completely. This doesn't include the waste involved in agile projects that fail only partially. UK and US CIOs now estimate that nearly a third (32%) of agile projects fail to some degree. The failure to apply agile effectively is a huge problem for the UK."

Staffing was also a problem in agile environments. More than two third (68%) of CIOs think their teams need more architects to produce better results. However, the average in-firm career expectancy of a CIO is only 14 months, which isn't long enough to make a significant contribution to the change required. 6point6 concluded that CEOs must take responsibility to ensure projects succeed, even if the CIO role changes hands.

Project management happens across all industries. Usually used to complete projects rather than business as usual (BAU) work, it comprises a number of stages, from developing the initial idea, to planning, implementing and analysing success of a project.

However, it is a pretty complicated process and here are lots of different ways of completing a project. Although most organisations use formal methodologies' to complete projects, some have their own way of working, which can further complicate matters.

We'll talk through the different methodologies common in the IT industry, stages and some tools to help manage projects without formal qualifications.

Types of project management

Waterfall

As the name would suggest, the waterfall method is a sequential project management technique, with stages that flow downwards, just like a waterfall. Stages include conception, initiation, analysis, design, construction, testing, production/implementation and maintenance, with each stage being fully completed before the next starts. It originated in manufacturing, but has since moved on to cover software development, although because it wasn't specifically designed for software development, it's not really the most effective.

PRINCE2

Projects IN Controlled Environments (PRINCE2) is a qualification-led project management methodology, which takes a project and organises it into manageable chunks and controllable stages. It was developed by the government for IT systems projects and because it's been designed specifically for that, it's more effective than Waterfall project management techniques for software and app design. It's led by principles and processes, making it an extremely structured way of managing a project. Processes are Starting up a Project, Initiating a Project, Directing a Project, Controlling a Stage, Managing Product Delivery, Managing a Stage Boundary, Closing a Project.

Agile (Scrum)

Agile is a much more team-based project management methodology, with collaboration and communication forming the basis of everything. Processes and tools are replaced by individuals and interactions, working software is more important than the documentation issued alongside it and everything is built around responding to change rather than following a plan. It's a much more evolutionary way of working, making it more flexible for projects that are likely to adapt over time.

Lean

Lean is a way of developing products fast. It's very responsive to change like Agile and, as its name suggests, aims to be the most efficient way of developing as possible, eliminating waste, sharing learning with flexibility built in its core. Although it goes as far against one of the formal project management principles as it possibly can, by ruling decisions should be made as late as possible, this makes it super-adaptable, although does put some organisations off using it.

DevOps

DevOps combines elements of Agile project management and lean methodologies and is increasingly the most favoured on technical, such as application development and digital transformation projects, because it's so flexible. It dictates that software and developers should work collaboratively with IT departments, through streamlined communications - the Agile element, but concentrates on getting products and services to market fast - the lean project management philosophy. It's a relatively new practice in comparison with the older methodologies, but is rapidly being adopted in cloud-forward organisations because of its flexibility and room for scale.

The process of a project

What's clear from breaking down the main project management methodologies is that they all comprise similar elements, which is the key to successfully managing a project.

Breaking it down into identifying the goal or problem (initiation), planning the steps to carry out the objective (planning stage), executing the project to meet the deliverables set in the initiation stage (while all the time monitoring and controlling the scope of the project) and finally, closing the project to ensure it's met its objectives, is crucial.

This last stage will also include handing over the project to the operations team, so it can be integrated into the BAU environment, so t can continually evolve if it needs to, making it fit for purpose. Obviously if there is a major update needed that doesn't fit into BAU, it will then need to be handed back to the project management team for the process to start again.

Best tools to help with project management

Although onboarding a project manager is the most effective way of ensuring projects get off on the right foot and are managed successfully throughout, if you're not ready to commit to a formal project management methodology to help plan and execute projects, there are lots of project management tools to help you management one-off projects.

Many of these are free and take the drag and drop approach, enabling you to assign and set tasks, moving them across columns as they progress. Others allow you to track progress to share with the board, if they're all about numbers, data visualisations and ROI.

Although project management requires investment to make it run smoothly, it's worth the outlay, speeding up the time it takes to get products and services to market, while also dramatically improving the productivity of your employees, across departments.

So you've decided to adopt an agile approach to software development. Agile has plenty of benefits over more traditional methodologies like waterfall, with its focus on speed of delivery and working software - but chances are you already know that . What you need to know now is how to implement it.

This is no easy task - as with any behavioural change, you can't force people into it. Rather, you need to make a convincing case to the business about why agile is the way forward for your organisation, and you need to show your developers why it will make their lives easier, too.

Find people who agree with you

The first step is to find people who share the same goal as you.

"People are important as they are the ones that drive as well as adopt change," says Andy Cureton, MD of DevOps specialist ECS Digital. "Finding people that are tired of their current ways of working increases the likelihood of buy-in and adoption of proposed change as they will be looking for ways to improve."

Forming a small group (or groups) filled with these people, rather than focusing on people with specific knowledge of the projects you want to tackle, is essential.

But while it's key to identify developers who support an agile approach, it's harder to get the business to buy into it upfront.

"What we find is that by and large it's relatively easy to get the IT staff on board," says Nathan Wilson, principal research analyst at Gartner for agile development. However, business people used to a more confrontational relationship with IT will not be won over as easily.

"[Businesses are used to] an IT model that says every few years we will do a big project, make something a whole lot better, then go away", says Wilson. Asking businesses to adopt to a continuous delivery model, where improvements are fractional but constant, is a difficult jump for them to make.

"It's really hard for the business to do this because they are used to thinking 'if I don't get my change now I will never get it'," Wilson explains. "The strategic goal is to earn the trust of the business and to build on it."

Or, as Tirrell Payton, managing consultant at SolutionsIQ, a management consulting firm, tells IT Pro: "Organisations are like living, breathing beings in that they have 'organisational antibodies' that will seek to derail any major change and keep things the way they were. People hate change."

To gain business trust, the next step is to find a project that you and your developers can embark on to show how agile benefits the business.

Find the right project

However, picking that project is a difficult step itself. You probably don't want to take on an important IT project with a lot of risk involved for your first agile endeavour.

"You are pretty bad at agile the first time because it's completely different to anything you have ever done before," explains Wilson. "Do it in a controlled environment and build competency and then expand."

But while a small project is a good choice to try out agile, you also want it to demonstrate success, so your business people see the value of the approach.

"Pilots, proof of concepts and MVPs require minimum investment to demonstrate the potential for the organisation," says ECS Digital's Cureton.

Wilson added: "A most effective project to apply this to is one with very low expectations. If everyone is expecting it to fail, that seems to be the broad benefit. I do get the call saying 'we want to try agile out on this major SAP deployment', and after I get done screaming no, I say 'we want to start a little small and someplace where the risk is different'."

But both acknowledge that while small projects are good choices, they are by no means the only choice.

"We don't believe that there is an ideal type of project," says Cureton. "A 'greenfield' project developing a new application or service may have less complexity, but selecting such projects may fuel speculation that agile can only be successful for 'new'."

As Wilson says, if you believe the benefits of agile can apply to a big, problematic IT project, then perhaps that's a logical place to start.

"We have clients where they have an innovation group doing agile stuff but their [bigger, more traditional IT projects] are over-budget and under-delivering, and we see it makes sense to implement agile there first," he says.

Focus on business goals

Once you've picked your project, it's important to focus on the right outcomes, and to ensure your team is set up to reflect those values.

Tirrell Payton, of SolutionsIQ, tells IT Pro that agile leaders should focus on the following questions: "How clearly is the product strategy [aligned with] the people that are implementing it? How strong and explicit is the connection between the strategy and the daily work? Crucial for this connection are the product owner and product manager roles, and the way these roles specify and prioritise work for the development team."

It's also essential to talk about the project with business stakeholders, to keep the development aligned to those outcomes.

Achieve business values

One common mistake on early agile projects is an all-consuming focus on speed, rather than quality, according to Gartner's Wilson. This won't produce the right results for businesses to gain trust in what you're doing, however.

"A danger of the speed is sometimes where clients fall into the quick and dirty trap - but it's better to be quick and small," he says. "You don't want something of very low quality - whereas if it works well and is very small, then you create the business opportunity to scale."

This is especially true if you've picked an agile project centered around innovation and exploring a new business opportunity.

In that case, "by definition there's no current revenue based on what you're doing", Wilson says. "You can produce a very small solution that doesn't do much very quickly, but if it's the first thing in an area then it's worth doing."

This is also the stage of an agile project where experience counts, according to Payton.

"An agile framework on its own, though, is not sufficient; teams need good execution practices as well," he says, explaining that execution refers to the technical process of building, testing and deploying software products.

Executing a project well within an agile framework leads to project management discipline, greater value delivery, and better predictability - all great and measurable benefits that will impress the business.

How do you know if you've achieved agile buy-in?

After doing all this work, how can you tell if you've made any progress on moving your business towards an agile model of operations?

"This will be different for every organisation," says Cureton."There will be key business objectives associated with the adoption of agile and if they have been met then you can argue success, at least for that project."

If you've managed to do that, this goes towards building more trust between the business and IT, and the two will be much more closely aligned on business goals.

He adds: "One key element of agile is continuous improvement. A culture where people are constantly challenging the status quo and looking to raise the bar demonstrates that agile principles have become part of the fabric of the organisation."

From Wilson's point of view, it depends on whether your organisation's mindset has changed. "What I ask CIOs is what happens if someone says, three weeks in, this project is in big trouble? If your response is 'go back and try harder', then agile won't work in your organisation.

"If your response is 'how do we change the plan or stop doing this?' then you will succeed."

But in one sense, agile is not a process that can ever be fully completed, Wilson adds. "At some level if you are ever done, you are in trouble," he warns. "When you realise that you are never done, you are probably there."

CA Technologies CTO Otto Berkes today laid out his vision for "the modern software factory" where agile business practices, round-the-clock feedback and DevOps workflows come together to create a continuous pipeline of new products and innovations.

"The modern software factory gives you the ability to rapidly build and deliver software experiences," he told attendees at the company's annual conference in Las Vegas.

"A great starting point? Agile. To rapidly realise value with minimal overhead, you need to change how you build software."

The sentiment echoes the overarching theme of CA World 2016, which has been predominantly focused on the importance of digital transformation, and completely overhauling how your business approaches software development.

Berkes noted that in many ways, software remains stuck in the past, centred around rigid silos in business areas like HR, sales and marketing. This is a hangover from when software was used to support the business, he said, but now software is a primary an enterprise driver.

"If this doesn't look like traditional IT, it's because IT itself is being completely redefined," Berkes said. "In fact, the new IT will be no IT. I don't mean that IT will go away - it's actually the opposite. Technology and business are becoming one."

"The pressure on you to innovate and achieve velocity is only going to intensify. The beauty of this is that nobody else can do what you can do. That gives you an edge you've never had before."

CA Technologies today overhauled its core product lines, including its security, agility and mainframe portfolios.

Speaking at CA World 2016, chief product officer Ayman Sayed said: "We are very focused on making our products quicker time-to-value, making them easier to use and consume, adding capabilities such as 'try and buy', catering to new personas such as the emerging and more prominent role of the user versus the buyer."

He also emphasised that user experience is a key area for CA, as is supporting line-of-business users, saying: "As more and more technology shifts from IT to the line of business, we need to be there to help these new personas consume our products and solve their most important problems."

The majority of the announcements focused around the company's DevOps products. Its Application Performance Management tool has been made easier to use, with a redesigned interface and intelligent suggestions to guide inexperienced users through diagnosing performance issues.

Integrating the company's APM software has gotten easier with the introduction of a 'zero-touch configuration' agent. CA APM has also added improved analytics capabilities, courtesy of an integration with the company's recently-launched App Experience Analytics product.

To speed up releases, CA announced that it has purchased open source performance testing product BlazeMeter, which it has integrated with its APM tools.

Cloud is a key part of DevOps for many companies, and in order to support that, CA has announced that extensions for Docker, Nginx and RabbitMQ will be available on the CA APM Marketplace.

Likewise, CA APM will itself be available on the AWS Marketplace for cloud-based projects. In terms of SDN platforms, CA has added support for Cisco ACI and Nokia Nuage to its Virtual Network Assurance products.

Also announced were two new security products. First, a behavioural analytics tool for CA's Privileged Access Manager that uses machine learning to identify and mitigate suspicious activity. Secondly, the company launched a new turnkey Identity-as-a-Service solution - CA Identity Service - designed to provide a hybrid identity and access management product to companies looking to quickly and securely adopt new cloud tools.

CA has released a new version of its Agile Central product, focusing on simplifying adoption and deployment. It features new navigation and customisation tools, allowing users to craft their own experiences. "No longer do you have to choose between powerful and easy," Sayed said. "You can get both."

Finally, CA unveiled a series of new mainframe tools. The CA Mainframe Operations Intelligence suite is a new tool that utilises machine learning and advanced algorithms to foresee faults and problems before they have a tangible impact on performance, reducing downtime.

For companies migrating their mainframe software, CA has released the CA Conversion Service, a SaaS offering that automates software investment rationalising when migrating vendors. The new product promises to make the entire migration project easier and more painless for staff.

DevOps tools have come to the mainframe now too, including Release Automation Connector for z/OS for continuous delivery and an updated Endevor Software Change Manager.

CA's Workfload Automation product has also seen a new release, focusing on speeding up API creation and deployment time and increasing testing efficiency. New mainframe-based data security tools have been announced too, including CA Data Content Discovery and CA Compliance Event Manager, helping to safeguard regulated data.

Are you trying to deliver on digital transformation? Sign up to our live webinar on 7 December, sponsored by IBM, to find out how.