AI hasn’t rendered traditional DevOps practices obsolete, new research suggests, but only enterprises with mature practices are recording success with the technology.

According to Perforce’s 2026 State of DevOps Report, 70% of organizations say DevOps maturity materially affects AI success. Indeed, those with mature processes are finding it easier to integrate the technology across the software development lifecycle (SDLC).

Nearly three quarters (72%) of high-maturity organizations told Perforce they’ve successfully embedded the technology in processes, compared with 43% of mid-maturity companies and 18% of low-maturity firms

"The market often asks whether AI will replace DevOps. Our research shows the opposite: AI amplifies DevOps," said Anjali Arora, CTO of Perforce and author of the report.

"Organizations with disciplined engineering practices, automation, strong collaboration, and focus on control, auditability, and governance are the ones scaling AI successfully and turning innovation into measurable business outcomes."

How AI is changing DevOps practices

AI is changing roles in DevOps, researchers found, particularly in testing. The vast majority of respondents (87%) believe that AI will enable engineers to focus less on scripting and more on system design and directing outcomes.

More than half (55%) of QA teams have increased their focus on quality analytics rather than test execution, and 53% said developers author tests directly.

Nearly half (41%) reported that QA teams are also evolving into Quality Engineering (QE) teams, with 39% citing a focus on orchestration across pipelines, environments, and data. Meanwhile, 38% said that business analysts are involved in test creation.

"The research confirms what we are already seeing: AI is helping teams shift up from execution to oversight and strategy, effectively elevating individual roles," said Jake Hookom, EVP of product at Perforce and report author.

"But the research also highlights that governance and auditability need to be a focus for organizations and the collaboration between teams."

Devs are starting to trust AI

Confidence in AI appears to be improving among DevOps teams, Perforce researchers found. More than three quarters (77%) said they have confidence in AI outputs, for example.

Meanwhile, 74% said AI was meeting or exceeding expectations, with 50% measuring AI's value through customer retention or acquisition, 48% seeing faster delivery, and 44% citing revenue or market share impact.

Perforce did warn that governance is often fragmented and incomplete, posing significant challenges for many enterprises. Compliance oversight is split between multiple functions, and only 39% have full automated audit trails, making measurement expensive and inconsistent.

Notably, 74% said that cloud/compute costs and energy usage influence their organization's decisions about AI adoption, and 37% cite these as limiting factors.

"DevOps practices are widespread but uneven. This split explains why DevOps is simultaneously described as 'solved' and 'broken'," the researchers conclude.

"For organizations with mature practices, DevOps works. For those with incomplete implementations, it doesn't. AI doesn't change this dynamic. It amplifies it, widening differences in outcomes, reliability, and cost."

FOLLOW US ON SOCIAL MEDIA

As businesses incorporate machine learning into their commercial strategies, the race to innovate and stay ahead of the market has highlighted some new challenges.

Traditionally, software development (DevOps) and machine learning (MLOps) teams have operated with separate workflows, tools, and objectives. In today’s environment, this leads to inefficiencies and redundancies that can hamper software delivery.

What are the risks of sticking with the status quo? How can the two teams be bridged together?

Siloed pipelines create blockers

DevOps pipelines are built around continuous integration and delivery (CI/CD), emphasizing speed and reliability. MLOps, on the other hand, introduces stages such as data preparation, training models, and validation. When these operations are managed separately, the handoff from data science to engineering can be slow and error-prone. Data scientists may work in one environment, while engineers work in another, often requiring manual steps that disrupt the overall software lifecycle.

Different toolchains only exacerbate the problem. DevOps and MLOps both require automation, reproducibility, and version control. However, keeping two systems running at the same time is a waste of resources when they’re designed to achieve the same goal.

Channel providers who serve these infrastructures typically have to deal with many of these cases, adding complexity without delivering extra value. Silos between teams further complicate matters, with broken communication and misaligned objectives.

Unlike traditional code, ML models often rely on dynamic, data-driven outputs that can change depending on their training data or hyperparameters used. As a result, they don’t always fit neatly into existing DevOps pipelines, meaning standard testing, validation, or security checks can be skipped or inconsistently applied.

These problems increase the time it takes to get AI-powered features to market. Limited traceability of model versions, training data, and hyperparameters makes troubleshooting and auditing cumbersome, raising concerns about governance, compliance, and accountability.

The case for unification

The solution, as many organizations are discovering, is to combine DevOps and MLOps into a single, cohesive software supply chain. This approach doesn’t overlook the unique requirements of machine learning, though. Indeed, it treats artificial intelligence (AI) as if it were any other software component, creating a system of consistent protocols, whether for code snippets or trained models.

DevOps and MLOps share some of the same goals: rapid delivery, automation, and reliability. Aligning around these goals helps organizations and channel partners to operate more efficiently, reduce redundant work, and foster better collaboration.

The way to achieve true unification is to treat ML models as first-class software artifacts. Like binaries, libraries, and configuration files, models should be versioned, tested, and distributed through the same automated pipelines. This ensures unified visibility, so teams can keep track of which model version aligns with which release, reducing confusion and ensuring reproducibility.

Integrating models into these workflows extends automation across the entire lifecycle, from preparing data to deploying it, which cuts down on human handoffs and speeds up delivery from start to finish.

This approach also improves collaboration between data scientists, engineers, and operations teams. Sharing infrastructure and using the same processes makes communication simpler and allows for smoother handoffs.

Governance is also strengthened by subjecting ML models to the same quality assurance, security scanning, and compliance checks as other software components. For channel partners tasked with safeguarding software supply chains, this consistency is essential.

Opportunities for the channel

For the IT channel, bringing DevOps and MLOps together is both a challenge and an opportunity.

Organizations want to use AI, but they often lack the skills or infrastructure to do so. Partners who help customers set up these pipelines enable them to deliver faster, more reliable, and compliant solutions. Channel providers are at the vanguard of AI-driven innovation when they bridge the gap between DevOps and MLOps.

Companies need to be able to quickly and safely migrate models from testing to production to realize AI’s potential. There has to be a single software supply chain where ML models are handled like first-class assets and workflows are automated from start to finish. For channel partners, this method helps customers grow their AI projects while making sure that quality, security, and governance are maintained throughout the software lifecycle.

As organisations race to adopt more software and models, the industry needs holistic governance. Currently, only 60% of companies have full visibility into software that is running in production. Combining DevOps and MLOps into one software supply chain can help companies achieve their shared goals of rapid delivery, automation, and reliability. This will create an efficient and secure environment for building, testing, and deploying the entire spectrum of software, from application code to machine learning models.

Nearly half of developers would quit their jobs over the "embarrassing" tech they have to work with – with three in ten considering resigning over a lack of AI.

That's according to a survey by Storyblok of 200 senior developers, which found that nine in ten use AI to code frequently.

The polled developers said they make use of AI in their coding in order to increase efficiency (30%), to automate routine tasks (23%), and to boost their own skills (22%).

But not all are getting the AI and other tech tools they'd like, the survey found. According to the poll, 47.5% of developers had thought about quitting over an outdated tech stack, with nearly a third (31%) of respondents said that the lack of support with the latest innovations, notably AI, was a main reason to quit their job – though more said they were frustrated by the lack of key functionality (51%) and the difficulties of maintaining it (47%).

Alexander Feiglstorfer, CTO and Co-Founder of Storyblok, said: "The message to businesses is clear – outdated tech stacks are making your developers unhappy to the point of quitting."

Outdated tech a hindrance

Of particular concern is content management systems, with more than two-thirds of developers saying their outdated CMS is holding back their work, and half saying that such old systems are a "constant" hindrance.

And it's not just the dreaded outdated CMS. According to the survey, nearly half – 47.5% of those polled – say the current tech staff at their employer made them think about quitting over the last year, with 31% considering handing in their notice over the last month.

The survey found that 74% of those asked felt their satisfaction and self-identity as developers was negatively impacted by tech stacks, with a quarter feeling embarrassed by outdated legacy systems, and a similar number shamed by their use of workarounds and use of outdated frameworks. Just 14% said they weren't embarrassed by the tech stack they needed to work with each day.

The biggest burden causing developer unhappiness was fixing bugs in legacy systems, dealing with stakeholders who lacked tech skills and don't understand limitations, and shifting corporate priorities.

Pay for upgrades

What can be done to cheer up developers? More than a third called for their employers to allocate resources to modernise text stacks, with a similar figure saying they'd like leadership to have tech knowledge and understanding.

"Rectifying this problem means either paying developers more to stay or investing in updating legacy systems," said Feiglstorfer, adding that pay rises weren't enough – though perhaps talk to your developers before failing to boost their salaries. "Pay rises are a short-term fix as our research shows how closely developers tie their professional identity and happiness to the quality of the tech stack they use."

Feiglstorfer added: "If you want to bring joy back into your development team, and consequently improve performance and productivity, you need to commit to a modernization roadmap. Not only will this reduce developer team churn, but it will open the door to your business being able to more effectively leverage the latest innovations within AI and composable tech."

Over a quarter (25%) of Google’s internal source code is AI-generated, according to CEO Sundar Pichai, hinting at what's in store for the future of development at big tech firms.

Speaking during a recent earnings call, Pichai explained that the firm is using AI within its development teams to improve coding processes, as well as to boost productivity and efficiency.

The code in question is subject to human approval, Pichai added, as engineers are kept in the loop to review and accept AI-generated code. This helps improve engineer workflows and allows them to move faster, he said.

Google’s AI source code forms part of its full-stack approach to the technology, which Pichai noted also involves rolling out its Gemini models more widely to developers and focusing on tools such as GitHub Copilot.

Google’s announcement is a sign of things to come, according to Camden Woollven, Group Head of AI Product Marketing at GRC International Group, though it might take time for the entire industry to catch up.

“Google's got advantages other companies don't, such as massive code repositories, cutting-edge AI tools, and the resources to implement this safely,” Woollven told ITPro.

“Most companies won't hit 25% anytime soon, but they're definitely heading in that direction.”

It’s also important to put the headline statistic in context, Woollven said. While 25% of Google’s code is AI-generated, this doesn’t mean AI is writing full systems from scratch - rather, it’s complementing human development.

AI coding tool uptick

News of Google’s sizable AI code portfolio comes as research increasingly points to the growing popularity of AI coding tools.

According to a GitHub report that surveyed 2,000 software engineers and developers, 97% said that they had used AI coding tools at work at some point. Many of the surveyed developers also reported better code quality and faster language adoption.

A similar survey from Black Duck Software found that nine-in-ten developers reported using AI coding tools, with some of the most enthusiastic adopters working in technology and cybersecurity.

This uptick could have major implications for application development, according to Manoj Chaudhary, CTO of Jitterbit.

"Traditionally, building applications required dedicated developers, creating a bottleneck for businesses with limited resources. This meant a long tail of unmet needs, with countless applications left unbuilt due to the dependence on centralized IT teams,” Chaudhary told ITPro.

“AI is now breaking down these barriers. AI-powered tools are lowering the barriers to entry, allowing anyone with a basic understanding of what’s needed to automate tasks and processes,” he added.

Chaudhary argued that AI coding tools can democratize automation and create more agile, more productive environments.

Question marks over AI code 

Despite the apparent surge in AI code generation, the industry is not without its concerns. Reports show that many developers question the quality of AI-generated code and its maintainability.

According to a report from Google itself, 39% of surveyed developers do not trust and lack confidence in AI-generated code, despite three-quarters (75%) using AI for at least one daily responsibility.

Other research from GitClear, for example, found concerning trends around maintainability in 153 million lines of examined code, predicting that ‘code churn’ - code that must be changed after its completion - could double in 2024.

Black Duck’s research also pointed toward growing developer worries in this regard, with two-thirds of respondents noting they’re increasingly concerned about the security and safety of AI-generated code.

Platform engineering has become somewhat of an industry buzzword in recent years as organizations look to rapidly accelerate the delivery of applications and unlock greater business value at scale. 

This emerging discipline, as Gartner describes it, helps markedly improve developer experience and productivity by “providing self-service capabilities with automated infrastructure operations”. 

Simply put, platform engineering centers around enabling developer teams to focus on building and fine tuning products while rapidly accelerating product teams’ delivery pipeline. 

Similarly, it helps solve the age-old issue of fostering closer cooperation between software developers and operators. 

By 2026, 80% of software engineering organizations will will establish platform teams, according to Gartner research. This prompts some to question whether platform engineering will render DevOps or DevSecOps obsolete. 

Does platform engineering spell the end of DevOps?

This was a question posed to a trio of participants during a roundtable discussion at KubeCon 2023 in April. 

The panel, which included GitLab CPO David DeSanto, Sarah Polan, Field CTO EMEA at HashiCorp, and Stu Miniman, director of market insights at Red Hat, explored at length the rise of platform engineering and whether it spells the end of DevOps as we know it.

While all three agreed this emerging trend has gathered, and continues to gather, pace in recent years, it’s more complex than simply heralding a new era in product engineering and development. It’s dependent wholly on the individual needs of the business, and their ability to navigate change. 

Platform engineering isn’t for everyone, but it could unlock significant improvements to productivity and application delivery if done currently, according to Stu Miniman. 

“When we talk to our developer communities, some of the biggest problems we hear is that there’s cognitive overload and dealing with context switching,” he said. 

“It’s a challenge that all of us have, but especially with developers. Platforms are not new, but platform engineering is a thing that’s been growing for about three years now.”

Miniman noted that, fundamentally, platform engineering is about enabling developers to focus on their primary role and delegate certain aspects of platform management and governance to teams specifically catered around that task. 

In doing this, platform engineering makes things “a little bit simpler” for developers and removes – to an extent – the cognitive overload that many experience when building and managing platforms. 

What problem does platform engineering solve?

For many years, businesses globally have focused significant resources in bringing together the developer, operator, and cyber security elements that all play a key role in delivering software applications rapidly to keep pace with insatiable customer demand.

But asking developers to build then subsequently maintain platforms has placed significant strain on teams, Polan noted. This strain has reared its ugly head frequently, especially with regard to security and risk management. 

“The past couple of years, the previous trend was very much DevOps. So we wanted to encourage developers and engineers to build and run their own infrastructure and then we quickly became cognizant, both from a business standpoint but also the technical and developer standpoint, that the developers didn’t actually want to own and run their own infrastructure because there’s a lot that goes into that,” she said. 

“Whether it’s risk based, velocity based, or not being able to focus on the business problem.”

Miniman echoed Polan’s comments, noting that this increased strain has frequently been referenced as one of the key hindrances to developer teams in addition to the increased security considerations required when developing at pace. 

“DevSecOps is great, but we know one of the impediments to rolling things out faster often are security issues that you need to worry about or getting everyone involved,” he said. 

“You’re talking about golden paths and about giving people the guardrails, governance, and control. In certain pieces, we want to take that off the plates of the developers so they can focus on writing code, modernizing things and taking advantage of technologies. 

“So, if we can make things a little bit simpler and allow developers to focus on their primary role and the stuff they want to be doing, that makes things easier.”

Platform engineering in an era of heightened risk

Surging security threats in recent years have, in part, led to the growing popularity of platform engineering as a discipline, Polan insisted. By charging responsibility of governance, control, and risk management to developer teams, this can lead to overload and causes vital issues to be missed or overlooked. 

Platform engineering helps mitigate this problem by delegating an aspect of this responsibility. 

“I think that it [platform engineering] is becoming increasingly important, especially as we start looking toward the Internet of Things (IoT) starting to explode and edge computing, and having to have this modularity but also looking from the security and risk standpoint,” she said. “We need to be able to follow these different patterns.”

Polan cited the SolarWinds and Log4j incidents as a prime example of this lack of clarity about platform risks and transparency of oversight for individual platforms within an organization’s estate. 

“I think if we look at SolarWinds, for example, not being able to follow exactly where all of this infrastructure and deployments went affected where the potential threat breach was after the initial supply chain attack. 

“So just making sure that we as organizations are in control, but also garnering business value from this by setting guardrails in place and say ‘we want you to be able to do what you want, follow industry standards, use open source’, but also make sure we have adequate control over these things.”

Why DevSecOps is here to stay

All three roundtable participants agreed that platform engineering doesn’t necessarily spell the end for DevSecOps as a discipline. Instead, this forms another part of how organizations build, modernize and manage their platform environments. 

Miniman suggested that platform engineering represents an “outgrowth” of DevSecOps, adding that it’s “an extension of what we’re doing - it’s evolutionary, not necessarily revolutionary.”

DeSanto echoed his thoughts, noting that there is room for beneficial coexistence that will ultimately deliver benefits for developer teams and the broader organization. 

“I sometimes struggle with why something has to die for something else to exist,” he said. “And so, I don’t know what nerd references are right for the audiences, but maybe it’s the Jedi versus the Sith? That’s what it feels like to me.

“I think there’s a lot more overlap and value to that. In essence, platform engineering is making sure your DevOps or DevSecOps teams are more effective - and they’re more effective because they’re working with the platform team and vice versa. 

“One of the things I’ve seen is that the more you go toward infrastructure as code, that could be in the DevOps system, but it’s for the platform team. You end up in this situation where they’re actually related and one is not necessarily killing the other.

Hybrid cloud was supposed to be a stepping stone on the way to full public cloud adoption, or so we were led to believe. Yet as time went on it became apparent that, for many, hybrid was the destination rather than just a temporary stop along the way.

The flexibility offered by hybrid cloud, particularly when it came to things like data sovereignty, IP protection, and running workloads in the optimum location – as well as some cost benefits – were key influencers in this. For enterprises across the board, the benefits of retaining a mix of colocation, public cloud and private cloud environments, often outweighed the lure of the “full public cloud” mantra.

There are certain specialisms and roles that really thrived in the public cloud infrastructure provided by hyperscalers, however, with DevOps being one notable example. The ability to quickly spin resources up and down as and when you needed them enabled this discipline to really come into its own and become the mainstay of development it is now. As such, it’s a discipline that’s very much rooted in the public, hyperscale cloud. Factors such as data portability, responsiveness, and agility (or lack thereof) have made it a challenge to make DevOps work in a private cloud environment.

Is it possible to square this circle and make DevOps truly work as part of a hybrid strategy? Or do IT decision makers have to choose between leaving it languishing on its own in the public cloud or abandoning the practice altogether?

DevOps vs Waterfall

DevOps and continuous integration and continuous deployment (CI/CD) have been a huge leap forward for software development, enabling organizations to move much more quickly through and improve services continuously. Trying to roll these advances back is both impractical and undesirable – they took off because they made a positive impact on business.

Whereas what’s now known as waterfall or linear development follows a predictable, often siloed approach, DevOps and CI/CD are far more dynamic and collaborative. This led to greater productivity, efficiency and creativity, but their less predictable nature is what leads them to struggle in a pre-defined, on premises data center: It’s not typically possible to access additional resources on the spur of the moment unless there’s serious overprovisioning in place, which comes with its own cost and resource problems.

Hewlett Packard Enterprise (HPE) has been sensitive to these issues for some time. Marrying up the need for modern cloud-native development practices with the need for some businesses to prioritize private cloud in their hybrid mix has been one of the areas HPE GreenLake for Private Cloud Enterprise has been seeking to address.

Cloud native wherever you need it

Launched in June 2022,  HPE GreenLake for Private Cloud Enterprise is among the newest elements of the HPE GreenLake portfolio and was built with DevOps in mind.

According to David Crozier, head of portfolio marketing for HPE GreenLake, Services, one of the challenges when it comes to operating a hybrid cloud environment is it rarely has the same feel, functionality, and ease of use that the public cloud does.

“Containers and Kubernetes have increasingly become the go to for enterprises wanting to run cloud-native apps, with options to run on VMs or bare metal deployment,” he says. “This is overshadowed, however, by the complexity and management demands of running them in a hybrid cloud environment.”

“What dev teams want is to be able to create cloud-native apps that offer exceptional digital experiences from edge to cloud. Whether it’s in a public cloud hyperscaler or in a more bespoke private cloud doesn’t really enter into the equation. They want to innovate and they want to be able to do it quickly and easily,” Crozier adds.

What this means in practice is an environment that gives a public cloud feel to DevOps teams through integrations with the likes of Amazon Elastic Kubernetes Services (EKS) Anywhere from AWS, as well as other infrastructure-as-code and cloud-native toolchains. As with the public cloud, HPE GreenLake for Private Cloud Enterprise is self service, meaning users can easily self-serve the management of APIs, command line interface (CLI), VMs, containers, bare-metal and infrastructure-as-code.

It also offers six workload-optimized instances for compute, memory, general purpose and storage.

And because HPE GreenLake for Private Cloud Enterprise is part of the wider HPE GreenLake portfolio, there’s no need for overprovisioning. The consumption-based pricing means users can spin instances up and down as they need them, just as they would in the public cloud, without the data leaving the corporate environment.

An accessible environment

HPE GreenLake for Private Cloud Enterprise makes the power of cloud computing more accessible than ever. Organizations that may have shied away from cloud-native development due to data sovereignty and security concerns can now take full advantage of it without making compromises.

Additionally, HPE understands that what’s generated in a private cloud environment may not need to stay there forever. Through other HPE GreenLake services, it’s easy to move workloads between the private and public cloud as needed.

Cloud computing services are now ubiquitous, used by the very smallest organizations to the very largest but one size doesn’t fit all. A trusted partner such as HPE can help you navigate the cloud landscape and discover what best suits your business needs. And through services like HPE Greenlake for Private Cloud Enterprise, you can be sure you’re getting the full advantages and flexibility of hyperscale cloud computing without ever having to worry about where your data is and who has access to it.

Learn more about HPE GreenLake

The recent acceleration in the artificial intelligence (AI) space represents a “tectonic shift” in DevSecOps and could herald a new era of efficiency and productivity for software developers, according to GitLab chief product officer David DeSanto. 

DeSanto notes generative AI developments have been a source of great excitement across the global tech industry, he tells ITPro at KubeCon 2023, not least of all in software development and cyber security. 

Within days of ChatGPT’s launch last November, devs the world over were fawning over the prospect of harnessing the tool in their daily workflows. The ability to generate and even test code proved tantalizing and appeared to be the silver bullet many had dreamt of. 

How AI can power software development

DeSanto believes the industry is currently “at the precipice” of a revolution with regard to the practical implementation of AI within the software development lifecycle. That’s not to say AI or machine learning isn’t already being used in this regard, but the scale of potential is highly evident across the ecosystem. 

This potential for improved productivity comes amid a period of troubling macroeconomic conditions that are prompting tightening purse strings and placing a strain on already stretched workforces. Simply put, AI tools could make or break how well teams work moving forward.

“I’ve been calling it a tectonic shift in how DevSecOps is done,” he says. “ The reason I feel that way is that there are still many companies who struggle to get a large enough workforce to support what they want to do.

“If you can make your existing team members more effective, then you also make them more engaged, and by making them more engaged, they’re more likely to stay. You are now building a retention component within how you support your team.” 

DeSanto points to code suggestions as an example, noting an AI-based support functionality within the development process could deliver significant benefits to both individuals and broader teams. 

Last week, GitLab announced it had moved its own code suggestions functionality into open beta. The signs so far are promising, with indicators it’s led to efficiency gains as well as improving both quality of life and quality of performance among developers. 

“Now you could have an intermediate developer, for example, functioning and performing like a senior developer,” he suggests. 

Other potential use cases, such as leveraging AI to identify vulnerabilities, would also drastically improve efficiency throughout the development lifecycle and help streamline a notable pain point in the process. 

“Those sorts of things are going to make everyone more effective, but also as a team, as a company. One of Gitlab’s core tenets is that everyone can contribute. You’re truly going to make it everyone can contribute if you apply AI properly.”

How to implement AI in DevSecOps

GitLab’s own research correlates with what DeSanto tells ITPro about the increasing use – and potential – of AI tools in DevSecOps. 

The company’s annual Global DevSecOps report, published today, finds nearly two-thirds (65%) of developers are now using AI or machine learning in testing processes, or expect to be doing so within the next three years. 

Similarly, 62% of developers told the company they use AI or machine learning to check code. DeSanto notes, this marks an increase from 51% last year. This, he says, highlights the fact developers view AI as a valuable tool within their proverbial kit. 

DeSanto was keen to emphasize there are certain nuances within this shift in appetite toward AI tools, though. While research shows a notable increase in interest, he believes seniority plays a role in how receptive developers or security personnel may be toward this trend. 

“It depends on the maturity of the developer in their career, or on their skillset in terms of the true impact [of AI tools],” he says. “A lot of the time, when it comes to improving developer productivity we see it tends to be newer developers who are getting almost like a coach to support them.

“But developers who are mature don’t have that same big thirst. They view it as a way to get things done faster and spend some of that saved time in code reviewing.”

DeSanto also suggests for teams seeking to leverage AI tools, it isn’t a case of deploy and relax. Deployment requires a concerted effort and razor-sharp focus across the enterprise to maximize the use of AI, else it’ll merely amount to a poorly harnessed stack of tools that add little value. 

“For AI to be effective, and for everyone to be effective as a result of this, it can’t just be applied to one part of the software development lifecycle,” he continues. “This is not just the developer, it’s everything involved in delivering software.

“GitLab is an enterprise DevSecOps platform, so obviously we’re focused on covering the entire development lifecycle. In our opinion, you can’t just apply AI to the developer experience and to developer efficiency, you’re not solving the whole problem.”

More than two-thirds (67%) of DevOps professionals have actively “slowed down” cloud native adoption due to lingering security concerns, according to a study from Red Hat. 

The report from Red Hat, based on a survey of 600 DevOps, engineering, and security professionals globally, highlighted security-related concerns as a key barrier to the adoption of cloud native technologies such as Kubernetes. 

More than one-third (38%) of respondents said that they don’t believe cloud native security is “taken seriously” enough and that security investment is inadequate. 

These security concerns represent a long-running theme in the cloud native space, Red Hat’s report noted. 

Over the past several years, adoption rates have grown rapidly amid a marked increase in cloud native technologies and heightened interest among large enterprises. 

But this growth rate is a key factor in why security concerns continue to loom heavy on the minds of engineers and security professionals, according to Fevzi Konduk, head of ecosystem market incubation at Red Hat. 

Speaking to ITPro at KubeCon in Amsterdam, Konduk said that the rapid evolution of cloud native technologies and the relative immaturity in the space are proving a key barrier to operational alignment and presenting dynamic security considerations.  

“Because the technology is evolving so fast, especially around cloud native workloads and the architecture and technology that drives that, behaviors change quite drastically,” he said. 

“This is not a saturated market, or a market that’s reached complete maturity yet,” Konduk added. “It’s still evolving and this is an underlying reason why there are significant security issues.”

Rapid development causing security headaches

A main driver of the uptake in cloud native technologies is the agility it provides to enterprises, the Red Hat study noted. 

“Faster time to market, adaptability, and reliability are all benefits of cloud native technologies and key drivers for enterprises to digitally transform their IT infrastructure,” the report said. 

However, Konduk said that because developers “are full speed ahead”, other functions within the enterprise are struggling to understand or accommodate for rapid rollout of cloud native technologies.

“The developers want to develop as fast as possible and have flexibility, but operations can’t always come up to speed and fully understand what they’re operating or consider potential security issues,” he said. 

“That’s why we talk about DevOps to try and bring this together. But it also means a major fundamental change in the process of how you do development and handle potential security scenarios,” he added. 

Greater engagement between DevOps teams and the security function was highlighted as a key necessity for organizations continuing to steam ahead with cloud native technology development, Konduk said. 

Recent research from Palo Alto Networks found that this closer relationship between vital functions is improving somewhat. 

Its 2023 State of Cloud Native Security report found that many companies are “encouraging a deeper level of engagement between application developers and security tools and teams”. 

81% of respondents said that they have embedded security professionals within their DevOps teams and have witnessed a marked improvement in their ability to identify security issues that might have previously gone unnoticed. 

Security failures cause significant business impact

Failing to adequately address potential security concerns can have a significant impact on businesses, the study found. 

37% of respondents identified revenue and/or customer loss due to a container or Kubernetes security incident. 

“These incidents could result in the delay of critical projects or product releases, as businesses must prioritize security efforts to address the vulnerabilities that were missed in the development stage,” the report said. 

This delay could have a major long-term “ripple effect” on the business, resulting in further lost revenue, customer dissatisfaction or potential loss of market share to competitors. 

“Those types of occurrences can also erode customer trust,” the study added.

To be successful, businesses must select a secure, scalable, and reliable computing platform. At the same time, they also demand openness so development teams can select the tools and technologies that they're skilled in using.

LinuxONE is a hardware system designed to support and exploit the Linux operating system based on the value of its unique underlying architecture. This eBook breaks the system down, sharing how it can provide security at every layer of the computing stack while also giving developers the flexibility they demand.

Download this eBook, to learn how to:

• Secure your data and spark innovation
• Leverage a large partner ecosystem
• Dynamically scale to meet your business needs

Provided by

One of the core trends for the future of software development will be the growing establishment of open source programme offices (OSPOs), according to GitHub.

OSPOs will be responsible for nurturing the relationship between organisations and open source communities as reliance grows on the tools developed by often-unpaid, passionate programmers.

Establishing an OSPO was one of the three key trends in the software industry identified by GitHub in its annual Octoverse report published at its yearly Universe conference taking place this week.

By the open source platform’s estimates, around 30% of Fortune 100 companies already have a robust OSPO working in their ranks. The figure is growing and is expected to be a significant area of growth over the next five years.

The responsibilities of an OSPO are numerous, one of which relates to compliance with open source licenses and also the responsible consumption of open source technology more generally.

“I used to work at Microsoft and my line was always: "You can't expect the community to respect your licence if you don't respect theirs"," said Martin Woodward, VP developer relations at GitHub to IT Pro.

“So you need to, as an organisation, especially if you're in a bank or insurance company, you need to make sure you've your compliance is tight, and so that's why OSPOs are increasingly important.”

OSPOs will also help foster a mutually beneficial relationship between the organisation consuming an open source library and the open source community that maintains it.

Organisations’ and private sector companies’ reliance on open source is growing, said Woodward, and if a large consumer such as a big organisation doesn’t contribute back to the community, the value of the open source code diminishes.

This is because when code is taken from a library by an organisation which then uses it and develops on top of it, but doesn’t contribute back to the project itself, then the organisation diverges from what the community is doing, making it more difficult to ingest the latest code from the open source community.

The final main responsibility of OSPOs will be to also promote innersource methodologies within in-house software development teams.

Innersource refers to the practice of adopting open source-style ways of working on projects that aren’t inherently open source. Innersource workflows are deployed to break down silos within an organisation and promote wider collaboration between different teams.

Woodward’s main advice to businesses looking to establish an OSPO was to first find someone who is passionate about open source development, which he said shouldn’t be a difficult task.

“If you go talk to any bank, or any IT company in the UK, there are people who want to do this at the [individual contributor] level and they just need somebody to organise, somebody to build a community, and they can make it be a grassroots thing,” said Woodward.

He went on to say that cultural change relating to open source within an organisation will start to develop when top-down permission is granted to those who want to kickstart a bottom-up revolution.

Other key trends to monitor

The role of commercially-backed open source projects is also contributing to the growth of the open source community more generally, GitHub said.

The company’s Octoverse report revealed that commercially-backed projects such as Microsoft’s VSCode integrated developer environment (IDE) are playing a significant role in attracting developers to open source contributions.

GitHub’s data showed that 50% of first-time open source contributors did so on commercially-backed projects.

Woodward said that this only works when these large companies have their engineers “working in the open”. In doing so, open source contributors are more likely to contribute to feel part of a wider community, an important incentive to try open source development for the first time.

“By doing that, what we see is the communities that grow around them are typically ten times the investment of people that [the large companies] put in, but it only works if they actually have their engineers working in the open,” he said.

The rising development for infrastructure as code (IaC) practices across GitHub was also identified as a key growth area for the coming years, thanks to HashiCorp Configuration Language (HCL) showing the fastest growth for a programming language in the past year.

IaC refers to the management of physical infrastructure through coded configuration files, rather than through manual methods such as setting up and placing servers by hand.

HCL is the language used for Hashicorp’s Terraform tool which has become increasingly popular for developing IaC practices.

Google Cloud has announced the launch of Software Delivery Shield (SDS) which aims to fortify businesses' resistance to software supply chain attacks that have proliferated in the last 12 months.

Announcing the new product at its Google Cloud Next conference, SDS is a fully managed software supply chain solution that was designed to address five key areas of supply chain security: application development, continuous integration (CI) and continuous delivery (CD), supply of software, policies, and production environments.

The new product aims to equip developers and security teams with the tools to create cloud applications free from supply chain vulnerabilities which has become a mounting concern across the industry

It will also work alongside other Google Cloud services like Google Kubernetes Engine (GKE), Cloud Build, Cloud Deploy, and more, bringing with it a score of features that promote collaboration between IT, DevOps, and security teams.

Providing protection for software at the development stage, SDS includes a new service called Cloud Workstations (preview) which gives developers managed development environments on Google Cloud's platform.

These environments will be browser-based and fully customisable, with options for collaborative configuration across IT admins and security teams that aim to promote better overall app development security, according to Google Cloud.

Cloud Workstations won't store code locally and come with features such as private ingress/egress, identity access management (IAM) policies, and forced image updates to proactively prevent common security issues in development.

Google Cloud's family of integrated developer environment (IDE) plugins, Cloud Code, will also support a new feature called Source Protect which offers developers real-time feedback within their IDE, identifying vulnerable dependencies and license reporting.

Many of the software supply chain-related issues from the previous few years have been rooted in vulnerable dependencies which can be difficult to check manually. GitHub's Dependabot, for example, has been available since April 2021 and offers similar dependency-scanning capabilities to developers with the aim of securing the open source software space.

Software supply chain security is an area of increasing concern for tech leaders. A recent survey of C-suite executives identified that software supply chain attacks are a ‘chief concern’, with just 32% of respondents indicating that their supply chain was ‘very secure’.

Google Cloud said one of the primary routes to attack open source software is through the compromise of CI/CD pipelines and has strengthened both its CI and CD platforms, Cloud Build and Cloud Deploy, as a result.

New security feature integrations will be added to both platforms such as deep IAM controls, isolated environments, and approval gates to give DevOps teams greater governance over app builds. The cloud giant said these are "key parts" of the overall SDS product.

Protecting the runtime environments is also as important has protecting the development pipeline, Google Cloud said, which is why it also added new security features to the GKE and Cloud Run containerised app runtime platforms.

New capabilities for GKE, currently in preview, will now offer a more comprehensive view of the security posture of clusters and workloads. Teams can now access detailed reports, automatically assigned vulnerability severity ratings, and insights into operating system flaws and workload configurations.

All the new reporting available to GKE can also be fed into Cloud Logging allowing for security event information to plugged directly into a business' a security information and event management (SIEM) system.

SDS also introduces a policy engine for the strictest environments, Binary Authorisation, to establish a chain of trust across the entire software supply chain.

It's what Google Cloud is calling a deploy-time security control that only trusted container images can be deployed on GKE or Cloud Run - images must be signed by trusted authorities during the development process in order for them to be deployable.

SDS can be used alongside Google Cloud’s Assured Open Source Software service, announced in May, which ensures that open source software used within an environment has been vetted by Google.

The service now covers over 250 packages, written in both Java and Python, all of which have been scanned and tested for vulnerabilities by Google. In this way, Assured Open Source Software and SDS can both speed up and improve the security of cloud deployment, through the collection and distribution of verified software.

Faster response with Chronicle Security Operations

A separate announcement made at Google Cloud Next was the introduction of Chronicle security operations, a cloud-based software suite that aims to allow teams to detect, investigate, and respond to threats more quickly.

Chronicle Security Operations will bring together SIEM technology with the security orchestration, automation, and response (SOAR) products from its earlier Siemplify acquisition, as well as its Google Cloud's own threat intelligence and Mandiant's incident management tools, and bundle it all under new Chronicle branding.

“With Chronicle, we no longer have to make the tough decisions around which data we can afford to log and how little retention we can live with,” said Mike Orosz, chief information security officer at infrastructure management provider Vertiv.

“Insufficient security event monitoring is a thing of the past, and there is no better time than now to align to best practices.”

The branding for Siemplify's SOAR and SIEM tech will now be Chronicle SOAR and Chronicle SIEM respectively, and host of new features have reached the product family, all in preview.

The suite seeks to provide teams with cloud-scale security data using Google’s hyperscale infrastructure and petabytes of contextual information, up-to-date threat intelligence, and automated responses to common security threats including phishing and malware.

With Chronicle Security Operations, Google Cloud also aims to provide security teams with a number of quality-of-life features and data aggregation tools to improve threat insight.

These include a single display that combines information on chosen entities from multiple data sources, including Google Cloud Threat Intelligence, integrated alert management between Chronicle SIEM and SOAR, and pre-built responses to cloud-based alerts to speed up threat resolution.

Data platform company Splunk has filed a lawsuit against rival Cribl, alleging that its CEO and other employees stole source code and technical secrets for its own advantage.

Splunk alleged that former employee Clint Sharp, who left the company in 2017, took code without permission that he then used when he co-founded Cribl the same year.

Additionally, it claims that since its formation Cribl has illegally sourced technical documents from outgoing Splunk employees in order to undermine the company.

“Mr Sharp founded Cribl using code he intentionally and unlawfully took from Splunk when he was a Splunk employee without a licence or permission to do so,” read the complaint.

“Since that time, Cribl and Mr Sharp have recruited numerous Splunk employees to Cribl, and have systematically encouraged employees to take Splunk’s confidential technical and business documents with them. In turn, Cribl has used the information it misappropriated to compete unfairly against Splunk.”

In a blog post, Splunk alleges that it attempted to settle the matter privately, but that this was unsuccessful. It characterises Cribl’s actions as a “coordinated campaign of misappropriation” and claims that the company used Splunk’s Technology Alliance Partner (TAP) programme to further its theft.

After launching in 2017, Cribl joined the TAP program, an agreement that allows partners to add to Splunk’s Enterprise platform, through the use and limited copying of Splunk software.

However, the TAP program agreement states that partners may only use the software to demonstrate the use of extensions with the software and to develop extensions further. Using copies of the software to determine source code would constitute a violation of the terms of agreement.

"The allegations in Splunk’s lawsuit are false, and we will defend against these baseless claims," Cribl stated in a blog post.

"We have built interoperability using our own hard work and open source implementations, such as Eventgen. While Splunk tries to stifle competition through litigation, we will keep our relentless focus on our customers to give them choice and control over their data."

Splunk has requested an injunction against Cribl, Clint Sharp and all associated Cribl entities, from further infringing Splunk copyrights. It has also requested an accounting of the profits that Cribl has made from the activity and subsequent damages. The company has requested a trial by jury.

"Proving infringement of copyright in source code, as Splunk is alleging here, tends to be difficult," said Michael Buckworth, founder at UK law firm Buckworths. "A former employee would be incredibly foolish to use a carbon copy of code he has misappropriated; rather he would likely recode the software to deliver substantially similar functionality.

"New code achieving the same outcome as existing code is unlikely to infringe copyright in existing code leaving the damaged party to look at patent infringement or breach of contractual obligations," he added.

"Splunk may believe that they can prove each allegation that they have made with the result that they could secure significant damages from Cribl, or even have it shut down.

"Alternatively, this may be a strategic move to dry up investment and other financial support in Cribl. There is nothing that will turn off investors more than a high-profile IP dispute! Either way both parties will now need to prove their position and live with significant costs if they fail.”

Splunk operates in more than 21 regions internationally, providing customers with a suite that provides oversight over their data and allows them to conduct big data analytics. It also aims to improve observability across multi-cloud environments. Last year it announced a new cloud security suite in addition to its other offerings.

Cribl supplies data management solutions to its customers, aiming to provide data flexibility and stack observability. Its service Cribl Stream is used by DevOps engineers and other industry professionals in order to process data in real time, supply context to data, and encrypt sensitive fields.

IT Pro has approached Splunk for comment.

This article has been updated to include a statement from Cribl.

This automatically-generated transcript is taken from the IT Pro Podcast episode ‘How to scale a tech platform'. To listen to the full episode, click here. We apologise for any errors.

Adam Shepherd

Hi, I'm Adam Shepherd.

Sabina Weston

And I'm Sabina Weston.

Adam

And you're listening to the IT Pro Podcast, where this week we're looking at the challenges of scale.

Sabina

Building an application from scratch is a challenging proposition in itself. But for a startup looking to make its name, that's only half the battle. The real challenge comes when you have to scale that platform, expanding its scope and capabilities at a rapid pace without overextending your resources, or burning out your staff.

Adam

It's a task that many businesses are forced to grapple with. And there are many factors for a CTO to take into account, including which technology choices to make, how best to structure the application, and how to build in redundancy. This week, we're joined by one technology leader that has been through this process: Avinash Gangadharan, CTO of car sharing company Turo. Avinash, thanks for coming on the show.

Avinash Gangadharan

It's a pleasure, Adam. Thank you for inviting me to the show.

Sabina

So to set things up a little, could you walk us through what Turo's infrastructure looks like?

Avinash

Oh, well, so we are cloud native. You know, since the inception of Turo, we have been primarily hosted, hosted on AWS; that's to start with. So majority of the services run on AWS. We are containerized, our apps are containerized. Our back end, runs mostly on Java Spring, our data tier, you know, is again, AWS services, MySQL RDS, we use Redis for caching, a bit of memcache. On our front end, web front end, we are primarily a React.js shop. And our apps are native, built on Java/Kotlin on Android, and obviously Swift and Objective C, we have a bit of Objective C left on our apps as well. That that's, you know, our programming stack, you know, if you can, programming language stack; going back on the cloud side, you know, besides just you know, real time databases,` WLTP on RDS. Our data warehouse is also primarily on AWS, we use Redshift as our primary data lake. We have a bunch of Jenkins jobs that runs ETL. We are actively migrating those to Spark EMR. Yeah, that's precisely what what our data warehouse looks like. visualisation is on Domo mode primarily. And there is work going on to move to Tableau. So that was, you know, data platform, a bit about our application stack. On a network security layers. We got CloudFlare, you know, that protects us, has been working really well. On observability, we got New Relic logging, you know, as well on New Relic and data science, you know, which is a critical part of Turo. You know, most most of our models are built, tested, run on Datarobot. They have a few that run in house, but that's what we primarily, that's at a very high level, what our stack looks like.

Adam

So that's quite a diverse stack. How quickly has Turo had to scale up from a back end perspective over the last several years?

Avinash

Well, the last two years, I'd say has been the most adventurous for us. But over time Turo haa scaled, business has scaled, you know, significantly. In fact, in the last 10 years, the business has scaled 100x.

Adam

Oh, wow.

Avinash

Yeah. So from that perspective, you know, the scale has more more, you know, been gradual, since, since the beginning to like, mid, or q3 of 2020. And starting q4 of 2020, you know, late q3 or q4 of 2020. That's when we saw massive growth, massive growth in our volumes, you know, magnitudes that we were not necessarily ready for. We were not prepared for. We knew that we were going to grow. We had plans to grow, you know, our platform, but our plans were like 18 month, 12 to 18 month plans. We had to expedite that and squeeze that into like a three month, four month plan.

Adam

Oh wow.

Avinash

It was a fun and adventurous time for us.

Adam

So what kind of considerations did you have to make when it came to dealing with that kind of really rapid need for scaling at that level?

Avinash

You know, if you think about it, once, you know, a stack, that's cloud native should have capability to scale horizontally fairly easily. Pour money, you know, get some more hardware, provision more hardware, and you should be able to scale horizontally. Theoretically, that's how it works, you know, practically, I don't think majority of the people out there who start, you know, building a product for a startup, build it in a way where this is actually true, you know, you can elastically scale horizontally, in theory, to some extent, but your bottlenecks are still, you know, your, your, primarily your data tiers, that's what happened with us. So on our applications tier, we could very well scale horizontally. But our bottleneck was our data tier, you know, our, primarily our MySQL RDS instance. You know, as any other startup out there, we also have a monolith, we're trying to break down, break it down. And, you know, so we focused on our data tiers, that was our biggest focus. And we knew that over the years, we have done things that got us out there in terms of future timelines, in terms of time to market really quick, but there was a lot of tech debt that we had accumulated. Off the bat, we actually clearly understood, if I, if we focus on the data tiers, if we, you know, just put fundamental patterns in place, you know, read/write splits, and offload certain things on off hours. Those are the kinds of things we did; tune our SQLs. It might sound very fundamental, but we put a lot of effort in tuning our SQLs, we identified, you know, these multiway joins that we had written many, many years over, we've written over and over again, just to get things done quickly. And we clearly identified how these were soon becoming bottlenecks. So to summarise, we focused on our data tier. And we knew that that was a bottleneck very clearly. And we put an effort to offload, you know, a bunch of workload from our primary source to read replicas where we could and tune the SQLs. Just by this effort, it was a few months, a few months of effort, we created more than 98% capacity on our database, basically. And that helped. So it was fundamentally, you know, resolving some of our tech debt, but in a way, which was quick, we did implement, you know, patterns, like read/write splits, but there's a long way to go, we have a lot more to do. You at some point, we will, we will hit that bottleneck again. And we are actively working on now. And another learning that we had, as part of all of this was, you know, we really had to put a lot of focus on our tech platform. I believe that engineering is successful only if the business is successful. There is no question about that, you know, if the business does great, you know, the success is translated to engineering success as well. So all this time, we will really making sure that, you know, Turo as a business is growing healthy, year over year. And that's what our effort was. But obviously, at the back of our mind, we knew that, you know, there's tech debt that's getting accumulated, we started getting more and more conscious about it, I'd say, you know, late 2018, early 2019, and we put in a lot of efforts to modernise, we just didn't anticipate that, you know, the end of COVID, end of the first set of COVID, would bring in volumes that would surprise us, and, you know, be putting some of these things in place to expedite some of this effort. We moved a lot of our effort towards platform scaling. And that just did it for us.

Sabina

So what has been the biggest challenge of this process?

Avinash

Well, if I think about it, I think the time we had on hand was the biggest challenge. We identified that we are getting volumes of traffic back to, back on Turo fairly quickly, but the time from then, when we said that yes, we're going to get you know, a lot of volume now in I'd say, q3 of 2020, we identified that you know, I guess early q2; the time that we had on hand, and the projection of the volumes, made it very adventure. So basically, very challenging. So basically, you know, the plans that we had, that spanned across many months had to be squeezed into, like, you know, three, four months of an effort. So the time that we had on hand to be able to manage the volumes, was the biggest challenge. Besides that, you know, I've seen in my previous, you know, places that I've worked for the challenge in managing tech debt. And resolving tech debt has always been the challenge of getting work prioritised. You know, you're working on a bunch of product features, business features, you know, and then you have a lot of this tech platform work that you got to do. How do you have a convincing argument that this is the right time, you know, we at Turo have a great culture from that perspective, we, everybody understands tech, everybody understands the challenge of tech. So it was, you know, that made that part of the equation was a lot more easier, there was no convincing required; everybody kind of got there was a, there's a lot of trust in each other. And you just, you know, focused on getting this done, we spent a lot of effort, actually majority of our effort was spent on platform stability, we really dialled down a bit on our, on our feature development work. And that focus really helped. So there was a company wide focus. In fact, we made, you know, the availability, security and the performance of our platform as the top three goals of the company. And since then, it has been like that.

Adam

So speaking of the people side of the equation, then, have you found talent acquisition to be a barrier to scale at all? I know a lot of organisations have struggled over the last couple of years, in particular, with recruitment of skilled engineers and developers as a kind of particular pain point. Is that something that you've experienced at all?

Avinash

Oh, absolutely. I think, you know, the scale of business comes with two challenges. One, how does the platform scale with the business? Two? How does the team scale with the business. So scale, you know, has a people impact as well, as well as the platform impact. So the ability to hire fast enough and quickly and build the team, onboard them, get them familiar with the codebase, platform, and get them up and running is a huge challenge. We had those too, it became bigger because of how, you know, work, the style of work was evolving because of the pandemic. You know, people wanted to be, people I feel started looking at what remote is for them. They got comfortable with remote, we have, we're doing this podcast remotely. And that change, actually was was a bit of a difficult one for us. You know, Turo really values its culture a lot, you know, we, we are a company, where we feel that, you know, the human interaction is critical for the success of our business. It's part of our culture, it's the core element of our culture. We, you know, were nervous about going fully remote. And the talent base out there was moving very fast, than where, than how we were on this aspect. So it did pose a challenge, we, initially, we were like, well, you know, we gotta go to hybrid, we're not necessarily going to go to fully remote, or make coming, coming to office as an optional thing. And that poses a bit of a challenge for us. So initially, we started with, you know, trying to find people from within areas that our business, you know, existed in terms of getting set up as a payroll, and that that was a limiter. We realised it, you know, early 2022, I'd say. We missed our 2021 hiring goals, to be very honest, you know, and we missed it significantly. But it was a stark learning for us, we, we, you know, changed courses, we felt that we got to be adjusting towards where, you know, the talent is. And in the last, I'd say, you know, three months, we've seen a huge difference. So, one was, of course, you know, challenge posed due to how we felt we want to be as a team, you know, how remote, hybrid, in person, the whole mix of things, you know, how we thought about it. And the second thing was, you know, the inflation in the competition of the tech talent market overall has been crazy. If you if you can, if you know, there's a global talent shortage when it comes to tech, right?

Sabina

Yes. that's happening in the UK as well.

Avinash

Yeah, exactly. It's everywhere. It's not just, you know, US, it's everywhere. I see this across the globe. And so hiring has been a challenge. So we've done very creative things to figure out, you know, how to attract talent, you know, and we have lots of competition, you know, Turo, has to compete with the Metas of the world, the Googles of the world, you know, and you're not, you know, we don't have as deep pockets as these companies have yet. So it's definitely a challenge, takes a lot of convincing. We knew, you know, that we have to really focus on right kind of sourcing, targeting the right, and the people who are, you know, not looking at just compensation, from a short term perspective, they're looking at being part of a mission, and they're really, you know, investing in a longer term future. So. So, yeah, to answer your question, you know, it has been challenging to scale. But I think we are now at a place where, you know, we feel very confident about growing it. Just to give you some numbers, you know, over the last 12 months or so, we've almost tripled the size of our team.

Sabina

Wow, that is, that is amazing, especially given the talent shortage and pandemic. And I was just going to ask, because we, there were Gartner studies, hiring is definitely a challenge nowadays. But there's another worrying statistic, which I came across recently, when we covered, I think some Gartner research, basically. But the report found that only less than a third, around like 29% of surveyed IT workers said that they are planning to stay with their current employer, which means that 71% were planning to move on in the near future. So can I just ask like, how, if when, when hiring is, is also a challenge, like what what steps do you take to retain this talent when you manage to employ them?

Avinash

Great question. I think attrition has never been a huge challenge for us. And that's a testament of what our culture is about. I think once you know, we get people in, and work with us as a team, we see retention not being a huge challenge. Attrition has always been in my past four years here at Turo has been not not a huge challenge for us. And same thing, you know, continued over the pandemic as well, even with you know, a lot of you know, attrition across the board, we were fairly, you know, healthy in that aspect.

Sabina

That's good to hear.

Avinash

Yeah, and, you know, our team is small, though, you know, we're not talking about, you know, hundreds and thousands of engineers, like other big places, you know, at the moment, our team has roughly about 110 I'd say.

Sabina

That's still pretty, very good.

Avinash

Yeah. So, attrition has never been, you know, a huge challenge for us in engineering and if across the board, you know, at Turo you know, a lot of things play into that; we are very transparent as a company, you know, we are really, you know, a people focused company. And we value our culture, you know, a lot and, and therefore, I think, once we see people joining Turo, they really fall in love with Turo, you can see that from you know, our Glassdoor reviews. And you know, lately we've been ranked number one in the Best Companies to Work For in various, various kinds of service.

Sabina

Wow. So maybe I should apply as well.

Avinash

We're always hiring, Sabina.

Sabina

No, I don't think you take me. I lack the qualifications, definitely.

Adam

Let's talk a little bit about some of the learnings you've taken from this experience. Was there anything that you found particularly surprising about both operating at scale and about reaching that level of scale in the first place?

Avinash

Yeah, I think there were many. And if I, you know, prioritise my answers right. I think the, the thing that surprised me the most was how attractive we were to the bots out there. And to these, you know, request based attacks that happened to us, you know, and I was very surprised, and why would someone try to, you know, DOS Turo or, you know, do a request based attack on us, you know, what's going on? And that surprised us for quite a bit. We were not necessarily sophisticated in that area, you know, 12 months ago, actually, sorry, not well, about 18 months ago, I'd say, you know. So that was, I think my biggest surprise, you know, keeping our incoming traffic in control, in check, was was something that, you know, was, was, was challenging earlier.

Adam

How did you respond to those problems?

Avinash

Yeah, the, you know, not very well, initially, to be honest, we, yeah, initially, you know, we had a lot of downtime it took, this was, it's you know, in, in 2019. Bit in 2019, a bit in early 2020, not early 2020, late the second quarter, 2020. We, we've faced challenges, and we were not necessarily, you know, doing the things we should have, thinking back.

Adam

Such as?

Avinash

Such, as, you know, like, you know, really securing our network traffic through products, like, you know, Cloudflare out there on the edge, or AWS Shields, you know, like edge security, you know, we didn't didn't put a lot of focus on, you know, securing our traffic on the edges, you would take them all in, you know, onto our clusters. And you would put all kinds of, you know, these rules, these policies, this logic on our app tiers to basically say, all right, bad traffic, you know, do a no op, it's too late if you do that, but, you know, if if the traffic makes all the way to a pod that's running, you know, your application. And that's one thing, which, which surprised us, you know, and, you know, we even didn't necessarily feel that we could be a target for these requests based attacks.

Adam

I suppose it's that thing where you kind of don't expect as a business to be hit until you, until you're, quote unquote, successful. Right. But I think what businesses class as successful for themselves is not necessarily what an attacker would class as successful enough to make it a legitimate target.

Avinash

Exactly. And that was a big learning for us. But, but, you know, the, once we had, we had an incident where, you know, purely because of request request flooding. You know, we took a downtime for about, I'd say, you know, over the period of three hours of that attack, about 45 minutes, we were down, and that was horrible. So, you know, that that's the time we decided that we got to just go all in on edge security. And we did, that helped us significantly. And then, you know, immediately after that we saw a massive efforts on account takeovers. So that was the next surprise that came in. All right, you know, there's a lot of interest in taking over Turo accounts. And we saw all kinds of credential stuffing attacks, many many, you know, different kinds of attacks to take over accounts, Turo accounts. So last year, I'd say somewhere around the middle of last year, we put in a lot of effort on account security; account takeover, specifically. And I think what we now have is something close to world class, I'd say, when it comes to securing Turo accounts. Besides these, you know, the other challenges are more usual in terms of scaling platforms, you know, identifying bottlenecks, putting in right, right kind of monitoring, you know, and alerting like observability overall, you know, observability I feel if you ask anyone out there in any, you know, head of engineering, business of our size, I think the answer would always be we could do a little better on observability you know, you're always like, you know, a tad short of where you want to be, I feel. And we will always a tad short, we kept on improving, we felt Oh, there's a little bit more that we got to do. And even last night, you know, I was talking to one of our infrastructure engineers, and we were like, alright, you know, there's flakiness on search. And maybe we could do a little bit more on our liveness and readiness probes, you know. So, yes, so those were, those were the other challenges, we felt that, you know, observability could be better. And we, again, invested a lot in that, you know, we consolidated our logging and monitoring platform. I think that that helped us a lot

Adam

Is that one of the things that you would kind of class as being a key enabler of scaling up, that kind of observability piece?

Avinash

Oh, yeah, I, you know, I, I think that it's very up there, you absolutely need to, I feel, you need to have world class observability on your platform, you need to know what's happening, you need to be able to reduce your, you know, mean time to detect issues significantly, if you can detect issues quick, you know, and quick, when I say quick, I'm talking about, you know, minutes, in fact, minutes going down to single digits for for your most gnarly problems, if you can detect, you know, incidents, the root causes of the incidents, you know, in single digit minutes, you I think you do, you're doing great as a business of our size, obviously, when you grow larger, you know, you got to reduce MTTD, you know, significantly, and you got to take it in sub minutes. And then it also helps you with, you know, resolve things; MTTR, mean time to resolve, I think, I feel that, you know, both of these are very critical for you to keep the platform up, you know, stay on top of availability, and it's critical, you know, a few minutes of downtime can have a huge impact on, you know, what your perception is to your customers. So observability plays a key role for you to identify these areas and and understand where your challenges are as a as a platform. And, and clearly identify those bottlenecks basically.

Adam

So were there any technology choices that you made early on in Turo's lifespan that you felt kind of maybe tripped you up a little bit as you got further down the road and needed to scale more rapidly?

Avinash

Yeah, I've been here for about four years, and one of the things that, you know, I was pleasantly surprised and very happy about was how modern our stack was, from the get go. There was, you know, a huge emphasis on, you know, being simple, being nimble, but still being fast. So, besides the huge bottlenecks on our data tiers, and a few basic, you know, things about splitting code into, you know, more meaningful chunks so that they can be deployed in, you know, in more isolated, you know, forms, like, clusters, and based on workloads or whatnot, you know, besides some of those simple things, you know, we've always been, you know, fairly good in terms of keeping the stack modern. Now, on one area, I feel that, you know, we could have done better is our velocity to move away from Objective C. And that, you know, I think we undermined a little bit, but over the last, I'd say, 12 months, we've really sped that effort up. And we knew that, you know, hiring individuals with Objective C in our job requirements are not necessarily, it's not necessarily an easy thing, you know, it's not very attractive, you know, obviously, the modern day modern day, iOS developer wants to be working on Swift, they moved on. And that's, that's one area, I'd say, you know, we could have done a little better, but over the last 12 months, we've done really good.

Sabina

So, what advice would you have for other organisations looking at scaling up their infrastructure?

Avinash

Yeah, I think, you know, I feel that when you are part of a budding business, you know, the business scale of the business is fairly small, you're focused on really growing the business, you know, you tend to make decisions which completely undermine the fundamentals of you know, platform design, basically. Some of these fundamentals are not very difficult or tough ones for you to go, you know, to go get from the beginning, it's just it's just fear, I guess it's you know, or maybe shortcuts that makes you think that alright, you know, I'm not going to, you know, architect my data tiers, so that the data is organised, you know, in a more, in a better structure, put everything on one instance, you know, write some SQLs and get stuff done.

Adam

Yeah, just get it done now and sort it out later. You never, you'll never do that.

Avinash

Yeah, I think I think, you know, there's always an argument about how important time to market is, when you're an early stage startup, I completely understand. But the effort in these days of how modern, you know, distributed computing has become, the effort involved, for you to identify and work on these fundamental principles of scale is not as much as it was 10 years, 15 years ago. Yes, I'd say, you know, people making these wrong choices now. It might appear to be a bit arrogant, but it's short sighted, I feel. You know, you've got to be putting these efforts from the get go because it's easy. It's not challenging anymore.

Adam

Well, on that note, I think that's about all we've got time for this week. But I'd like to thank Turo's Avinash Gangadharan, for joining us.

Avinash

Thank you, Adam.

Sabina

You can find links to all of the topics we spoken about today in the show notes, and even more on our website at itpro.co.uk.

Adam

You can also follow us on social media as well as subscribe to our daily newsletter.

Sabina

And don't forget to subscribe to the IT Pro Podcast wherever you find podcasts. And if you're enjoying the show, leave us a rating and review.

Adam

We'll be back next week with more analysis from the world of it but until then, goodbye.

Sabina

Bye.

Building an application from scratch isn’t exactly easy. Startups wishing to create a digital platform for their business will need to contend with technical recruitment, infrastructure choices and the numerous headaches inherent in the development process. However, even once your platform has been built, the struggle still isn’t over.

As your business grows, your platform will need to scale too, and managing this process without reinventing your architecture from the ground up can be challenging. This week, we’re speaking to Avinash Gangadharan, CTO of car-sharing startup Turo, to find out more about the company’s approach to platform scaling, and how he addressed some of the challenges it presented.

Highlights

“A stack that's cloud native should have capability to scale horizontally fairly easily. Pour money, provision more hardware, and you should be able to scale horizontally. Theoretically, that's how it works; practically, I don't think the majority of the people out there who start building a product for a startup, build it in a way where this is actually true. You can elastically scale horizontally, in theory, to some extent, but your bottlenecks are still primarily your data tiers - that's what happened with us.”

“I think the thing that surprised me the most was how attractive we were to the bots out there, and to these request based attacks that happened to us - and I was very surprised. Why would someone try to DOS Turo or do a request based attack on us; what's going on? And that surprised us for quite a bit. We were not necessarily sophisticated in that area about 18 months ago, I'd say. So that was, I think, my biggest surprise, keeping our incoming traffic in control, in check, was something that was challenging earlier.”

“I feel that when you are part of a budding business, the scale of the business is fairly small, you're focused on really growing the business, you tend to make decisions which completely undermine the fundamentals of platform design, basically. Some of these fundamentals are not very difficult or tough ones for you to go get from the beginning; it's just fear, I guess, or maybe shortcuts that make you think that alright, I'm not going to architect my data tiers so that the data is organised in a better structure, I’ll put everything on one instance, write some SQLs and get stuff done.”

Read the full transcript here.

Footnotes

• How to become a DevOps engineer
• Streamlining DevOps in hybrid, multi-cloud, on-premises, and edge environments
• IT Pro Live: Scaling enterprise DevOps with a platform team approach
• The IT Pro Podcast: Stamping out scope creep
• The IT Pro Podcast: Everyone needs Kubernetes certs
• The IT Pro Podcast: Coping with technical debt
• IT Pro Panel: Defining DevOps
• Scale automation more easily
• How to scale your organisation in the cloud
• The 'new normal' is pushing us closer to the edge
• What is edge computing?
• Top considerations for building a modern edge infrastructure
• From edge to cloud – and everywhere in between
• AWS to invest £1.8bn in UK data centres and other cloud infrastructure
• Amazon Web Services review: AWS packs in more features than any other cloud service provider
• Best practices for running Microsoft SQL Server on AWS

Subscribe

• Subscribe to The IT Pro Podcast on Apple Podcasts
• Subscribe to The IT Pro Podcast on Google Podcasts
• Subscribe to The IT Pro Podcast on Spotify
• Subscribe to the IT Pro newsletter
• Subscribe to IT Pro 20/20

Containers are increasingly being used to improve security and speed up application development - with businesses noting better app performance as a result - and are becoming standard practice for implementing SecDevOps workflows.

However, as organisations feel under pressure to launch new products and services at a faster rate, innovation can be limited when resources are focussed on managing infrastructure and security.

Download this whitepaper to learn how to address application development challenges with Red Hat OpenShift on IBM Cloud so you can focus on your business objectives.

Provided by

IT teams are tasked with ensuring their organisation has the right infrastructure to support its business applications, while developers develop and support these apps. These two areas often collaborate in order to achieve business targets, but this collaboration doesn’t come without its challenges.

DevOps can alleviate these issues by bringing together development methods, extensive testing, governance and automation, for continuous integration and delivery. However legacy IT infrastructure can hinder its agility unless IT teams are committed to its evolution.

Download this whitepaper to learn why investment in the modernisation of legacy storage infrastructure results in greater DevOps efficiencies.

Provided by

This automatically-generated transcript is taken from the IT Pro Podcast episode ‘Stamping out scope creep'. To listen to the full episode, click here. We apologise for any errors.

Hugh Armitage

Hi, I'm Hugh Armitage.

Adam Shepherd

And I'm Adam Shepherd.

Hugh

And you're listening to the IT Pro Podcast where this week we're channelling Thom Yorke and talking about creep.

Adam

There are any number of potential pitfalls associated with running a live development project within an organisation. But one of the hazards that frequently ends up causing problems for dev teams is the dreaded spectre of scope creep.

Hugh

As projects progress, it's easy for their remit to gradually morph and expand, eventually growing to the point of becoming unwieldy and impossible to manage. Controlling and limiting scope creep is a finely honed craft, and one that the best project managers have perfected, because without keeping a close eye on it, unrestrained creep can have major negative impacts on an organization's development strategy.

Adam

Joining us this week to talk about the dangers of scope creep, as well as some of the strategies for preventing it is Jess Cregg, developer advocate at LaunchDarkly. Jess, welcome back to the show.

Jess Cregg

Thanks so much for having me.

Adam

So, Jess, you've worked on a number of projects. To start us off with, can you give us some examples of scope creep in action?

Jess

Yeah, absolutely. So scope creep. It's, it's, it's a very human problem. And I was chatting to a lot of people about it recently, I found one person put it really well. They said that scope creep often happens when you confuse your project team, with your product team's aims, it can often happen when you're designing for the problems that you're going to have in the future, as opposed to the problems that you're facing right now. So one of my favourite instances was I was chatting to a speaker that I met at a conference recently, and he was talking about how he was once working on a team where they were creating a UI and a database for changing the country value in their product, a value that could have been tweaked with three lines of code and was probably going to be changed once every three years. So that's thousands of pounds for something that gets used once. It's maddening.

Adam

And that seems like it should be, as you said, a fairly simple problem to address and one that really doesn't need that much engineering, you'd think.

Jess

Absolutely. But it's, it's one of those problems that's hardwired into us, we experience the pain once of something going horribly wrong. And we know the worst case consequences. So we try to avoid those in a very helpful human evolutionary science. Scope creep can come really, it can really easily infiltrate your team. And one of the ways that, one of the one of my favourite ways of preventing it from happening is to keep your teams really small. Everyone's talked about the, everyone speaks about the the two pizza rule. Have you heard about that?

Adam

Yes. It's one of my favourite rules.

Hugh

I'll have two pizzas, please.

Jess

So the two pizza rule is a rule that was made really famous at Amazon. It's another FAANG concept that's taken the entire industry by storm, essentially, it's the idea that your team should only be big enough so that they can be fed by two large pizzas.

Adam

Yeah, it's a good rule, but slightly flawed in that I know many workplaces where that would result in a development team of one and a half people.

Hugh

I was gonna say two, but one and a half. Yes.

Jess

I mean, it's not only great for keeping team delivery orders down, but for preventing teams from growing. But when I was recently reading the DevOps Handbook they're talking about one of the main reasons why this is implemented is, is due to, we have a hard limit of the number of relationships that we can form with people.

Adam

This isn't something I'm familiar with.

Jess

So it's the phenomenon that as humans, usually we can form quality relationships with about 150 people.

Hugh

It's like the policing thing, how you can control a small town is fine, and then you get more people and then you get chaos.

Jess

Exactly. And when those sort of chaotic relationships start to breed, we can kind of start to misinterpret where the boundaries lie between our commitments to a project. When we're faced with a problem, we think more heads equals faster solution, right? But that can only be true when when work is completely interchangeable, when you can just throw a bunch of people onto a project and they can get on with it without without any interaction. My favourite analogy for this is, is I don't know whether on not you used to play Age of Empires?

Adam

Oh, yeah.

Jess

But whenever you used to throw just as many villagers as possible on a build project, they could just get on with it, and it was absolutely fine. But it's just not really the case with software, unfortunately. You need to be able to speak between each other, you need handoffs for different subtasks, and it's actually been proven that the more people that you throw on a software team, the later the project becomes, so you not only end up compounding things in terms of time, but then that puts everyone under pressure and they're pressured to deliver more over their agreed aims.

Hugh

It sounds a bit like a sort of personnel problem, like we're hardwired to devolve to scope creep, if we're not guarding against that.

Jess

Definitely, I mean, we want to, we don't really want to meet the aims, we want to be able to like exceed what we're going for in our team.

Adam

So one of the one of the best kind of illustrations of the principles behind scope creep, for me, is, when somebody leaves a business, often in the interim, before a replacement is hired, one of their colleagues will end up assuming some or all of their responsibilities in the kind of in the interim. And in a lot of cases, that just becomes the norm, you know, the hiring process goes on, maybe it's a couple of months, or, you know, even longer before a replacement for that person who's left can be hired. And over the course of that period, those kind of temporarily assumed responsibilities, just become part of that person's day job. And that is, I think, a really good illustration of how easy it is for scope creep to creep in, if you like, to, basically, anything, and it's especially the case in software projects, I think, particularly when you have large numbers of business stakeholders who aren't actively involved in the development process, and don't necessarily understand that, you know, when they say like, oh, maybe we could add such and such, you know, capability, or, you know, maybe it would be good if it could also cover, you know, this business area, or geographic or whatever, isn't just a case of tweaking a couple of variables.

Jess

That's the version of scope creep, I've become the most, that I've had the most personal experience of, and it's really easy for it to happen, because as soon as it's just one more thing, things can spiral out of control. And the just one more thing rule doesn't usually end up eating into the main sort of time that you frame for your project, for the thing that you're developing, it falls into the sort of, into the the time you've allocated to either cut down on technical debt, or the time you've allocated to perfect your overall process, you know, that 20% time that we talk about in DevOps. I've seen a lot of, a lot of companies recently, they've been organising their teams differently to stop scope creep. And I think this is kind of a bit of a nod to Conway's Law, which is the idea that - have you heard of Conway's Law?

Adam

I have not.

Jess

Okay. It's this idea that we produce, the things that we produce are indelibly impacted by the way that we're organised. Anyway, it was studied by Professor Conway. And he found that he tasked two teams to produce two different compilers, and five people working on one project, I think, a COBOL project and three people working on an ALGOL project. And it turns out the finished state compilers each had three or five stages, depending on how many people were working on that product. So it shows that, we see this all the time in products that we use, usually, the way that we're organised if you have, if you have security experts on your team, you usually work in a different way. And it's can be, it's I mean, it's often to our detriment, because we often end up working with the people that we have, you know, the skills crisis, you're kind of, you have as many people as you can, you can get your hands on. But by kind of building teams really specifically and organising people into different squads and giving squads real specific remits on how, which work they're allowed to factor into their daily processes, you kind of are able to sort of clamp down on scope creep a little bit.

Hugh

What is it about scope creep, that makes it damaging?

Jess

It's really damaging for a few problems, it can really, it infiltrates the way that your, your order of processes, so it can circumvent your established way of doing things, which means that it just interrupts your team from being autonomous. If you if people know where their information is coming from, then they know where to direct their efforts. And as soon as things change, it throws... if this is different, why, is this potentially different? You start to distrust the entire system. There's like four main models that people can use to like measure organisational health, and one of them is deployment frequency and you only really, you're only really going to deploy if you're really like confident in what you're deploying, if you're confident in your product. So it shakes your confidence in what you're doing, but also shakes your confidence in the overall way of doing things. So there's the idea of a microservices architecture, it only works because you're designing for things that can fit into your head, that's what the author of Team Topologies said, you've got to design for software that can fit inside your head. So as soon as things start to become complicated by one or two degrees, quickly gets amplified, it also gets amplified because it's, you're communicating a different thing through many different people. And it's got the, you've got the added risk factor in there, right?

Adam

Yeah, absolutely. I think that one about designing software that can fit in people's heads, I think is super important. Because when you are dealing with particularly kind of business systems, as soon as you start needing complex flowcharts, and diagrams, to explain kind of how your app works, or how your software works, and what it does, it becomes much much easier for bits to fall through the cracks. And to end up with this Kafkaesque nightmare of code where, you know, there's 18 different modules, no one's entirely sure which specific functionalities each one governs. And then kind of that's, that's the stage where, you know, when one element breaks, the entire thing has to be forensically broken down and troubleshot to find out, you know, what's wrong, and how to fix it, which - and again, it goes back to the question of technical debt, I think - the larger you allow scope creep to become the greater risk it introduces of unfeasible technical debt build up.

Jess

But we're talking about scope creep like it's, I mean, it is, it is a bad thing in a lot of instances, but you kind of brought up the the idea of incidents there and scope creep can be really powerful from an incidents perspective. I mean, if we're looking at game days and chaos engineering, the idea of creating incident days where things go wrong in a way that's outside of your scope to correct, is a fascinating way of finding out just how things fit together. So I think that's like a really good way that scope creep can be used to, to kind of strengthen your organisation.

Hugh

That's a really interesting confluence, because scope creep coming out of a place of people being worried about making errors and you know, we've got to account for everything and then, but the positive side of it being creating errors and creating chaos that could be beneficial, rather than just negative.

Adam

Are there any other benefits to kind of permitting a certain amount of scope creep within your kind of development environment or project management environment?

Jess

I think that the main one seems to be stress testing your organisation, the main thing seems to be that if you permit a little bit of scope creep, scope creep, you're able to see whether or not people can, your engineers can rise to the challenge a little bit. But the only issue is, it's it's such a good idea on paper, but it's just often really, it's often carried out in bad faith. We're not in the situation yet where by rule, organisations tend to vouch for their people. And when that is the standard, then sure, we can use scope creep to stretch ourselves and to challenge ourselves. If you permit it within a set period of time, I mean, for instance, like hackathons are all scope creep, they're like a scope creep challenge to see how far you can take the bounds of something.

Adam

So we've talked about some of the ways that you can, if you like, deliberately introduce certain elements of scope creep, or a certain level of scope creep into your organisation, things like hackathons and chaos engineering, and all that kind of stuff. What are some of the main ways that scope creep comes into projects unintentionally, do you think?

Jess

So okay, so scope creep always comes into projects unintended and unintentionally when organisations are either heavily weighted towards the revenue side of the org and that happens all the time, right, because salespeople are speaking to customers all of the time, so they know exactly what they want, but also salespeople are wired in a way that they work in a very explicit manner, which is excellent, but also in very immediate terms. So hearing about a potential feature can sometimes become the difference between a deal happening and a deal not happening. And then sudden, suddenly you're, you're running towards a product with really disparate aims and no main strategy, that's often when you start to drift away from the idea, or the idea of your customer profile, right? You start designing for this person that doesn't exist, it's like the Doctor Who problem; you're demographically kind of designing something for an audience that doesn't quite... I mean, is, do people actually use that feature? And is that just this small organisation over here? That's why positioning the product team and making sure that the product and product management team is able to be essentially a bit of a fortress against these customer success aims is, is really, really, really important.

Adam

Yeah, and one of the growing categories that we've seen over the last several years really growing in prominence, is that product owner, product manager role, which is essentially kind of geared entirely around around that, around nailing down, right, what is it that we need this particular product within our organisation to do? What aims is it meeting? What, you know, what is the roadmap for developing it? And where do we need to essentially cut off development and say, No, that is actually not necessary for this, or not feasible.

Jess

Yes. One of the, one of the really encouraging things we're seeing at the moment is that a lot of - I've seen this loads on Twitter - it's a lot of senior engineers are taking, their like developing this sort of sense, this sort of this reflex to understand Wait a second, is this, does, do do we actually need to build this? I remember seeing someone recently talk about how being a senior engineer is essentially having endless conversations about why you shouldn't make something.

Adam

Yeah, exactly. It's it's the whole, you know, you were too busy thinking about whether or not you could, you didn't stop to think about whether or not you should, from famous DevOps pioneer, Dr. Ian Malcolm. Like, yeah, I think, yeah, I think that's a really, really shrewd observation, a lot of a lot of being a senior technologist in general, and a senior developer in particular, is knowing when not to do something.

Jess

And it can often be really counterintuitive, right? Because if you are, if you're an engineer on a team, and you really want to step up into into, you really want to develop your career, you often need to find your niche and able to, to be able to kind of carve out your unique path, you know, you want to be the change management person, you want to be the, you want to be the person that understands feature flagging. For you to be able to do that you need to be able to carve yourself out as a specialist. But for the general overall team to thrive, you need to still be able to be like a generalist, essentially, like you need to be able to build a team where you can hedge your bets on their knowledge, and you can bet, okay, I believe that this team will be able to encounter this area of development that they haven't had any experience in because of these sort of things that they've experienced in the past. So it's very easy for you to take scope creep and see it as an opportunity to develop not only your career, but the kind of the, the reputation of your team. But that's that's I mean, that's, that's ego speaking. That's ego speaking, and that's only going to be successful for the short term. It's not a sustainable way forward.

Adam

Yeah, everyone wants to, to stand out and to be the, you know, the team or the staff member that goes Yep, we can, we can make that happen. We can deliver on these things that that you want to do. And also it's you know, in a lot of businesses, particularly ones where the development team aren't kind of, aren't necessarily the central pillar of the business, it can be hard to say, to say no to business stakeholders who go right we want such and such functionality or you know, we want this project to deal with X, Y and Z. It takes a lot of soft skills, ability and confidence to be able to push back on that and go, Well, no, actually, this isn't going to be a feasible project or this isn't going to deliver on what you want it to do for these reasons, and to have that be, you know, received and understood and acted on by, by business stakeholders. It's a big ask.

Hugh

That's a universal problem really, isn't it, just not in dev work but in all sorts of, you know, every areas of business, it's a problem we deal with.

Jess

Yeah, it really is. I mean, it's it's with with, you know, within the within the context of a development team, you can, you can end up wasting thousands of pounds, hours, it can become an expensive headache, but we're taught to say yes, as a way of falling in with the, the people that we want to see be seen as allies to. And not to blow this up too much. I mean, I've become a bit obsessed with, with with tech failures recently, and I find it amazing how these things flow , like Juicero or even Theranos.

Adam

Beautiful.

Jess

It's incredible stuff. But people say yes, when they really should say no, and you don't really want to be seen as a downer, you don't really want to be seen as a blocker. But in essence, you need to be able to all unite around the same idea. And that sometimes requires you saying no, right?

Adam

Yeah, I mean, it's it's organisational peer pressure.

Hugh

I think it's still quite hard, though, because it's not like we all work in institutions where that isn't frowned upon. And, you know, you have to, you have to have higher ups who are willing to hear no. And I mean, and whether that's true or not, there's there's often a feeling that you can't say No. And, you know, you need to combat that.

Adam

Yeah, I think a lot of that, as well feeds back into, you know, company culture and company environment, and how, how much are you listening to your project teams and your dev teams, and, you know, just your kind of junior staff in general, you know, because if you're, if as if, as a business, your leadership are not willing to actually sit down and listen, properly listen, to what your junior staff members are telling you, then yeah, that's gonna be a problem.

Jess

These sorts of bad decisions, they thrive in silence, they, the the best people on our team are usually the ones that really want to avoid context switching, they usually are the most adverse to communication and communication could sometimes be the most intense, like time intensive training activity that we need to, that they need to engage in. So I remember reading a study recently was talking about how collective intelligence in teams, you know, the ability to be able to do something well, on a collective scale, it thrives in teams that often spend fewer words communicating with each other, the longer they spend with each other, the more they seem to understand their nonverbal cues, and whether or not that's virtually or in person, they start to understand their innate cues around something. So this is why observability has become such an important part of preventing things going wrong.

Adam

So let's quickly address the differences between scope creep and feature creep. Because both are brought up fairly often in, you know, the development world. But there there is a distinction there that's that's worth noting. So what's what's the kind of the key difference for you?

Jess

Scope creep can happen to anything; scope creep can happen to a product, a project, it can happen, it's that additional few pieces of functionality, whereas feature creep, feature creep happens when a feature diverts from its original intended purpose. It becomes, it becomes a, like, an improv game; a 'yes and' situation.

Adam

That's a marvellous way of looking at it.

Hugh

I mean, in this case, is feature creep a sort of a species of scope creep?

Jess

Yeah, scope creep is more the, is the overarching term. And feature creep is, is the additional overload og features again and again, in a sort of a continuous stream. So it's...

Adam

It's like a subset of scope creep. I think it's the the phenomenon that non developers are probably most likely to be familiar with, in in like a software context. You know, we're all familiar with social media platforms are the ones that springs to mind first for me in terms of feature creep. We're all familiar with, you know, companies like Facebook and Twitter, bolting seemingly endless additional capabilities onto their core platforms, most of which, you know, no one really asked for. And it just, it seems in a lot of cases like adding new functionality for the sake of having new functionality rather than for what it actually offers in terms of benefits.

Hugh

Twitter stories and LinkedIn stories.

Adam

Exactly. LinkedIn stories - that much demanded and widely used feature.

Hugh

I'm sat at my desk, I love working, it's great.

Adam

Definitely isn't going to end up being quietly dropped in 18 months time.

Hugh

That's interesting, because feature creep does, as you say, kind of like that's quite easy to conceive. But scope creep is kind of, the more you think about it, the more vast it feels, and the more different ways it can come in. And different ways that it can cause chaos.

Jess

Feature creep is usually influenced directly by commercial pressures. So I don't know whether or not you've noticed this, whether or not, I think it's more of a sort of a thing if you're, if you use an iPhone, but your notifications are increasingly being used as sales, collateral or marketing, marketing collateral, and that's not necessarily what they're for. And you can kind of see that this has been led by the top down, even when you could jump into the settings feature, you can see sometimes a try this feature for x pounds per month. And that's it's that's the that's where you kind of see the commercial pressures of elsewhere in the business being leaked down into the into places where we shouldn't frankly be touching, and the highest person's opinion is getting weight over the others.

Adam

Yeah, absolutely.

Hugh

I mean, talking about saying no, try saying no to, to the higher ups about monetizing things.

Jess

Exactly. Because if you say no to monetizing, you're saying no to the success of the product overall, aren't you? Well, I mean, don't you want a job, I mean, this kind of... But that's, this is the thing. So some things need to stay local, you know, if we, if we argue for everything on a global scale, then it's really easy to find yourselves quickly going from discussing the use of a certain tech stack to whether or not we should be paid x pounds per month or what or whatever it's it's, we kind of need to keep things at a certain level to be able to argue for their categorical importance or unimportance.

Adam

So we've touched on this a little bit already. But let's talk a little bit about some of the methods and strategies that organisations can use to prevent scope creep from infiltrating their projects.

Jess

Sure, so scope creep, it's, it's a people problem, it's an organisational problem. So we need to look at how we're, how decisions are being made, and standardise that across the business. And I know that sounds like such an eye rolling exercise for everyone involved. But every single year, whenever the state of DevOps report is completed, it seems to find uniformly that when orders of interactions are defined, people become more satisfied with the outcome of their decisions. That's not to disrupt the org chart or the hierarchy of the people that you're working with. It's just so the expectations are set up so that people know how to have their voices heard. And people have a direct route to saying, to raising concerns about something, about about saying actually, I think this is going to, this is going to have us drift away from the main thing that we're focusing on. Another really important part is including feedback loops, and closing and making sure they're really fast. One of the merits of of working with production data is that you know what you're, you know that you're grounded in fact, whether we're talking from a sort of testing perspective, or, or just in terms of testing our, testing our theories, there is there is no production like production, we can't stage our decisions, and we can't stage our environments. So we need to keep things grounded in reality, and that means that we get tied to the same version of the truth. So testing in production is a great way to stop scope creep from happening because it prevents us from going down and into these imaginary realities where people will buy this feature that you haven't made yet, that doesn't necessarily exist and that might not even save the company keeping teams small, we've spoken about that. But the squad model I think is really interesting, because the squad model unites a small number of people like say eight people around one problem. And it forces you because because these teams are quite small and because the the the point of those teams is so specific, it causes you to create many more of those teams. And look at enablement teams way faster than you otherwise would. You stop having these big silos of teams and think, oh, gosh, we've now got this many engineers, we need to be able to support the engineers, let's have a supporting engineering team. And it's easy, it gets you away from having those layers, and it starts, it gets you to think about how to support the output a lot sooner. One, one thing that - it's really early on, but it's a project that I'm working with, with a couple of people - is around the idea of making it compulsory for team members to switch roles. A while ago, we saw that DoorDash was really, it was really insistent that everyone throughout their company, no matter whether or not they were on a super high pay packet or working in product, or if they had to have some sort of customer experience hours in in their contract. And I think grounding people in the actual work that you're, the actual kind of aim of the businesses is really important. But it's, it's about what form that takes, right? I mean, is it enough to just talk about your problems, is it enough to just to just have monthly meetings where the technical support team feed back the problems that that that that your users are having? Or do you actually need to be... how much do you need to consume your own cat food, dog foods, animal foods metaphor, for your activities to be productive?

Adam

I think that's a wonderful kind of idea. We were discussing a very similar thing with Ocado's James Donkin a couple of episodes ago about how when Ocado first started, basically, everyone did a couple of shifts in the warehouse doing the, you know, picking and packing and loading orders and everything. And I think that that is really vital, not just for, you know, customer service businesses, but for every, every organisation should have some mechanism for, as you say, grounding, you know, the workforce at large in what the organisation does, and what the priorities of of the organization's should be. When looking at, you know, expansion and, you know, new features and tightening up processes, you know, what are, what are the actual priorities, effectively?

Jess

I recently recently spoke to Patrick Debois, he works over at Snyk, and he's just a really interesting person, it was talking about how the majority of our problems come up when we tried to cram 100% utility into our teams, right? We try to make everything the peak of efficiency, and to meet all of our aims with all of our hours. And that's just not how we work, we need to break down - while we don't need to, we shouldn't be working in silos - we need to break down our time a little bit to make sure that we're sectioning off certain hours for being able to allow feedback to happen and to understand whether or not we're actually meeting the problem that we're trying to solve.

Hugh

Well, sadly, we're out of time for this week's episode. But thanks once again to LaunchDarkly's Jess Cregg for joining us.

Jess

Thanks so much for having me.

Hugh

Pleasure.

Adam

Always a pleasure to have you on the show, Jess.

Hugh

That was really interesting.

Adam

You can find links to everything we've spoken about today in the show notes and even more on our website, itpro.co.uk.

Hugh

Don't forget to subscribe to our social media and YouTube channel for more great content and leave the podcast a rating and a review.

Adam

We'll be back next week with more analysis from the world of IT. But until then, goodbye.

Hugh

When you’re working on a large-scale software development project - especially as part of a sizable enterprise business - it’s easy for the project’s original goals to become muddied over time. As the goalposts shift further and further from their starting point, timelines can slip and the efficiency of the project can be undermined.

This kind of scope creep can be seriously damaging for organisations, but thankfully the software engineering community has been hard at work for years on strategies to help limit its spread within organisations. Returning to the podcast this week to discuss some of these strategies - as well as how scope creep can sometimes be beneficial - is Jess Cregg, developer advocate at LaunchDarkly.

Highlights

“It's really easy for it to happen, because as soon as it's just one more thing, things can spiral out of control. And the ‘just one more thing’ rule doesn't usually end up eating into the main time that you frame for your project, for the thing that you're developing; it falls into the time you've allocated to either cut down on technical debt, or the time you've allocated to perfect your overall process - that 20% time that we talk about in DevOps.”

“Scope creep can be really powerful from an incidents perspective. I mean, if we're looking at game days and chaos engineering, the idea of creating incident days where things go wrong in a way that's outside of your scope to correct, is a fascinating way of finding out just how things fit together. So I think that's a really good way that scope creep can be used to strengthen your organisation.”

“One of the really encouraging things we're seeing at the moment - and I've seen this loads on Twitter - is a lot of senior engineers are developing this sort of sense, this sort of reflex, to understand, ‘Wait a second, do we actually need to build this? I remember seeing someone recently talk about how being a senior engineer is essentially having endless conversations about why you shouldn't make something.”

Read the full transcript here.

Footnotes

• What is a microservices architecture?
• How microservices help Perlego deliver textbook availability
• What is DevOps?
• The driving forces behind DevOps today and how to integrate it into your business
• How to become a DevOps engineer
• IT Pro Panel: Defining DevOps
• Seven skills you need to become a front-end developer
• How to become a developer: A beginner’s guide
• 7 UX skills every designer should have
• Synk to move into cloud security with Fugue acquisition
• The IT Pro Podcast: Picking a programming language
• The IT Pro Podcast: Building the supermarket of the future

Subscribe

• Subscribe to The IT Pro Podcast on Apple Podcasts
• Subscribe to The IT Pro Podcast on Google Podcasts
• Subscribe to The IT Pro Podcast on Spotify
• Subscribe to the IT Pro newsletter
• Subscribe to IT Pro 20/20

London-based cyber security unicorn Synk has announced it is acquiring Fugue, a cloud security compliance firm, for an undisclosed fee.

The deal marks Synk's first move into cloud security where it hopes to capitalise on its increasing market value, which is predicted to be worth $77.5 billion by 2026, according to analysts at MarketsandMarkets.

Fugue, a US company headquartered in Maryland, is best-known for its Unified Policy Engine, which covers systems development lifecycles for areas such as cloud posture and configuration codes. The firm also offers API and command-line tools for developer automation, as well as compliance frameworks.

The company was founded in 2013 by former AWS solutions architect, Josh Stella. It originally offered its tech as an "infrastructure-as-code" service before switching to security. And that original focus on code has proved to be a key part of the acquisition as it suits Synk's focus on devops.

Synk plans to integrate Fugue's suite of services into its Developer Security Platform, which it claims will be the industry's first cloud service posture management (CSPM) solution designed by, and for, developers. As such, Synk customers will be the first to benefit from "unmatched" feedback loops that allow developers to secure codes before deployment and maintain their integrity while running them.

"Welcoming the talented Fugue team as our newest Snykers is a fantastic way to kick off 2022," said Peter McKay, CEO, Snyk. "Together, we'll collectively reimagine what cloud security can and should look like for today's modern DevSecOps teams, ensuring more secure innovation can flourish worldwide."

This is Synk's fifth cloud-based acquisition in the last 18 months, following on from deals for CloudSkiff, FissID, Manifold and DeepCode. The London-based cyber security business was one of IT Pro's startups to watch in 2021 and has managed to scale up rapidly since its inception in 2015.

Container technology has gone from a niche, experimental application and infrastructure tool to a cornerstone of modern IT transformation - but its explosion in popularity has resulted in a corresponding demand for skilled professionals to manage these deployments. Kubernetes certifications are particularly sought after, and holding one is almost a licence to print money for IT professionals.

With the market for Kubernetes skills hotter than ever before, we’re joined this week by OpenUK CEO Amanda Brock to discuss why it’s become so popular, what this says about IT transformation in the UK, and how organisations can make sure they don’t fall into the Kubernetes skills gap.

Highlights

“Everybody is looking at Kubernetes, but they're not looking at it in isolation. What it's done is opened a floodgate in many ways for those who might not understand that they're using open. And the fact that they bring it into organisations, or they're using it via third parties, has opened up the opportunity for more tolerance, acceptance and understanding of open source generally, I think, so I don't think that you can say ‘everybody's using Kubernetes, nobody's using other open source’, I think what you're seeing is people are acknowledging and understanding their use of Kubernetes. And that's allowing their internal processes, their policies, their governance, to improve to facilitate the adoption of other open source.”

“With Kubernetes certification, there's a lot of structuring around it, and it's quite different. And I think there's that difference that's making it so popular alongside the scale of adoption. And what I mean by that is that the certification isn't your usual multiple plot choice test. You can't just tick box A the whole way through and find that you've passed. You actually have to answer.”

“I think that we can all accept that there are skills gaps. I was on a podcast a few months ago with OVH, ATOS; all of them were looking to recruit in the UK, with these skills. Now, I think the skills gap, we just have to accept is there. Obviously it's great to train people. And it would be ideal to bring people in with those skills. But there's a shortage. So you're balancing those two. And I think that we will have to retrain people, or encourage people to train in this space. I suspect that that's where the work that's been done in the last few years will really come into its own with CNCF, the Cloud Native Foundation, which has got this whole ecosystem around Kubernetes. So it doesn't just offer Kubernetes training. It offers training in associated products as well.”

Read the full transcript here.

Footnotes

• What is Kubernetes?
• Rancher Labs unveils new Kubernetes certification programme
• How the tech industry can retain talent
• Getting started with Kubernetes
• Is the UK government's plan for digital upskilling on track?
• Government launches 400 free qualifications to help with economic recovery
• What are your tech skills worth?
• Education is needed to end apprenticeship stigma, says BCS
• The lost year: How 2020 tech graduates and apprentices are coping with the coronavirus pandemic
• Is a degree still necessary to get ahead in IT?
• OpenUK Entrepreneur in Residence and Founders Forum Initial Findings November 2021
• The IT Pro Podcast: The teenager who started a networking firm
• The driving forces behind DevOps today and how to integrate it into your business
• IT Pro Panel: Defining DevOps
• 11 best python courses
• How to become a Python developer
• Python founder Guido van Rossum joins Microsoft
• UK's growing skills gap could stall COVID recovery
• Linux on Mars!
• The IT Pro Podcast: Picking a programming language
• Why SMBs are fishing in the wrong technology talent pools
• The 2021 Open Source Jobs Report
• How apprenticeships are fuelling the future of UK tech
• IT Pro Panel: Do we still need certifications?
• IT training courses and where to find them
• Learn to code online with these six websites
• How to confidently change careers to IT
• How to become a developer: A beginner’s guide
• The best cloud computing jobs and how to land one
• IBM job ad calls for 12-years of experience with six-year-old Kubernetes
• How to become a DevOps engineer
• The IT Pro Podcast: Why the tech skills gap doesn’t exist
• CNCF.io
• Open UK Kid’s Camp
• The IT Pro Podcast: Bringing space tech down to earth

Subscribe

• Subscribe to The IT Pro Podcast on Apple Podcasts
• Subscribe to The IT Pro Podcast on Google Podcasts
• Subscribe to The IT Pro Podcast on Spotify
• Subscribe to the IT Pro newsletter
• Subscribe to IT Pro 20/20

This automatically-generated transcript is taken from the IT Pro Podcast episode ‘Everyone needs Kubernetes certs’. To listen to the full episode, click here. We apologise for any errors.

Adam

Hi, I'm Adam Shepherd.

Sabina

And I'm Sabina Weston.

Adam

And you're listening to the IT Pro Podcast, where this week we're digging into the recent demand for Kubernetes skills.

Sabina

According to the Linux Foundation's 2021 Open Source Jobs Report, demand for IT professionals with Kubernetes certifications has shot up by more than 450% since 2019, with almost 90% of hiring managers reporting that they are prioritising the recruitment of certified professionals.

Adam

But what does this explosion say about the state of enterprise open source adoption, and is certification the right route for organisations seeking to plug skills gaps? This week, we're joined by Amanda Brock, CEO of OpenUK, to find out more about why Kubernetes is big business. Amanda, thanks for joining us.

Amanda

Thanks very much for having me along today guys.

Sabina

Okay, so, Amanda, how does the level of interest in Kubernetes certification compare to the demand for other open source technologies?

Amanda

That's a very good question. I don't think we have the data on some of the other open source technologies in the way that we have it for Kubernetes, simply because Kubernetes has been such an explosion. And it's so essential today to business. I see particularly in the UK that we have a lot of leadership and growth around Kubernetes, in particular. So it seems to be that it's pretty much ahead.

Adam

And I, I would argue the fact that we have so much more data on Kubernetes, in particular, kind of demonstrates the overwhelmingly dominant position that it has kind of among other open source technologies.

Amanda

No, no, no, no, I think you're comparing apples with pears there. So what you're comparing Kubernetes against is other cloud technologies and containerisation technologies, really, whereas other open source may be organised in a slightly different way. Kubernetes is super structured, brilliant governance through the Linux Foundation who do a lot of reporting, you might not see that in every other area of open source. So I'm not sure that the comparator should be other open source. I think the real comparator should be other containerisation and cloud technologies.

Adam

So platforms like Docker, for example, which, you know, enjoyed a lot of early success, I think it's fair to say, but was just absolutely eclipsed by Kubernetes over the last several years. But anecdotally, when you're when you're talking to organisations in the field, to, you know, member organisations and partners and people like that - what is the kind of the ratio of organisations that are looking at Kubernetes versus organisations that are looking at other kind of open source infrastructure platforms; not necessarily kind of containerisation platforms, but other open source things?

Amanda

I would say they're at equilibrium. So when we talk to people, they're running Kubernetes, everybody is looking at Kubernetes, but they're not looking at it in isolation. What it's done is opened a floodgate in many ways for those who might not understand that they're using open. And the fact that they bring it into organisations, or they're using it via third parties, has opened up the opportunity for more tolerance, acceptance and understanding of open source generally, I think, so I don't think that you can see, you know, everybody's using Kubernetes, nobody's using other open source, I think what you're seeing is people are acknowledging and understanding their use of Kubernetes. And that's allowing their internal processes, their policies, their governance, to improve to facilitate the adoption of other open source.

Adam

Cause of course, historically, open source has had a bit of a mixed relationship with, I think enterprises in particular, where there's a huge appetite for it in the in the IT team, because, you know, there are just so many useful tools and utilities and kind of back end things, bits and pieces that kind of rely on open source. But on a high level, there's quite often a bit of trepidation. I think it's fair to say, about deploying open source at a kind of policy level, if you like.

Amanda

Yeah. And you obviously know my background. I don't know if people listening will, I was a lawyer for 25 years, I stopped being one a couple of years ago, I spent five years from the beginning of 2008 in a company called Canonical, one of the biggest open source companies. Definitely biggest in the UK, at least when I was working there, and I set up and ran the legal team. So for me, my conversations day on day on day, particularly as I was the escalation point in all the commercial contracts, was explaining risk; getting people to understand open source, getting them to understand governance. Now, I left there at the end of 2012, almost 10 years ago. And what's happened in that 10 year period has been that there is a massive, massive shift. And I don't know how much even you two will understand what's caused that. Some of it is definitely Kubernetes in the cloud. Some of it is around Git and the the massive growth of these distributed repos where the DevOps community, the internal engineers, can now go and get the software they want, they don't have to go through procurement, they don't have to go through legal, if your organisation is doing it right, you will have policies and processes that govern how your open source is brought in. And you'll know what's going on in your organisation. But your developers are going to be doing this and using it, whether you like it or not, and then don't have to come back to legal and procurement until you get to the stage of money changing hands. And that tends to be for a subscription or a support contract or some sort of implementation that you might need. So actually, the power shift, and the risk shift has really come from the internal engineering CIO, CTO community.

Sabina

Interesting.

Amanda

It's massively different from a decade ago. And what that's done is cause you know, we did some survey outputs last year or earlier this year. And what you see is 89% of companies in the UK actually running open source and knowing they're running it. And I think the trajectory, we're on now is just so rapid, and so high, that there's no way of avoiding it. So you're right, Adam, Kubernetes has been a big opener up of that. But what we're seeing is Kubernetes taking over from proprietary cloud tools and container orchestration tools, and facilitating greater adoption of open source generally.

Adam

Yeah, cause, of course, Kubernetes is very big in the kind of infrastructure and kind of DevOps side of business. But there's also in recent years been increasing adoption of things like Python, for data science and data modelling, which has also kind of exploded in the last couple of years.

Amanda

Absolutely. And I think if you look at our kids course, which is part of the OpenUK Kids Camp, I think it's lesson five, where we talk about the Python Foundation and Guido van Rossum who created it. We have a lovely fact. And the fact is that there are more children in the UK learning Python than French. Now that might be a post-Brexit thing. But there we go. I knew you'd like that! And I was at something. I was at an event this week at the House of Lords where Chris Philp, the new minister for digital spoke, and he was really emphasising the need for AI and data scientists. And I think that's going to be massive on the government's agenda next year. And of course, as we know, most of the languages that are sitting behind that are open. So we're only going to see more push into that education and skills for data scientists.

Sabina

Why have demand for Kubernetes certification in particular exploded so dramatically?

Amanda

I think it's a couple of reasons. So when you look at Kubernetes, you see something that is infrastructure that's really important, digital transformation has gone at the scale it's gone. Kubernetes allows you to have interoperability. In theory, it removes vendor lock in because it's used universally, it's a sort of de facto standard in the cloud. So and then containerisation, of course, generally, not just in cloud. So what you then see is a need for those skills. Now those skills are in demand and in short supply. So there's a mixture of hiring people from the marketplace, and also where those can't be found, training your existing staff. And it's one of the things that we've identified as a straight benefit of open source is the skills development on the job. With Kubernetes certification, there's a lot of structuring around it, and it's quite different. And I think there's that difference that's making it so popular alongside the scale of adoption. And what I mean by that is that the certification isn't your usual multiple plot choice test, right? You can't just tick box A the whole way through and find that you've passed. You actually have to answer; do you know, you're laughing, but the American Bar, American lawyers qualify in that way. That the bar exam has multiple choice as well. But what you've seen here is a shift and you actually have to be able to write the answers. So you either know it, you either understand it or you don't. And I think that's been a big part of the certification. I also happen to know that they did something called the massively open online course for Kubernetes. And that had 165,000 enrollments in 2020. It's massive.

Adam

So Kubernetes certification is obviously something that's very attractive to learners and developers who want to broaden their skill set. But what I think is really interesting is that there has been a corresponding demand from employers and from businesses who want these certifications; more so, in fact. The Open Source Jobs Report stat of 450% growth is crazy, crazy high. And that's really interesting, because certifications are often something particularly in you know, emerging technologies like Kubernetes - you know, depending on whether or not you think Kubernetes is still emerging - it's, it's often something that's looked on as a nice to have, rather than something that is being proactively sought out and fought over by organisations.

Amanda

It's interesting, I think it's way beyond a nice to have. We have at OpenUK an entrepreneur in residence, chap called Matt Barker, who I worked with at Canonical. And he set up one of the early stage Kubernetes companies, one of the first in the world. And as I'm talking to you, I'm quickly flicking through his profile on LinkedIn. So he set it up in 2015. So you know, that's what, seven years ago, he's created, it sold it, now running it as part of another enterprise. And, Matt, I'm going to give you a sneak preview, this will probably have been released before this podcast comes out. But Matt's about to share his entrepreneur in residence report. And he steals a line, I can't remember who originally gave this quote, I know Tony Blair used it but he's not the originator. And Matt talks about education, education, education. And he talks about educating our young people, he talks about reeducating our business people, and educating our government in that report. And you know, after all the work he and the Founders Forum have done, they come back to that. Now, Matt's also been involved at Bristol University in helping them create a Kubernetes course for students. We'll be running a founders course from 21st of January, helping UK founders broaden out their skills around things like product development, revenue models, that kind of practical open source stuff. But we're also talking to universities at scale, and we're talking to a number of organisations about building an apprenticeship module. And it will be inevitable that Kubernetes will be part of that. There is no way around it at this stage.

Sabina

That's amazing, that's really good. Can I ask what like, how do you see an apprenticeship programme with Kubernetes involved? Are there any sort of like specifics?

Amanda

Yeah, we've been very good at teaching people. So we've been very good at sort of teaching go and code Python, but not teaching them why they do things in a certain way, or how the, the openness of that's originated and the governance and the practicalities. So what we're trying to do is merge both the functional practical training with the understanding, and I think that understanding really empowers people and lifts them up to a higher level.

Sabina

Definitely, yeah.

Amanda

We're very keen on that.

Adam

So is this massive demand for Kubernetes from the business community, is that being driven by a skills gap, do you think?

Amanda

I think there is a skills gap and I think it's a universal skills gap caused by the scale of digitalisation that has really been put under a magnifying lens through the pandemic. You know, if you go to McKinsey or Deloitte, they'll show you the massive pace of adoption and digitalisation in the last two years, I used to refer to them as reluctant technologist because there are all these companies, you know, I mean, basically, they're either producing their products, distributing it, or having it consumed via digital. And that's been a shift over the last decade. For those of us who worked in it, it was very confusing that it wasn't already happening. But you can understand how industries had to shift to that. And really, I think there's a bigger picture here, which is that we're also going to see the public sector and governments moving to it. You know, we're going to see UK utility shifting over to that. I think Laura Sandys who's running the digitalisation task force for the energy sector, calls it the plumbing. Our infrastructure, our roads and bridges; the future of that is digital in the UK and beyond. And we're going to see, I think, real leadership in the UK around that transformation. One of the things that we've always done well as infrastructure, whether it's digital or not, and we have amazing engineering communities, which I think is why the UK has a leadership role already around CNCF, around cloud native. You see people I've mentioned, Matt Barker, we've got Alexis Richardson. People like Liz Rice who's the CTO at CNCF, Cheryl Hung, Katie Gamanji, there's a whole crowd of those folk, based in the UK. And I suspect that that will really help us in building those skills now that people are understanding the gap.

Sabina

And what is the potential impact of this skills gap?

Amanda

There's obviously an immediate problem with the lack of those skills and the need to get people trained quickly and effectively. I think that governments across the globe are recognising that. I don't know, I don't really want to get into the politics of it all. But I actually think that the UK also is really beginning to come to terms with the fact that we need to equip people digitally and quickly. We've got to implement it. But I think there's an understanding in the public and private sector now, about that.

Adam

Do organisations risk kind of falling behind competitors if they don't have Kubernetes skills, specifically?

Amanda

I think it's a no brainer, you're going to have to have it because it has become such a de facto standard to avoid lock-in. And, you know, the theory is it's entirely interoperable. I know, sometimes that isn't quite the case with cloud companies. But I think you will see that. And then we've got these conversations going on across Europe, across the UK, in fact, across the globe, again, about sovereignty. And everybody is looking at managing their own cloud, about managing their infrastructure, building out that infrastructure locally, which means that there's a, an increased desire locally to have it, as well as just companies in their day to day digitalised transformation. So yeah, I think we have to have it and we have to have it as soon as possible.

Sabina

Should organisations, according to you, consider retraining existing technical staff to fill the supposes skills gaps?

Amanda

So I think that we can all accept that there are skills gaps, I was on a podcast a few months ago with OVH, ATOS; all of them were looking to recruit in the UK, with these skills. Now, I think the skills gap, we just have to accept is there. Obviously it's great to train people. And it would be ideal to bring people in with those skills. But there's a shortage. So you're balancing those two. And I think that we will have to retrain people, or encourage people to train in this space. I suspect that that's where the work that's been done in the last few years will really come into its own with CNCF, the Cloud Native Foundation, which has got this whole ecosystem around Kubernetes. So it doesn't just offer Kubernetes training. It offers training in associated products as well.

Adam

What are some of these associated products? Because obviously, one of the the advantages of Kubernetes is that it's part of a wider ecosystem that is kind of very complementary. So stuff like serverless, serverless computing, dovetails very naturally with it. Microservices architectures, of course, is a big part of Kubernetes. What are some of the kind of inroads, if you like, that can lead people into Kubernetes?

Amanda

So rather than trying to list things out, I would send anybody to the CNCF.io site. There's a chap sadly, he died last year and not as a consequence of the pandemic, who did the most amazing work in creating our cloud native landscape. And there is actually a card that you can see on there, which sets out the whole landscape and breaks it up into categories. And then within those categories, shows you products and what's open and what open interfaces with. And I think that's been used in by Linux Foundation since Dan Kohn created it, across other areas of their work, but it was a first and I remember seeing it maybe 2017,18 at one of the KubeCon conferences, and I would send anybody to that to get themselves a picture of the landscape. The other thing that they have that you may not have seen is Phippy. Do you know who Phippy is?

Adam

I do not.

Amanda

Ahhh, Phippy's a giraffe. Phippy is a little giraffe. And again, we've shared Phippy and friends across our kids camp. Cloud Native have these amazing books for very young children explaining containerisation orchestration, microservices, the whole infrastructure, for I would say late primary school, early high school kids. And there's a whole series of characters around it steering the Kubernetes ship.

Adam

Absolutely. We'll put a link to that in the show notes.

Sabina

Yeah, it'll be amazing. So it's a giraffe, yeah?

Amanda

It's a giraffe.

Sabina

That's so cute.

Adam

That is adorable.

Amanda

She is. Yeah, well, she's a bunch of friends, there's little zebras and all sorts in there. It is really wonderful the way it's been done. And I think it was a community initiative. But what what they've done, which in some ways I tried to emulate is they've created really accessible user interfaces. So it makes it something when you come to that learning, and that skills development, it's less intimidating than some areas.

Sabina

Yeah. Yeah.

Adam

So speaking of that side of things, you've mentioned the kids camps, and the fact that you're teaching Python in them. Firstly, are we teaching kids enough about these kinds of platforms, about Kubernetes, and containerisation, and all of the kind of related ecosystem stuff? And also, how long do you think it is going to be before we start seeing Kubernetes and containerisation as a foundational, you know, everyday part of hobbyist citizen development?

Amanda

Really interesting. So I'm hoping that you will see some of that via us next year. We talk in our kids course. So I'll take a moment. Yeah, it's not a, an advert as such, but I'll take a moment to talk about what we did. So we did a first course last year, we didn't intend to do it, we intended to run a competition, we pivoted because of the pandemic, we created a course to give to kids who were stuck at home during lockdown. We realised it was probably something that other children would enjoy, other young people would enjoy. So we did a glove kit giveaway of the Mini.Mu glove. And we did over 3,000 of those and kids joined a summer camp. And the Mini.Mu glove is inspired by MiMU; Imogen Heap, the singer worked with a lady called Helen Leigh to create this glove. And it was retailing for sort of £30-£50. So we gave away about 3,000 of those last year. This year, well this year was even better. So this year, I got the bit between my teeth. And we started understanding that we're building a course. And what we did was we took the Open Source definition, which has got 10 elements to it, we broke that down. We added coding skills where we have I think, five make code which is a block code lesson that goes with the glove and the Micro:bit. Then we did four Python lessons and one JavaScript. And around those we explained the Open Source Definition and sustainability. And we start to teach the sustainable development goals and interface the two, we were super lucky, our friends at NASA created a column across those for us, the ODI creating something on data; I know NASA, gets me every time as well.

Sabina

It's very, very good to have friends in NASA always.

Adam

Yeah. Friends in high places, you might say.

Amanda

Yeah, well, lots of open source on Mars. So it's one of those things that allowed us to really engage with kids. And unfortunately, the MiMU, Mini.Mu glove was withdrawn, and I'd raised the money for them. So we went out and built our own glove with MiMU's help. And we've just distributed 5,000 of those to allow kids to do this course. So you can go to openuk.uk/openkidscamp, we're on lesson seven on Monday, that course teaches data centres, a little bit about cloud, containerisation, it starts to contextualise some of this stuff, and for us, it's key stage three. So age 11 to 15. By next year, we will have an apprenticeship knowledge module; as with everything we do, it'll be Creative Commons, people will be able to take it and use it wherever they want to. But I'm hoping that that will also lead into either a GCSE or a T-level or a Scottish higher, something like that, that we can put out and just share with people and let them start to learn and be educated and educate themselves around it.

Adam

That's really interesting.

Amanda

If we wait, I mean, the current curriculum, I'm told and I, you know, I'm not an educational specialist, but the current curriculum is from a few years ago and changing that in a formal way takes lots of parties to agree. And it will take time, and I don't think on digital skills that we have that time. So we've just gone ahead and done what we can.

Adam

Absolutely, it's the problem that you see so often with kind of government tech policy. Policy takes so long to agree and finalise that by the time you finalised it, the technology that it's dealing with has moved on.

Sabina

So we've spoken about how this can be used with, with younger people, especially specifically children. But let's talk about people who are on the job market right now and are applying for jobs; do you think that that organisations should be looking for certifications in particular, when trying to fill these gaps or does that risk excluding potential talent?

Amanda

I hope it doesn't risk excluding potential talent. I don't believe that it does. Because the ability to go and get the certification is not massively expensive. It's something that's freely and widely available. What I've experienced is the and this influences everything I do around education in coding. A number of people I know, good friends of mine have not gone through the university system, they've not gone through that traditional tertiary education system. And they've, if you measure success in money, which I don't, they've made a lot more money than me.

Sabina

So did they, did they go through like apprenticeship programmes, or...

Amanda

Sadly not, because most of them didn't have the facility and I think apprenticeships programmes, the right apprenticeship programmes, I think really would help. For many of them, they didn't fit within the university structures, they just didn't appeal to them, or they dropped out, right, and they've gone off, and they're largely self taught. And I think for a lot of engineers, that's the way they've come through. And the more guidance we can give, and the more that that's simple, practical, accessible, the better. So having something like a certification rather than requiring somebody to go off and commit to years of college, creating something that they can do while they're on the job, I think is really important. And it makes it much more accessible.

Adam

Yeah, absolutely.

Amanda

What we also have to be doing is encouraging a more diverse community into that. And that still comes back to schools. And if it's not the computer science teachers, it's the careers advisors, encouraging, you know, an ethnically and gender diverse group into this space.

Adam

Yeah, absolutely. And that whole self taught programmer thing, I think, is becoming a lot more, as you say, a lot more accessible and a lot more common. I would encourage listeners to check out the podcast episode that we did with friend of the podcast, Jessica Cregg, who did that exact thing of, of retraining; not even retraining, but kind of teaching herself programming, and she is now firmly entrenched in a tech role at LaunchDarkly. So then, you've mentioned that these certifications are kind of fairly, fairly widely available, and the courses and the course materials are very open and accessible. Would you encourage developers to have a go and to get stuck into learning Kubernetes if they're interested?

Amanda

I think many of the community that I work with already has, to be quite honest. And they certainly are advocating it across the board. I definitely recommend it. I think if you look, as I've said many times in this interview, if you look to the UK, our engineering community is really really good at infrastructure. We've got an incredibly strong Kubernetes community, incredibly strong cloud native community. They're very welcoming Cheryl Hung, who you probably know runs meetups in London. There's other meetups. Lewis [Denham-Parry] is running them in Wales, you know, there's folk doing this all across the UK, where you can go and dip a toe in the water if you want to sort of chat to people about it. If you can't find them, come back through me. You'll find OpenUK or me on Twitter, DM us and we can put you in the right direction.

Adam

Well, unfortunately, I'm afraid that's all we have time for this week. But thank you once again to OpenUK's Amanda Brock for joining us.

Amanda

Thanks very much, Adam.

Sabina

You can find links to all of the topics we spoken about today in the show notes, and even more on our website, itpro.co.uk.

Adam

You can also follow us on social media and if you like leave us a rating and a review. We'll be back next week with more analysis from the world of IT but until then, goodbye.

Sabina

Bye.

It’s no secret that security and development teams do not see eye-to-eye. But is that the end of the story? Absolutely not.

The value that each team brings to an organisation is too vital to let it stay that way. According to a recent commissioned study conducted by Forrester Consulting on behalf of VMware, the consequences of these negative relationships can have a big impact on the teams’ ability to meet goals, the culture of the organisation, and the security and innovation each one is striving for. This dichotomy–if left unresolved–will continue to sabotage the agility and security that both teams work hard to provide.

Download this study to start bridging the DevSecOps divide.

Provided by

Did you know that Forrester research reports that only four in 10 of developers understand why a zero trust framework is important?

Only 41% understand the impact zero trust has on the organisation. To address this, understanding how improving the relationship between security and development teams can ensure that ownership, responsibilities, and priorities around zero trust are crystal clear.

This study provides the insight and actions needed to understand and act. Use this insight to lead your DevSecOps teams to success with a zero trust game plan.

Provided by

The study unpacks what’s lacking and the key components needed to create a cohesive environment and common language between these siloed teams.

As security professionals work to secure their organisations, developers are often left out of the security planning processes but are then tasked with carrying out those procedures—further complicating an already strained interaction. It’s no secret that developers are major drivers of business revenue particularly with the increased adoption of cloud and modern apps. Yet, developers see security as stifling their innovations. Security sees it differently.

VMware commissioned Forrester Consulting to introduce the development professional and explore the current state of the IT, security, and development relationship dynamics.

In this study, you will:

• Learn how successful collaborative relationships lead to tangible business outcomes
• Gain understanding of what the groups see as the building blocks of collaborative relationships and how this impacts their work
• Gain actionable steps to strengthen the security, IT, and development relationships for optimal DevSecOps results

Download the study to start bridging the DevSecOps divide.

Provided by

This automatically-generated transcript is taken from the IT Pro Podcast episode ‘Coping with technical debt’. To listen to the full episode, click here. We apologise for any errors.

Adam Shepherd

Hi, I'm Adam Shepherd.

Sabina Weston

And I'm Sabina Weston,

Adam

And you're listening to the IT Pro Podcast. Now, if there's one phrase that's all but guaranteed to strike fear into the hearts of IT managers, it's technical debt. The result of continual design decisions and iterative changes to a system, technical debt is the term used to describe the time investment and cost required to restructure and improve a system after it's already been created.

Sabina

While it may not initially seem like much to worry about, the gradual buildup of technical debt over time can leave organisations looking at a truly colossal task when they need to update their systems. Although some level of technical debt is unavoidable, there are steps that organisations can take to reduce its accumulation and to manage it in a more sustainable way.

Adam

This week, we're joined by Sean Leach, chief product architect at edge security vendor Fastly, to discuss how organisations can cope with their technical debt without letting it overtake them. Sean, welcome to the show.

Sean Leach

Thank you very much for having me.

Adam

So, Sean, let's start off with talking about where technical debt comes from. Now, technical debt can accrue from a variety of sources. I've heard it said that it can often be caused by engineers taking shortcuts in order to deliver outcomes quickly. Would you say that's a fair assessment?

Sean

Yeah, I think if you look across the board in the IT world, there's too few people working on too many projects with not enough time. And sometimes the bosses don't care of all those things, they just want it done. And so in some ways, they're forced to take on some technical debt to get their projects done on time and, you know, keep their jobs and to keep their, their livelihood and whatnot. So unfortunately, it is something that is a, you know, a reality in the IT space. It doesn't have to be... I think sometimes people look at technical debt as all bad. Like, there's, it's only bad, there's no good around it. And our founder, Artur Bergman used to use this great line about technical debt that he would somewhat look at it like financial debt, sometimes you go into financial debt to finance your growth, to finance your expansion. And so it's not always a negative thing for an organisation, especially if you're trying to move fast, ship products, get the business off the ground and successful, you just know that you're going to have to take some on early, and then prepare to pay pay it down later.

Sabina

And can you tell us what impact can technical debt have on an organisation?

Sean

I think there's a few there, there's like, you know, technical outcomes and repercussions. But then there's emotional repercussions as well. Right? Like, I think the stress level of developers goes up in, you know, on new projects on existing projects, the more technical debt that they see that they're going to have to deal with later, right. So they can't just focus on the task at hand, they have to think about, well, I'm going to do this now just to get this thing done. But I know I'm gonna have to deal with this later. So it just builds up the stress level of those developers. And I think that's really one of the like, the technical and the process, like the business reasons, the technical debt can be fixed easier. I mean, I don't wanna say easy. Nothing's easy, but easier than sometimes the emotional, sometimes you lose people. I mean, we all see it; right now, they're calling it the great resignation, right, where a lot of people are leaving their their jobs and moving to new jobs. And part of that, obviously, is due to the pandemic. So some of it is, you know, we feel like the pandemic has brought out feelings that were maybe repressed before about the stress and difficulty of working in an environment where you do have so much technical debt, where you have, you have too many projects you're working on. Like, not enough time, not enough people. So I mean, I think technical debt, the emotional side of it can be, you know, worse than the business side.

Adam

Yeah, absolutely. Because especially when you're in a large organisation that maybe has a bit more rigidity in terms of its processes, when technical debt gets to a certain level, it can feel really just overwhelming, because when it's past a certain point, how do you even approach tackling it? You know, if it's a system that has maybe 5, 10 years of accumulated technical debt, you know, it can almost feel like it's an impossible task to start paying that down.

Sean

Yeah, and I think sometimes people, I think they think of certain things as technical debt, which aren't technical debt, like I think sometimes people think, okay, I built this system in a not new and hip language and environment, right. Like I didn't build it with React with a Node backend or something like that. I had to build it in Java or C# or Go or one of the - I'd say Go's a little hipper than those other two - but like, right like those go, you think of Java and C Sharp, they think, oh, that's technical debt. Like because I built it in this not the hippest new technology. But in reality, that tech is the most stable and should actually lower their stress over time, right? They're not having to, like, keep up to date with the latest tech and trends and like, tech that seems cool now, but in reality, it isn't that great. And so they built this entire project with it, they can't hire people to work on it, they can't, they can't maintain it. So like, that is not technical debt. To me, that's actually smart engineering, is taking tried and true technologies, like you look at Fastly, like our, some of our most critical systems are written in C. C's been around a long time now. Now we're moving a lot of that to Rust, not because C isn't hip anymore, that because we want the memory safety of Rust on some of our edge services. But we still have code in Go we have code in Perl, like some of our big coolest systems are written in Perl still. And so we don't see that as technical debt. That's just technology that wasn't the new hip. The technical debt for me that I see out there is when I cut corners, when I say well, I really should be writing tests for this. But I'm not, I'm just, I got to get this thing out. So I'm not gonna write tests. So like that's for recommendations, what we do is, every code check in has to have tests that goes through full review by a team member. And if there's no tests, if they see, you know, shortcuts taken, they push back on the PR, and that goes all the way up to the CEO, and Artur, our founder and chief architect, and our CTO, Tyler, they do not let that code go into production if there's not tests, if there's not, because we're also in an environment where code bugs can cause, you know, real issues on the internet. And so we're especially focused on it, but everybody really should, because, you know, there's very few non critical systems anymore, everybody's going ahh, nobody cares about this thing. I'll just write this. And if it's broken, like, that's okay, like IoT devices are like that. But how many IoT devices we see out there are compromised and huge security threats, because developers took shortcuts, and that's the technical debt.

Adam

Yeah, I think for me, one of the biggest areas of technical debt doesn't actually have anything to do with code bases themselves. I think one of the biggest areas of technical debt is actually documentation or the lack thereof. You know, documentation is something I don't think anybody actually likes writing documentation, and everybody has a tendency to think, Oh, it's fine. You know, I know how the system works. I know how it's all, how it all fits together. You know, I can write the documentation next week, or after the project's done, or once we've got past this latest sprint, and it just so often ends up falling through the cracks. And then in 5, 10 years, when the team that built it have all moved on, the people that come in to replace them have this huge, tangled Spaghetti Junction of a code base that they have no idea how it works, because nobody bothered to write it down.

Sean

Yeah, for sure. And it's like, it's not even, there's like multiple types of docs too, right? Like, there's the the pull request or the merge request, I think, in GitLab terms, like there's that documentation, like, Why did I make this code change, and really good details that the reviewer can look at it, that's critical; the document like in the code comments documentation that sometimes gets ignored too, where you do this, like, real creative thing, and nobody else is gonna know what you did, because you didn't comment it. And then there's the post documentation of how this thing works. But there's another piece that we spend a lot of time on, and I've seen some of the best engineering org spend a lot of time on, they're called like, there's different term for them. It's like architectural decision records. And what they are is, when you're deciding to do a thing, you write down everything about why; you're like, What am I thinking? What is the problem? It's almost like a PRD, or, you know, a product requirements document. But it's for the architectural decision that you made. For that exact reason you called out, that you can, somebody five years from now, when they come, to be honest, new developer comes in and like, oh, Why'd that person do that? That's silly. That was a dumb decision. Like I always like, instil in my team, understand the history, but you don't have to be bound to it, but you have to understand it. And what these architectural decision records do is they're documents that tell the future developers and future tech technologists, this is why I did a thing. These are the options I looked at. These are the reasons I went with this options and the trade offs I made. And then this is what we decided to do. And that document is so valuable, not just for your own like, I forgot why I did that. Oh, I remember now. I know I'm guilty of that a lot. New developers come in all the time, like, and they look at it, they're like, Okay, I get it. That way, they don't come into it like, oh, that's dumb, I would have done B. Well, we looked at B, this is why we didn't do it. Now, B might be a good option now. Five years ago, B might not have been a good option. Right? A lot of technology choices are all based on time, the time that you made that decision, not just the technology.

Adam

Yeah, absolutely.

Sabina

Sean, could you tell us like, what steps can organisations take to minimise the buildup of technical debt?

Sean

Yeah, I think there's some really critical ones. Right? One, I just talked about the architectural design records, decision records, whatever ADR you want it to stand for. Those are key. And, and I think more so because sometimes things are labelled technical debt that aren't, right? So like, at the very beginning, you want to say, Okay, how do we pay down this debt? Well, you need to know how much that debt is. And I think by knowing what's debt and what's not debt, like I talked about earlier, as well, like using stable tech is not tech debt. Right? So that's one step; know what what is actually debt. And then another one that we've seen is like, it has, I kind of mentioned this earlier, it has to come from the top down of the executives, which is you have to build in time into the schedule, to not just avoid technical debt, but pick off the technical debt in small chunks, right? Like, okay, so you decide you've got X amount of technical debt you have to deal with right now. It's just like any project, if you look at it, and it's like, three weeks of work, you're like, oh, dear Lord, I'll do this later. Like, it's too much, I can't do it right now. Like, it's too big. But if you break it down into small chunks, right, you break a big project down into tasks, right, the old checklist manifesto, checklist, you know, check, check, check, check, then you start to get some progress, and you feel like success is happening. You need to do that in the tech debt side as well, where you're saying, Okay, we've got this old database. And I don't, I don't even mean like a just one that's, you know, stable tech, I mean, it's really old, you got to get rid of it, it's true technical debt; it's gonna take months to get rid of. Yeah, if that's how you look at it. But if you break it down into really small chunks, and you take at each sprint, or each interval, or whatever it is you do on a development cycle, take small chunks off of that project, you start to feel some, like, success. And we also have seen that it's not good to put one person whose job is to handle technical debt. A, that's a, not a fun position...

Adam

Huge job as well!

Sean

Yeah. Huge job, not that fun, because you're not working really on new things. And it's like, you really want to make it a team effort, like, make it part, like everyone in the team should be bought in and had, and be metric'ed and KPIed or whatever, you know, tool you use on the success of this project, not just this one developer, right.

Adam

Because also, if you've got one person whose job is to manage technical debt, then essentially all they're doing is going going around cleaning up after everyone else on the team. And aside from that being an awful job to give someone, presumably, that's also going to encourage the rest of the team to get into really bad habits, you know, they can effectively say like, oh, you know, I can I can take these shortcuts cause this person is managing the technical debt for me. So if I'm making these decisions, then I've got that safety net.

Sean

Exactly. Yeah. Like, like you said, that's a terrible job. And it's too much burden for one person. And it doesn't allow them career growth, right, like, Oh, I just sit around cleaning up all the, you know, all the garbage of everybody else. And then nobody respects the project, either, right? That's what I was saying like, make it a, make it a goal, make it a metric, make it a task of the entire team and spread the load across all of them. Because then all of them are responsible for that technical debt. Because think about it too, right, when you're dealing with technical debt, you're going to need code check ins to the main project. And if like, nobody will review your code, because they're not working on this. Or they say, Oh, I'm too busy, because I'm working on these cool other features to like, you know, to merge your your code changes in, it's just never gonna happen. You're, the person's going to get frustrated, they're gonna leave, the work's never going to get done; the entire project, the entire team has to be part of that project.

Adam

So, Sean, you've spoken a lot about the tendency to mistake, older technology for technical debt. One of the phrases that is often brought up as being symbolic of a negative attitude towards technical debt is the phrase if it ain't broke, don't fix it. Right? That's that's something that people often bring up as a a symptom of a culture that is maybe a bit more lax towards technical debt than it should be. Would you say that's fair? Or do you think that that's actually a perfectly reasonable attitude to take?

Sean

I think it's a reasonable attitude to take, if you can truly gauge the value of ain't broke. Right? Like, nobody can truly get, Nobody I know gauges that well. Oh, it's not broke! Don't fix it. Well, yeah, it's actually quite broke, you just don't see it yet. Right? We see that a lot on our security side with our with our security customers, where, oh, I haven't hit by, been hit by ransomware yet, so I'm not susceptible to it, or I haven't been hit by a DDoS attack yet. So I don't need DDoS protection. It's like, that's that same mentality where it's like, well, just because you haven't seen it yet, or you haven't realised it yet, doesn't mean it's true. So I would just say, assume it's either, if it's not broken now, it will be broken soon, right? You don't want technology to just sit there, right? Like, it's just like anything else, it just kind of rots over time. Right? Not the tech you're using, but the system you've built with it, right? Like you always need to update it, look for bugs, look for security vulnerabilities, new things that are, new vulnerabilities that are coming out. So every system is a living, breathing thing that needs to constantly be cared for. And so it's not is it broke or not, it's how do I maintain it? How do I keep it healthy? And with that, there's just a lot of care that needs to continue to happen. You can't just write something, put it on the internet and leave it there for 10 years, we see that a lot. Those are all the systems that get broken eventually; broken into or compromised eventually, because new tech, new trends, new security vulnerabilities came out, and they just kept didn't keep up the pace with them. So it's just the fact that everything's always broke. So everything really always needs to be fixed, is a better way of looking at it.

Sabina

Yeah, that's it. That's, that's interesting, yeah, points. According according to you, like at what point should addressing technical debt to take priority over expanding, for example, in a company's IT capabilities?

Sean

Yeah, I don't know if I would make it a 'or' question. It's more like, you think of security. Security, the new world, the way, the right way to do security now, you call it secure DevOps, DevSecOps, whatever your preference is, is continuous security. Meaning, if you look at the old way, it was write a bunch of code, throw it over the fence to the security team, say, hey, please secure this. And then they maybe they do, maybe they don't, but they say okay, and then you launch it, you lather rinse, repeat, right. But that doesn't work, as we know, because A, like it's, it's almost impossible to secure something after the fact; you want to embed security in from the beginning. And not just security, you want to embed the security team in in the beginning, just like I talked about before, like metricking, metricking the engineering team, as a team on a project, the security team should be part of that shared metric, right? Like their success should be the success of the project, not the success of how many security vulnerabilities they found in your project, right. So if you think about that as the right way to do security, it's embedded in the team, constantly, continuous security, you right, you put a new pull request into GitHub, they're secured, there's automated security verification of that, that there's some great tools out there that do that. And then there's the security team is constantly working with the developers to build secure software, not to audit software later for security. So that is the right way to do security. That's also the right way to handle technical debt, which is technical debt shouldn't be looked on as a do I do new things? Or do I do technical debt? It's how do I do both of those? How do I do both of them at the same time, make it part of an overall, like if I'm doing a new, adding a new widget to the website, add in some time to do technical debt as part of that project, don't create another project called technical debt next to the widget project, and then try and say which one do I work on? Because you, guess which ones developers are gonna want to work on? Right? It's the, it's the fun one, you have to embed it into the team's attitude, their daily routine, their metrics to make it part of ongoing development, right?

Sabina

And how does one do that? Like, how does one like persuade sort of like the team or like, you know, how does one create those sort of routines that everybody's just like, follows, is there any sort of like experience or advice that you would give to to team leaders?

Sean

I mean, A, it's the team leader needs to be on board and and want to do this, that's that's a big one; if they're like ugh every time you mentioned this, like it's not gonna be successful, right. But if they're bought in, then they know, okay, I'm doing, I mean, again, choose your development methodology of choice, agile, scrum, whatever it is that everybody uses out there. But it's like, okay, I know that I have four engineers for this - I'm gonna use sprint as the example - I got four engineers for this sprint, I've got 37 items that I need to do that are on my checklist, like, whatever you call your backlog, whatever it is. And 25 of them are new things, and set 12 I don't know if I did the maths right there, but just ignore it, 12 are technical debt, burn downs, or, you know, work downs. You choose the set, you choose the same number for that sprint, like you don't like I said, you don't do two projects, one with new stuff, one with technical debt, you make that part of the overall sprint, you spread it across all the developers that are working on it. Again, it's, it's that philosophy part of that shared goal is more important than the specific specific steps you take. It's everybody seeing this as the right thing to do. They're bought in, they're helping and participating. And it's just like writing secure code, getting one developer, wanting developers to write secure code, they need to want to write secure code, they need to be interested in and excited about it. It's all philosophical and like, like, ahead of time work that you have to invest in to get that going. And then it just starts happening, right?

Adam

So does there come a point then, where technical debt can build up to such an extent that it's easier to just burn a system to the ground and rebuild it from scratch?

Sean

Yes, and there's actually a way to think about that before that happens, right? There's a new, there's some cool new technology out there, there's some new focus areas. And it really spawned out of the DevOps and cloud movement of immutable infrastructure, where you're focused on instead of you having this big old Ubuntu server with like, you know, every piece of software running on it, and you have to worry about upgrading the OS every two years to maintain their LTS, you know, their long term support schedule, and then you've got all this other software running on it, and it becomes a nightmare to upgrade because there's 6000 packages to upgrade every time you need to do this, the kind of like, like I said, the DevOps microservices world, this with Docker, and containers, these these become immutable infrastructure, what you do is you take a big system, and and it's easy to say this for a new project. I get it`, there's a lot of existing projects out there that you can start to do this. And I'll get back to that. But for a new project, you want to start it with this immutable infrastructure concept where everything is broken up into components, and every component can easily be deleted and restarted instantly, right? Like, I think Netflix really pioneered this with the Chaos Monkey stuff early on, where you build a system to assume that every component could go away at any moment. That's a good one for like resiliency. But it's also a good way to design the system, so that you can basically destroy an entire component and rebuild it from scratch daily, hourly, if you needed to. At that point, you can upgrade the software and components of that component pretty instantly, and like Docker, and containers and Kubernetes have really made that much easier than it used to be. So now you've got the system and it's, it's composed of all these easily deletable and, and like restartable components, then you're not doing that one massive upgrade a year that everybody dreads, so they never do it, and you go out of PCI, you know, scope because like, or PCI compliance, because like all your software is out of date, you can upgrade individual components as you need to, and you do it instantly. And you can build the system to do it automatically. You don't have to have that one person log in with root and type, you know, sudo apt upgrade and like cross their fingers, it's just part of the workflow system to rebuild those containers instantly when a new version comes out, right. And then if you build testing in and like you do good testing and workflow testing, like you can feel confident in that system. Now, I mentioned existing projects, those are harder, but it's okay. You take little pieces of it, you move it to a smaller component. You take another piece, you move it to a smaller component. We do this, we do this at Fastly. I've done this in previous lives at VeriSign, Newstar and all these other places. You have to just take, you make it a goal of the team to take that big, monolithic legacy system pulled apart in new pieces. And then you can slowly move... don't, like I said before, like don't plan 'replace big monolithic system with microservices project' like one project and you have to do the whole thing. Do it in pieces.

Adam

Like shaving it down gradually until it's manageable.

Sean

Exactly.

Sabina

Refactoring a system can be a pretty daunting prospect for organisations. Are there any ways to make the process less painful?

Sean

Like, I think the main ones I kind of talked about before, which is like, do it in small pieces, as part of an ongoing multiple sprint, whatever timeline project, right? Don't try and, like do the, don't try to boil the ocean, that's one term, you know, you hear out there, like, don't try and do that, do it in small pieces. And, like make it part of the team's goals to slowly replace a legacy system. And it's like, you can get developers excited about that, too. Like, they don't want to just keep adding, you know, cruft to the existing system. But if they are adding, like functionality to an existing system, and then they also get to replace a component or like split a component out into a new micro service or container, whatever, like, that's that individual piece to start to get rid of that technical debt. That's an exciting thing too, right. It's helps motivate them to want to work on it. And like developers want to ship, like, that's, that's what drives them. They want to ship and they want to see people use their stuff. And spending six months adding functionality to some monolithic legacy app doesn't give them the excitement that they want, but shipping daily? That does. Right?

Sabina

Yeah.

Sean

I ship something daily, that gets people excited. And it makes them remember why they love development. Sitting in in a dark room for six months working on the same boring project, they're like ugh, maybe I should look at a new career.

Adam

Yeah. You want releases to be a thrill rather than a relief, if that makes sense.

Sean

And there's also a thrill of the massive release, too. It's a thrill of like, pending doom. Oh, I spent six months working on this thing. I'm gonna push it out. I sure hope this doesn't break the world, as opposed to small quick wins, right? That's less less of a scary, that's scary for them.

Adam

Yeah, it's kind of, you see a lot of developers in that, in that scenario, where they're hitting the, they're hitting the push to production button, and just crossing their little fingers and saying a little prayer that it holds together.

Sean

So much, so much, so.

Adam

Well, I'm afraid that's all we've got time for on this week's episode. But thank you once again, Sean, for taking the time to join us.

Sean

Absolutely. It's great being here. Thanks so much for having me.

Sabina

You can find links to all of the topics we've spoken about today in the show notes, and even more on our website, itpro.co.uk. You can also follow us on social media, sign up to our newsletter, or subscribe to our YouTube channel for more great content. And don't forget to subscribe to the IT Pro Podcast wherever you find podcasts.

Adam

We'll be back next week with more analysis from the world of IT but until then, goodbye

Sabina

Bye!

Technical debt is one of the most dreaded concepts in enterprise IT; CIOs constantly struggle to stay on top of their technical debt and prevent it from accumulating out of control, and the knowledge that all of your shortcuts are going to catch up with you at some point is enough to keep any engineer awake at night.

However, technical debt doesn’t always have to be bad, and there are ways for organisations to manage its growth in a sustainable way. This week, we’re joined by Fastly’s chief product architect Sean Leach to explore why technical debt is so feared, the potential impact it can have, and even how organisations can use it to their advantage.

Highlights

“If you look across the board in the IT world, there's too few people working on too many projects with not enough time. And sometimes the bosses don't care about all those things, they just want it done. And so in some ways, they're forced to take on some technical debt to get their projects done on time and keep their jobs and to keep their livelihood and whatnot. So unfortunately, it is something that is a reality in the IT space.

I think sometimes people look at technical debt as all bad. And our founder, Artur Bergman, used to use this great line about technical debt that he would somewhat look at it like financial debt: sometimes you go into financial debt to finance your growth, to finance your expansion. And so it's not always a negative thing for an organisation, especially if you're trying to move fast, ship products, get the business off the ground and successful - you just know that you're going to have to take some on early, and then prepare to pay it down later.”

“I think sometimes people think of certain things as technical debt, which aren't technical debt. I think sometimes people think, ‘okay, I built this system in a not new and hip language and environment. I didn't build it with React with a Node backend or something like that. I had to build it in Java or C# or Go’, they think, ‘oh, that's technical debt’. But in reality, that tech is the most stable and should actually lower their stress over time. They're not having to keep up to date with the latest tech and trends... That is not technical debt. To me, that's actually smart engineering.”

Read the full transcript here.

Footnotes

• Businesses don't understand how security can be integrated into DevOps
• Developers welcome major overhaul to GitHub's support documentation
• How technical debt can damage business agility and competitiveness
• The IT Pro Podcast: There’s no such thing as obsolete tech
• IT Pro Panel: Why IT leaders need soft skills
• How to become a T-shaped employee
• IT Pro Panel: Why is patch management so difficult?
• The IT Pro Podcast: Keeping an eye on observability
• IT Pro Panel: Defining DevOps
• The driving forces behind DevOps today and how to integrate it into your business
• IT Pro Live: Scaling enterprise DevOps with a platform team approach
• CloudBees: DevOps initiatives increased during the pandemic
• What is DevSecOps and why is it important?
• IT Pro Panel: The CIO's guide to software

Subscribe

• Subscribe to The IT Pro Podcast on Apple Podcasts
• Subscribe to The IT Pro Podcast on Google Podcasts
• Subscribe to The IT Pro Podcast on Spotify
• Subscribe to the IT Pro newsletter
• Subscribe to IT Pro 20/20

Security and development teams are groaning under the strain of securing organizations, according to a report released this week.

The report, released by web application security tools company Invicti Security, found 78% of respondents reported increased stress levels over the last year. One in five DevOps and security professionals have considered quitting their jobs due to these pressures.

The report blames the problem on a backlog of security tasks, caused in part by a cyber security skills shortage. It says that the average IT team member would need a two-week break from their regular work just to catch up with what it calls 'security debt'.

The report, which surveyed 600 executives and hands-on practitioners across security, development and DevOps roles, found that the heavy workload had an effect on the security process. 70% of respondents frequently or always skipped security steps when delivering projects, it said.

A lack of security in the software development lifecycle isn't helping. Almost half of all developers said that application security testing is completely separate from development, with only one in five reporting that they have fully integrated it into the development process. The result is less secure software, with one in three security issues making it through the development and testing stage to production.

A lack of focus on post-deployment application scanning exacerbates the problem, as professionals fail to allocate enough resources to it, the report said. Only seven in ten of those that fully adopted security in the software development phase regularly scanned more than three quarters of their applications for vulnerabilities and then remediate them.

Security professionals want more automation to help lighten the load. One in six of them said that their companies do not have enough automation in place to test and remediate security issues.

That's due in part to a lack of trust in the tools. Only half of the respondents were confident enough in the accuracy of their vulnerability scanning software, prompting almost four in five to manually verify results. Each verification takes around an hour.

Invicti recommends better training for developers and security teams, paying more attention to post-deployment vulnerability scanning, and automating manual tasks where possible. Machine learning is also making tools more aware of vulnerability context, it concluded.

Provided by

Trust is an essential part of doing business. Whether it is the reliability of the supply chain, the accuracy of financial predictions, or the assurance of product availability, trust from customers, vendors, and suppliers is non-negotiable. For businesses that are AI-driven, this trust hinges on the confidence that their AI solution can help them make their most critical decisions.

This eBook looks at how machine learning operations (MLOps) helps companies deliver machine learning applications in production at scale and covers the importance of secure MLOps in the four critical areas of model deployment, monitoring, lifecycle management, and governance.

It also looks closely at other areas related to trust, including:

• AI performance, including accuracy, speed, and stability
• AI operations, including compliance, security, and governance
• AI ethics, including privacy, bias and fairness, and explainability
• How MLOps helps bridge the production gap between systems and teams

Provided by

As machine learning models are put into production and used to make critical business decisions, the primary challenge becomes operation and management of multiple models. Machine learning operations (MLOps) is the technical response to that issue, helping companies to manage, monitor, deploy, and govern their models from a central hub.

Our report, The Business Value of MLOps by Thomas Davenport, highlights some of the most impactful benefits of MLOps tools and processes for different types of organisations. It is based on interviews with MLOps user companies and several MLOps experts.

Download the report to find out:

• How enterprises in various industries are using MLOps capabilities
• Which organisational challenges affect MLOps implementations
• How to determine the benefits of an MLOps infrastructure

Provided by

With a Kubernetes platform offering like Red Hat OpenShift, your organisation can gain a foundation for developing and running important business applications with increased efficiency and automation.

This whitepaper reports the findings of an IDC survey of 14 organisations using OpenShift, revealing how their development teams have been empowered to pursue business goals with better digital products and services.

Download this report now to see how Red Hat OpenShift can:

• Improve DevOps productivity and effectiveness
• Increase revenue
• Reduce costs

CIOs face an almost never-ending barrage of requirements from multiple directions – just as one challenge is overcome, another one rears its head. From managing the transition to home working due to the coronavirus pandemic to dealing with fresh customer requirements in the post-COVID age, constant change is now the new normal.

While digital systems and services can help CIOs and their businesses overcome the problems they face, the rapid pace of technological innovation creates another significant challenge. CIOs who don’t look to continually disrupt their existing business models face being disrupted themselves. So how can tech chiefs deal effectively with the challenges they face?

One answer is DevOps, a set of practices that brings together the resources and requirements of development and IT operations. Analyst firm EMA says that DevOps helps CIOs balance the business demand for software-enabled innovation with the business requirement for IT performance, risk control, and governance.

Such is its potential power as a business strategy that EMA research finds as many as 85% of medium-sized companies and almost 100% of large organisations self-describe as being active in DevOps at varying levels of maturity.

Bola Rotibi, research director of software development at CCS Insight, also recognises DevOps is a growing trend within organisations that are embracing digital technologies while also transforming applications, operational systems, processes, and cultural mindsets.

“DevOps involves the removal of artificial barriers between development and operations teams,” she says. “These barriers are often created because of inflexible processes that create ownership silos. DevOps is about finding a new working relationship that creates benefits for the entire software process.”

DevOps has a close tie to agile software development, which is a set of practices that allows applications to evolve through collaboration between self-organising, cross-functional teams. Like agile, DevOps has a focus on iteration and this becomes manifest through practices such as continuous testing, delivery, deployment and monitoring.

While agile development represents a radical improvement over traditional waterfall development practices, EMA research suggests DevOps can help businesses go further, playing a crucial role in taking software from ideation to production. DevOps, as such, is the practical working bridge between agile development and increasingly agile operations.

EMA research suggests that some of the key drivers for the implementation of DevOps are speed of software delivery, business agility, reliability of applications, and customer and employee experience. At a time of seemingly constant change in which the future is unclear, DevOps brings increasing certainty – and that’s something that resonates with Rotibi, too.

“Agility and an ability to industrialise output brings considerable value to operating successfully and sustainably in the digital economy. Rigor in execution is a vital discipline proven to deliver reliability and repeatability, thereby underscoring the flexibility of agile organisations served by good DevOps practices,” she says.

Claire Dickson, group CIO at multinational packaging business DS Smith, also recognises the importance of speed to delivery. She says the application of DevOps at her company is all about recognising the power of the team. The aim is to try and push decision making down into the IT organisation and to create a flatter management structure.

Dickson uses Power BI dashboards to check performance targets are being met and then uses the insight she gleans to guide her people. “I do believe that we have to be running a disciplined operational hygiene organisation – that's important, keeping the company safe and secure. But we should be able to do that using data and then lead the organisation with the right behaviours,” she says.

Cynthia Stoddard, CIO at technology giant Adobe, agrees that DevOps and its close partner agile are all about focusing on the qualities of your team. In an ideal world, however, CIOs will be able to extend some of key principles associated with DevOps, such as iteration and continuous delivery, into the wider operational and development ecosystem.

“If you're trying to do DevOps within your development organisation or your product teams, and your business people are still thinking about this one, giant-sized deliverable that they want, then you’re not going to get where you need to be,” she says.

Stoddard’s determination to push some of the principles of DevOps and agile beyond the tight confines of the development team has led her to run training sessions where she’s worked with other employees to get them to be able to change their operating models to think in a more iterative manner.

“That approach can bring the operations and development teams together,” she says. “And that’s both thinking about how you're going to deliver capability, and the capability that you need to deliver, in much more of an agile way. And then you can match all the other back processes that need to deliver and support that strategy.”

What’s clear is that DevOps is now ingrained in the way that many IT teams work. DevOps is quickly becoming mainstream, suggests Gartner – but the analyst firm suggests questions remain on how this relatively new approach to culture, automation and platform design can deliver what it promises: Improved business value through fast-paced, iterative IT delivery.

Gartner believes 75% of DevOps initiatives will fail to meet expectations through 2022 due to issues around organisational learning and change. Recruiter Harvey Nash, meanwhile, reports that two-thirds of digital leaders say skills shortages are hampering their transformation efforts, with DevOps cited as one of the three areas of IT with the most acute shortages.

The development of tools that help to automate and improve DevOps processes should help. Rotibi says organisations with mature DevOps practices have semi-to-fully automated processes and tooling that deliver application and infrastructure deployment.

You can expect the impact of technology in DevOps to continue to grow, too. IDC says global spending on DevOps software tools exhibited double-digit growth during 2020, with worldwide spending totalling $11.9 billion (£8.6 billion). The analyst forecasts more growth in spending through 2025, with total expenditure on DevOps tools expected to more than double during this period.

One crucial area of growth will be AIOps, which is the application of machine-learning analytics technology to enhance IT operations. Gartner estimates 40% of product and platform teams will use AIOps in DevOps pipelines by 2023, reducing unplanned downtime by 20%. Rotibi says AIOps could be a game-changer.

“Organisations increasingly find that their IT estates span multiple operational paradigms – client server, monolithic application architectures, microservices and serverless functions. AIOps can offer an underpinning mechanism for strengthening interactions and collaborations across the delivery process,” she says.

Provided by

If your organisation doesn't have the strategic intention to join development and operations instead of pitting development against operations, you may end up with friction and delay that gets in the way of innovation.

DevOps is a way to come alongside what is already being done well in both development and operations - using existing toolchains that are well-loved -and make it all work better for everyone.

Read this 2021 eBook from Enterprise Management Associates to gain insights into:

• What's driving DevOps today
• Maturity trends across large and mid-size enterprises
• The importance of change management and value stream management in DevOps success

IBM is acquiring DevOps consulting company BoxBoat Technologies as part of an ongoing effort to bolster its cloud software capabilities.

BoxBoat was founded in 2016 to help create strategies for container-based software development. It advises companies on how to build software development pipelines for cloud-native applications and on how to convert existing applications for container-based environments, and offers a range of training services to support this.

This is the latest acquisition in IBM's push to establish dominance in the cloud software development space, which has seen it invest heavily in Kubernetes-based container infrastructure, on which many modern cloud applications depend.

IBM has said it will fold BoxBoat into its Global Business Services unit to bolster its hybrid cloud portfolio, focusing on container strategy and services. The deal's value has not been disclosed.

One of BoxBoat's focal areas is increasing the security of DevOps processes and has spent time recently addressing software supply chain security following the SolarWinds attack. Security experts are increasingly worried about adversaries compromising software development processes and inserting malicious code into software before it is deployed.

BoxBoat has been working closely with the Cloud Native Computing Foundation (CNCF) on its Secure Production Identity Framework for Everyone (SPIFFE) project. This is an open-source initiative that assigns secure identity certificates to cloud workloads, making it easier for microservices to authenticate with each other securely in the cloud.

The company also works with another Linux Foundation initiative called in-toto, to help secure DevOps pipelines from intruders who might try to compromise software in development.

BoxBoat has parlayed some of this work on third-party software security into a contract with the US Department of Defense under its Small Business Innovation Research (SBIR) initiative to help secure software supply chains.

This is the latest acquisition in a series for IBM, which acquired Red Hat in 2019 for $34bn. More recently, it acquired cloud implementation services company Nordcloud in a December deal that closed in Q1 2021. It also bought cloud managed services provider Taos a month later.

Technology skill sets continue to be greatly sought after, with the highest-paying tech jobs in 2024 closely linked to skills shortages in key areas including cyber security skills, cloud skills, and AI skills.

According to the results of the surveys that informed Hays’ US and UK 2024 Salary Guides, this has led to a 3-5 % average salary increase across these technology sectors over the last year. 

Unsurprisingly, roles with the biggest demand from employers have seen greater salary rises. Some of these roles require formal qualifications but role-specific certifications can also help with job prospects – for example, prospective workers could weigh up a cyber security certification versus a degree. 

We’ve compiled a list of some of the highest-paying full-time roles from the most in-demand disciplines, with data based on figures from Hays’ 2024 UK Salary Guide and 2024 US Salary Guide. 

1. Cyber security architect 

Average salary US: $208,000/year

Average salary UK: £115,500/year

Cyber security architects are in charge of producing and maintaining an organization’s IT security systems and ensuring they evaluate vulnerabilities and risks. It’s important for them to keep on top of current trends in the cyber security landscape and keep staff in the loop on new procedures and standards. 

The responsibilities of a cyber security architect include running penetration tests, creating cyber security budgets, staying abreast of cyber security compliance obligations, and evaluating new technologies such as the impact of AI on security. 

They are expected to maintain good communication with their organization’s chief information security officer (CISO), to allow the executive to perform their demanding role with all the necessary security insights.

2. Solutions architect 

Average salary US:  $203,500/year

Average salary UK: £93,000/year

Solution architects develop and implement solutions that meet business needs, which involves working with stakeholders to understand business requirements and cultivate the right tools. 

Strong analytical and problem-solving skills are essential, and when it comes to technical expertise, employers look for experience with design and software development.

3. Enterprise architect

Average salary US: $171,500/year

Average salary UK: £95,000/year

While solutions architects evaluate business requirements and develop products or services accordingly, enterprise architects identify business needs in relation to external and internal factors including industry competition and the business’ own IT landscape.

Enterprise architects analyze current trends in technology architecture, perform gap analysis, educate IT teams about new frameworks like agile development. With an overarching view of the organization, they’re also often in charge of developing its IT roadmap. 

4. Cloud engineer AWS/Azure (UK) / Cloud solutions engineer (US)

Average salary US:  $152,000/year

Average salary UK: £67,500/year

Cloud engineers manage the maintenance of cloud-based infrastructure, allowing businesses to run their cloud applications and services. Cloud computing has become increasingly important for organizations of all sizes as digital transformation efforts have matured and the cloud engineer has become an all-important role at many firms.

Programming skills are crucial for this role, as well as a strong grasp of DevOps, Kubernetes and containerization, and database management.

5. Infrastructure specialist/architect 

Average salary US: $142,000/year

Average salary UK: £78,000/year

Infrastructure specialists/architects are involved in the development, configuration, integration, and maintenance of their firm’s IT hardware and programs. They are expected to regularly assess IT infrastructure to ensure it meets business needs and look for optimization opportunities such as eliminating legacy infrastructure.

They are well-versed in common operating systems, possessing critical technical problem-solving and analytical skills. Employers also look for strong SQL skills and CCIE or CCNP certifications.

6. Data scientist

Average salary US: $123,000/year

Average salary UK: £65,5000/year

A data scientist's main focus is to help the business assess the questions it needs to ask about data and to collect it in a manner conducive to this, in comparison with data analysts who answer these questions with data-driven insights.

They do this through the use of data mining, the creation of mathematical algorithms, and predictive analytics models for collecting, sorting, and analyzing information. Data scientists also need a firm grasp of programming languages such as Python, R, and SQL.

To become a data scientist, candidates will need at least a computer science degree but typically data scientists will hold a Master's degree or PhD. Data science bootcamps can also be a helpful path to reskilling within the sector.

7. Data analyst

Average salary US: $81,000/year

Average salary UK: £43,000/year

Data analysts are responsible for taking the data that organizations collect and looking for trends that will allow them to provide actionable insights. In today's sector, data analysts will need to be well-acquainted with machine learning (ML) and AI, as well as SQL.

They also need a strong understanding of data management, to raise insights from unstructured data. Strong analytical skills are crucial but, just as importantly, they must be able to effectively communicate their insights to the rest of the business.

The most in-demand tech skills

As mentioned above, the highest-paid tech jobs are primarily those connected to the skills that can help address gaps or an anticipated demand within the sector. For the UK specifically, this includes cyber (a 4% rise), architecture (4.1%) and cloud and data (both 4.3%) per Hays.

“With organizations collecting increasingly large amounts of data and opting for off-site storage solutions, cloud and data professionals have seen a significant uplift in demand,” says Amanda Whicher, director at Hays specializing in technology. 

“Cyber experts are equally sought after – especially in the current geopolitical climate – with employers seeking specialists who can review their security frameworks and protect against heightened cyber threats.” 

Are tech salaries stagnating?

Salary increases do look to be levelling out however, as the UK average was down from seven percent on the previous year. 

But they’re far from stagnant, with Hays UK reporting that over the last twelve months more than three quarters (78%) of UK technology employers increased salaries, with over a third (35%) of these by five percent or more.

In the US, Hays’ survey found that 81% of IT employers plan to increase salaries in 2024, with 39.4% set to raise incomes by 3-5%, and 12% planning to increase salaries by more than 10%.

“US salaries shot up a couple of years ago, but over the last year they have remained somewhat the same – which is high,” says Gene Smith, senior director, Delivery – Cyber, Projects, and Non-Core, Hays US.

“They were increasing at a very fast rate, which resulted in ‘the great resignation’. This caused companies to overpay many tech professionals and ultimately led to mass layoffs in the US tech business. Companies are now in cost-saving mode while levelling things out and seem to be focused on the best value.”

How tech unionization has affected salaries 

Worker are undergoing a shift, as unionization within the tech sector is on the rise. Conventional wisdom had been that, as well-paid workers, there was no need for collective bargaining. Increasingly however, employees are realizing compensation alone doesn’t guarantee fair treatment or a voice on the issues that matter. 

“Workers want more of a say in things like ethical product development, harassment policies, diversity and inclusion efforts – the list goes on,” says Doug Stevenson, technology consultant and founder of website BlinqBlinq. 

In 2023 the sector marked a rise in unionization efforts due to a number of factors including, but not limited to, the changing nature of work (i.e. remote and flexible working), widespread layoffs and employee feelings of being undervalued and overworked.

Levels of organization differ greatly between the US and UK with the former often facing roadblocks from major firms who remain staunchly anti-union and varying state laws on unionization. But we are seeing changes, as evidenced by Microsoft’s first recognition of a union in early 2023.

Forrester research manager Leonardo Bulgarelli Freitas adds that awareness is also an issue across the board. 

“Many tech workers are not aware that unionization is even a possibility. But there are focused groups trying to organize themselves, and if awareness grows, we could see more consolidation over the next few years.”

UK businesses will struggle to attract talented developers over the next 12 months unless more are willing to embrace remote work, new research has warned.

Less than a quarter of available developer jobs in the UK come with an offer to work from home, according to tech recruitment platform CodinGame.

The company analysed 13,000 tech jobs across more than 100 UK towns and cities, excluding areas with less than 50 available developer job listings. Just 23% of full-time listings had an offer of remote work, rising to just 36% when roles listed as "temporarily remote" were included.

CodinGame believes that without an offer to work remotely, UK businesses will not be able to attract DevOps specialists, cloud experts, and other in-demand developer roles.

"It's concerning just how few developer positions are remote roles, with more than three-quarters of the 13,000 jobs we analysed being advertised as office-based," said Aude Barral, the CEO of CodinGame. "Even though the country is slowly coming out of lockdown, we would still have expected the number of companies offering remote programming roles to be higher."

"For example, when we polled HR professionals recently for our latest annual survey, almost half said they would offer developers the opportunity to work remotely full-time. We also polled developers, and 70% said they would like to work remotely if given the opportunity."

Manchester was listed as having the highest number of developer roles at 855, but just 268 (31%) of these offered the option to work remotely. Birmingham, however, had the highest proportion of remote roles, with 38% of its 330 listed vacancies. This was followed by Bath and Southampton, at 35% and 32% respectively.

"By limiting their searches to developers who can travel into the office, companies may find they'll struggle to fill highly specialised tech positions," Barral added. "Opening up their searches to include developers who aren't local and are able to work remotely, could enable companies to tap into a much larger tech talent pool, not just in the UK but globally."

Software developers can work well alone, if they're in control of all their software assets and tests. Things get trickier, however, when they have to work as part of a team on a fast-moving project with lots of releases. A group of developers can contribute their code to the same source repository, like Git, but they then have to run all the necessary tests to ensure things are working smoothly. Assuming the tests pass, they must build those source files into executable binaries, and then deploy them. That's a daunting task that takes a lot of time and organisation on larger software projects.

This is what Jenkins is for. It's an open-source tool that co-ordinates those stages into a pipeline. This makes it a useful tool for DevOps, a development and deployment approach that automates the various stages of building software, creating an efficient conveyor belt system. Teams that get DevOps right with tools like Jenkins can move from version roll-outs every few months to every few days (or even hours), confident that all their tests have been passed.

Jenkins used to be called Hudson, but its development team renamed it after Oracle forked the project and claimed the original name. It's free, and runs on operating systems including Windows, Mac, and Linux, and it can also run as a Docker image.

You can get Jenkins as a downloadable from the Jenkins.io website, but you'll need to run the Java runtime environment to support it. Alternatively, you can install it as a Docker container by following the instructions on the official Jenkins site, which is what we'll do here. Docker takes a little extra work to set up, but the advantage here is twofold: it solves some dependency problems you might run into with Java, and it also enables you to easily recreate your Jenkins install on any server by copying your Docker file and the Docker run command to run it, which we put into a shell script for increased convenience. Jenkins' Docker instructions also install a souped-up user interface called Blue Ocean. If you don't use the Jenkins Docker instructions you can also install Blue Ocean separately as a plugin.

First, we must create a Python program for Jenkins to work with. We created a simple file called test-myapp.py, stored on our Linux system in /home/$USER/python/myapp. It includes a basic test using the Python PyTest utility:

Create a Github repository for it using git init. Commit the file to your repo using git add ., and then git commit -m "first commit".

Now it's time to start Jenkins using the docker run command in the Jenkins teams' Docker instructions. Once Jenkins is running, you can access it at localhost:8080. It will initially show you a screen with a directory path to a file containing your secure first-time access password. Copy the contents of the file to get logged into the administration screen, and from there it will set up the basic plugins you need to work with the software. Then, you can create a new user account for yourself.

If you're not already in the Blue Ocean interface, click on that option in the left sidebar. It will ask you to create a project. Call it myapp and then select Git as the project type in the Blue Ocean interface.

Blue Ocean will now ask you to create a pipeline. Click yes. We're going to write this pipeline ourselves in a 'Jenkinsfile', which we'll store in our myapp folder.

A Jenkinsfile is a test file describing your pipeline. It contains instructions for the stages of each build. It looks like this:

Each stage reflects a step in the build pipeline and we can have as many as we like. Let's flesh out this template.

Python programs don't need building and deploying in the same way that, say, C++ programs do, because they're interpreted. Nevertheless, Jenkins is useful in other ways. We can test our code automatically, and we can also check its formatting to ensure that it's easy for other developers to read.

To do this, we need to compile the Python program into bytecode, which is an intermediate stage that happens when you run Python programs. We'll call that our build stage. Here's the Jenkinsfile for that step:

The agent is the external program that runs this stage. We don't define a global one, but we do define one for the individual stage. In this case, because we're using Docker, it's a lightweight Alpine container with a Python implementation.

For our step, we run a shell command that compiles our python file.

Save this in your project folder as 'Jenkinsfile' and then commit it using git add . and git commit -m "add Jenkinsfile".

Back in the UI, ignore Blue Ocean's prompt to create a pipeline. Once it spots the Jenkinsfile in your repo, it'll build it from that automatically. Go into the Jenkins dashboard by clicking the exit icon next to the Logout option on the top right, or clicking the Jenkins name on the top left of the screen, to get to the main Jenkins screen. Look for your new project in the dashboard and on the left, select Scan Multibranch Pipeline Now.

Wait for a few seconds and Jenkins will scan your Git repo and run the build. Go back into the Blue Ocean interface, and all being well you'll see a sunny icon underneath the HEALTH entry that shows the build succeeded. Click on myapp, then Branch indexing, and it'll give you a picture of your pipeline and a detailed log.

Now we will add a test stage. Update your code to look like this:

We're using another Docker container to run a simple PyTest (which we included in the code of our myapp.py file). Save this file and update your repo with another git add a . and git commit -m "add test stage to Jenkinsfile". Then, scan the multibranch pipeline as before. When you drop into Blue Ocean, you'll hopefully see success once again. Note that Docker stores everything it runs in its own volume, along with the results. Although you can work some command line magic to access those files directly, you don't need to; Jenkins shows you those assets in its UI. Click on the latest stage to open the build details, and find the entry that says py.test –verbose –junit-xml test-results/results.xml testmyapp.py. Clicking on that shows you the results of your test:

Everything passed! Now we're going to bring it home with the final stage in our demo pipeline: checking the code formatting. There are specific rules for formatting Python code as outlined in the language's PEP-8 specifications. We'll update our Jenkins file to use a tool called PyLint that will check our code. Here's the full Jenkinsfile for all three stages of our pipeline:

Follow the same steps as before: save the file, commit it to your Git repo so that Jenkins sees it, and then rescan the multi-branch pipeline. Then go into Blue Ocean and look at the result. Oh no!

The pipeline stage failed! That's because our code is badly formatted, and PyLint tells us why. We'll update our test_myapp.py file to make the code compliant:

Now, save, commit to your repo, and rescan. Blue Ocean shows that we fixed it (note that in our demo it took us a couple of runs at the Python code to get the formatting right).

You could run all these steps manually yourself, but the beauty of Jenkins is that it automates them all for faster development. That makes the tool invaluable for developers working on a fast cadence as part of a team, but even a single freelance dev, or a hobbyist working on open-source projects, can use this to refine their practice.

Much effort has been put into trying to divine the secrets of what makes tech titans like Netflix, Amazon and Spotify so successful. There are many factors in their meteoric rise, but one common thread is the fact that they all use DevOps methodologies to build their software and services. They’re not alone; DevOps has become a pervasive trend within the software development world, and a huge number of organisations across a range of sectors have adopted the practice.

DevOps can mean different things to different people, however, and one of the most persistent debates within the community is how to actually define a DevOps structure; what essential elements have to be in place for it to qualify? In this month’s IT Pro Panel discussion, we asked some of our expert panellists who have undertaken this journey for themselves what makes DevOps so compelling, and how they’ve navigated some of its more challenging aspects.

Speed freaks

DevOps has many advantages, but the most commonly cited benefit is development velocity. IT leaders want to be able to release more code at shorter intervals, speeding up time to value for the business and addressing changes in a more timely manner. The practice is built on the concept of continuous integration and continuous delivery (CI/CD), and it’s this focus on pushing code as frequently as possible that drives much of the thinking in its implementation.

“Three years ago, we did around one release every two weeks, and it took us about two to three days to get to production,” explains Yolt CTO Roderick Simons. “It was too painful, took too much time and we had to improve.”

“As a first step,” Simons says, “we increased the frequency of delivery from once every two weeks to once every week. If something hurts, you have to do it more often.”

This allowed the team to get faster feedback on changes, and from there, they mapped out their development process in order to identify major bottlenecks. Now, he says, Yolt is able to do daily code releases. To support this, Yolt uses a highly-automated CI/CD pipeline, which is something that SmartDebit CIO Gavin Scruby and TempCover CTO Marc Pell have also been working on.

Scruby, Simons and Pell are all using automation tools to speed up their code releases, but some of our other panellists say they’re concentrating on alternative elements beyond speed of delivery. For instance, design house Studio Graphene has a number of internal checks and controls to ensure that code is stable before being pushed out to clients, which founder and CEO Ritam Gandhi notes must be balanced with agility and speed.

For Craig York, CTO of Milton Keynes University Hospitals Foundation Trust, making sure that all new code is stable and fit for purpose is also paramount. Even outside of the ongoing COVID crisis, healthcare IT systems operate with a razor-thin margin for error, and York’s team focuses on maintaining the availability of systems as much as possible.

The hospital’s IT estate includes systems from many external suppliers, so every time one of them releases an update, the MKUH DevOps teams are responsible for the customisation and integration work necessary to ensure it doesn’t impact the delivery of clinical care. This often involves working as “testing and development partners”, in York’s words, as well as liaising with non-IT teams across the business.

“User engagement from our organisation has been critical for us over the past few years,” he explains, “with a clinical consultant and nursing staff now embedded within IT. Each week, our change/release board includes these staff and managers from across development groups and operational IT to ensure that, as a group, we’re ready and confident of all releases.”

The DevOps journey

It’s a common mantra within the community that establishing a DevOps culture is an ongoing process, and for most of our panellists, the finish line isn’t yet in sight. Newcastle Building Society, for example, is still in the relatively early stages of establishing its DevOps capabilities. The company has set up a pilot programme, putting its mobile app product in the hands of a ‘model team’ to act as a testbed for DevOps practises and tooling, which CIO Manila McLean hopes to roll out across the rest of the development organisation.

“This team is fully DevOps,” McLean says; “multi-skilled, and accountable for product ownership, user experience, development, testing, release, et cetera. The team is accountable for the holistic product including the operational performance and future roadmap, so they need to balance between business as usual and strategic developments.”

She plans to expand this strategy over the course of this year, but notes it’s not simply a case of spinning up more DevOps teams. In order to replicate this model team’s structure across other products, buy-in will need to be obtained from the rest of the business first. While McLean is positive about the progress made on this front so far, she admits that it’s going to take time.

“It's really important to take the business on the journey, so we can't rush this.”

This journey doesn’t stop once product teams are successfully in place, however, as Pell can attest. While he’s already established CI/CD pipelines and automated elements of TempCover’s infrastructure, he says he’s closer to the middle of the process than the end, and that there’s still more to do to automate further aspects of its infrastructure to allow for more frequent releases.

“Similar to Marc's response, we are somewhere in the middle,” Gandhi adds. “It's a mindset and skill that we really started to focus on a few years ago and one of the nice things about constantly working on new blank canvas projects is that we have been able to adapt and try different approaches, before adopting them holistically across our business.”

For those panellists who have successfully productised their engineering organisations, the focus now is on maximising the efficiency of their internal tools and pipelines, tuning processes and deploying further automation where possible. As Scruby notes, improvements in one area of a DevOps structure often naturally lead into another.

“Once you start with automating testing a bit to improve quality, it links within similar tools and skills to more automated deployment, then more automated integration, and then a second cycle with more efficient tooling once people have become familiar.”

Tooled up

This feeds into what is arguably one of the most important aspects of DevOps: An organisation’s choice of tools. The library of tools that DevOps professionals have access to is vast, much of it open source, and the variety of options available allows teams to customise their toolchains based on the specific details of their requirements.

The most popular software tends to be built to assist with automation and testing, largely because it’s so fundamental to the DevOps process. There was a wide spread of favourites among our panellists in this regard, including JUnit, NUnit, Cucumber, Jenkins and Selenium, demonstrating the sheer breadth of choices in this category.

Almost all of our panellists, however, were united in their use of cloud infrastructure for DevOps. In addition to a pair of on-site computer rooms, York makes heavy use of Microsoft Azure and its PaaS capabilities, which his devs tell him make it easier to automate their processes. McLean, meanwhile, uses the YAML pipelines offered by Azure DevOps, as well Terraform for automated infrastructure builds and Packer for creating machine images.

Pell also uses pipelines in Azure DevOps for the actual automation aspect of TempCover’s releasing, although he notes that the company does still have a few legacy on-premise CI/CD systems that it’s yet to port over. This will be a priority for 2021, he says, as it represents a large proportion of the company’s remaining on-prem estate and will prove to be an efficiency gain and cost saving once the migration is complete.

SmartDebit is the main outlier in this case; the company is mostly based on-premise, and uses Nutanix’s private cloud technology (which Scruby describes as “expensive but less effort than OpenStack”) to deliver its applications. This naturally has an impact on which tools are most appropriate, but SmartDebit’s situation isn’t all that unusual.

“We used to run on a private cloud but transitioned to the [public] cloud a year and a half ago,” explains Simons. “We primarily started with IaaS but are moving higher in the stack, with some managed services on the data side and data storage. Similar to Manila, we use Packer for images and use Terraform for all our infrastructure-as-code. In our DevOps journey, this was actually a key enabler; giving all teams their own ‘team environment’. To keep the costs down, we scale it up in the morning and destroy it at night.”

It’s easy to see, then, that cloud technologies like PaaS and IaaS are a natural fit for DevOps toolchains, but there was a little less consensus around the necessity of container technology. Kubernetes, in particular, was identified as a framework with significant potential for enabling DevOps, but it’s not without its drawbacks.

For McLean, Kubernetes allows her organisation to develop, deploy and scale their applications faster, with greater predictability, but others noted that the value proposition isn’t as straightforward as something like cloud infrastructure. Gandhi points out that cloud platform tools like AWS ECS can often be used to accomplish the same goal, while Scruby adds that additional systems such as general virtualisation, load balancing and distributed storage may already be delivering many of the benefits of containerisation.

“There isn't the same step in advantage from virtual machines to containers as there was from tin to virtual machines,” Scruby explains. “VMs are, in essence, just flabby containers, so can do most of the job. Changing a whole infrastructure to something that provides an incremental advantage is a hard sell to a board, especially when public clouds often have their own serverless technologies to promote as well.”

“Exactly,” Gandhi adds; “I think migrating a large scale existing infrastructure is much harder compared to when you're kicking off a new blank canvas build. Hence, I completely agree that it just takes time and if the incremental benefit isn't in line with the effort, it's difficult to prioritise.”

Priority transmission

This highlights one of the principal challenges identified by our panellists in relation to DevOps, which is the issue of prioritisation. For many of them, it’s been difficult to balance working on elements of their DevOps platform – such as CI/CD pipelines or automation tooling – with keeping the lights on and performing normal business tasks.

Part of Yolt’s strategy for addressing this problem was to set up what Simons calls a “platform team”; an internal group made up of members of various existing teams, with a remit to work on creating self-service tools for other dev teams within the organisation. In addition to the company’s CI/CD pipeline, this team is also responsible for maintaining elements such as the end-to-end testing suite, performance testing and code security tools.

“We share the same challenge of prioritising resource between normal business development and DevOps,” Pell says. “I'm a firm believer in ownership of concerns and therefore completely agree with the principle of a platform team; coincidentally, we're hiring a team with exactly that name right now!”

This approach has been somewhat challenging for Scruby, however; like Yolt, SmartDebit is beholden to a number of regulatory and compliance requirements, which he says can make breaking down the barriers between teams difficult. He also cites the impact that a DevOps model can have on processes outside of the IT organisation.

“There’s a knock-on effect outside of product. For example, the ancillary teams covering customer engagement, marketing collateral generation and support who can struggle to keep customers up-to-date with change, especially as some customers are institutions that may not have strong employee training or engagement in place. There's also a little-mentioned aspect of accounting practice that’s impacted by DevOps improvements – how to account for system capitalisation and R&D costs. Accountants are still used to big product releases for the balance sheet and struggle if something does not fit neatly into ‘widget factory’ type models.”

Squad tactics

All of our panellists agree that DevOps is as much about people as it is about technology, so the structure of teams is key for a successful implementation. The general consensus is that groups of between four and ten individuals represent the ideal format for a DevOps team, although while Gandhi encourages these squads to be fully cross-functional, Pell takes a slightly different tack.

“We've followed a similar squad structure which features natural specialisms, but with an understanding that cross-functional requirements exist,” he says. “This covers all of our software development across the entire business, so everything from an entirely new project to a feature iteration or bug fix.”

McLean also runs this kind of structure, with a scrum team of around ten people currently acting as the model team in charge of the development and operation of Newcastle Building Society’s mobile app, supported by a platform engineering team. Scruby, meanwhile, says that SmartDebit is too small to support a separate platform engineering team, but the company is extending and empowering its existing scrum team to do more. For compliance reasons, however, it does maintain a separate Ops team responsible for reviewing and controlling its automation pipeline.

“For us, a feature team consists of all the roles needed to deliver the end-to-end journey,” says Simons. “We promote the ‘t-shaped’ or full-stack principles, and all engineers have access to the code base and can make merge requests if needed. We don't have special QA roles; the team is responsible for testing themselves.”

This is a common theme; giving DevOps teams the autonomy and responsibility to run their respective products was cited by the majority of our panellists as an essential part of building a solid DevOps foundation. McLean and Pell also advise that technical and non-technical staff must work extremely closely, and must all be equally responsible for successful project delivery.

On the other side of the coin, McLean echoes Scruby’s point that DevOps professionals need to consider what their code actually does in practice, not just how well it runs. This, Scruby says, can be a hard mindset for some developers to change.

“I completely agree,” McLean adds; “A DevOps team is like running your own business, you need to care about every facet of your product.”

“I'd say the most important factor is an eagerness to embrace a DevOps ‘mindset’ and step outside your comfort zone,” Gandhi says. “Without it, the cultural shift within the team won't really happen and one will just adopt the status quo. One of the pitfalls is to think that setting up the team means your responsibilities are complete. It's an ongoing journey that will continue. There are systems and processes that need to be embedded within the delivery ecosystem - which is a gradual process.”

To apply to join the IT Pro Panel, please click here to enter your details. Please note that we are not accepting applications from technology vendors at this time.

Artificial intelligence for IT operations, typically shortened to AIOps, is a term that refers to the integration of artificial intelligence (AI) in IT management, usually in the form of monitoring tools.

Its global market is set to grow to around $1.1 billion (£801 million) by 2024, according to research from Omida. Despite this, it’s adoption isn't likely to be uniform across all verticals.

Roy Illsley, Omida’s chief analyst, suggested entertainment, healthcare, and media are the main industries leading the way, with professional services and energy also predicted to adopt the technology at a slower pace.

On top of this, research from Gartner underlined the estimated market size for AIOps is $1.5 billion, with a compound annual growth rate of around 15% between 2020 and 2025.

AIOps – deeper insights and greater automation

AIOps allows IT software and infrastructure to be monitored and operated by AI tools. This allows businesses to use it rapidly to identify and solve tech issues like security breaches, or even predict them in advance.

AIOps has been growing in relevance, especially as organisations take on increasingly complex hybrid cloud and multi-cloud architectures.

“Ultimately AIOps will be used to improve monitoring and incident response through gaining deeper insights and greater automation to address various operational challenges,” says Liam Rogers from 451 Research, who is a research analyst in the applied infrastructure, cloud native, and DevOps teams.

“We see organisations leveraging AI/ML (machine learning) enhanced tools to meet a number of needs, including improving correlation across previously siloed datasets such as logs and metrics. This will remove the manual task of setting thresholds and also augmenting simultaneous adoption of other new technologies, such as Kubernetes monitoring.”

A natural evolution of DevOps

AIOps represents a natural evolution of DevOps, as you might be able to guess from the name, according to Illsley. Its benefits can include being more inclusive of all the activities that impact both the employee and customer experience, and potentially even business outcomes.

"It's effectively a new name for the collective of different operation management solutions from information technology service management (ITSM) to information technology operations analysis (ITOA)," he explains.

Illsley goes on to suggest that the market for AIOps is being approached differently by various vendors, but that they're all aiming to find a similar place. As such, he expects to see automation, integration, and data correlation systems advance over the year. He also suggests the market will begin to consolidate next year, as more capabilities will see an increase in mergers and acquisitions.

Bola Rotibi, research director, software development at CCS Insight, adds that we’re likely to see more acquisitions in the security space as this is a topic at the front of minds; we may even start to see a possible link-up between AIOps and 5G “but as of yet it remains unclear how meaningful this will be”.

“It’s more likely we’ll potentially see stronger trends towards IoT and Edge operations, with AIOps coming up against the process control industry. Now, that could be an interesting and worthwhile pairing,” she points out.

How to prepare your business for AIOps

Introducing the latest technologies can be seen as a way for organisations to accelerate growth, increase efficiency and improve customer service; AIOps presents many benefits, such as reducing downtime, resolving issues faster and freeing up engineers to work on more pressing projects by automating tasks. However, it adds a new level of complexity and if this isn’t matched with organisational readiness, it will fail to deliver on these business outcomes.

One of the first challenges is to educate businesses about what AIOps actually is. “It’s a fairly new discipline and many organisations still don’t fully understand it,” says Sumant Kumar, director of digital transformation at consultancy firm CGI.

Without this understanding, organisations won’t fully discern what they want to achieve, won’t put the effort into doing proper assessments of their needs, capabilities and what they already have, and will end up with unnecessary purchases and poor implementations.

“The adoption of new concepts that claim to be a silver bullet traditionally failed to deliver fully on their promises. AIOps is no exception: It’s not a shrink-wrapped solution that can simply be deployed in order to automatically generate an improvement in performance,” Illsley explains.

“Instead, it’s the application of AI to the different activities IT performs. By linking all these, sharing knowledge and automating actions, AIOps can deliver. But this requires the IT department to be honest in terms of the current level of organisational maturing and what it can realistically expect to achieve in the next 12 months by using AIOps.”

Ensure an organisation-wide approach to AIOps

Although challenging, professor Andy Pardoe, founder and managing director of AI consultancy Pardoe Ventures, believes that designing an organisation-wide approach to AIOps is key to success.

“For large organisations with multiple disconnected data science teams, there’s a risk that they might independently define different approaches to AIOps, which then makes standardisation across groups more difficult. Therefore, it’s critical to get the timing of this definition right. Only invest in AIOps once you understand what AI capabilities your organisation needs, and are ideally able to standardise the AI development approach across the different data science teams.”

Dig deep

Gualtieri advises that organisations ensure they do due diligence before purchasing an AIOps solution and dig deep into the way vendors use technologies such as ML, as they may “use it in a trivial way”.

The final challenge for teams to then overcome is integrating with data sources, from log files and database statistics to customer incident reports. “By enabling the AIOps system to access as much data as possible it’s performance will increase exponentially,” says Adam Leon Smith, Fellow of BCS, the Chartered Institute for IT and chair for the institute’s special interest group in software testing.

Only by taking the time to both understand AIOps and the business’ needs and maturity can an organisation truly unleash its full capabilities and power. However, when implemented correctly, this technology can deliver greater agility and flexibility – highly advantageous in these challenging times.

Amazon Web Services (AWS) has announced plans to bring its 5G edge compute service to the UK in early 2021.

First unveiled at last year’s re:Invent event, AWS Wavelength offers optimised solutions for mobile edge computing applications, simplifying application traffic in order to fully utilise the latency and bandwidth benefits offered by modern 5G networks.

The service manages to shorten the time of mobile data response from seconds to milliseconds, making it ideal for time-sensitive sectors such as driverless cars or surgeries, as well as less critical scenarios like gaming.

Speaking at this year’s re:Invent, a three-week event which commenced on 1 December, AWS CEO Andy Jassy said that AWS Wavelength will be launched in the UK in partnership with Vodafone Business.

This will be part of the new Vodafone Business Edge Innovation Program (EIP), which has opened its registration submissions today. The programme will provide startups, ISVs, businesses, as well as freelance developers exclusive access to edge computing training to help them develop, test and deploy a Proof-of-Concept (PoC) 5G application on AWS Wavelength and Vodafone 5G network.

Vodafone and AWS will roll out Wavelength in spring 2021, starting with a commercial Multi-access Edge Computing (MEC) centre in London. The MEC centre will use Vodafone’s 5G network in order to provide an ultra-low latency zone over the extended area of the UK capital.

Commenting on the announcement, Vodafone Business CEO Vinod Kumar said that “working with AWS on edge computing means we are making it simpler for both independent software vendors and our customers to experiment with this emerging technology”.

“We’re doing this by offering an incubation space to create and test applications that we can then industrialise and scale. And we’re already seeing some innovative applications that provide positive business outcomes from Dedrone, Digital Barriers, HERE Technologies, Groopview, and Unleash live, with so much more to come once our MEC innovation programme is running,” he added.

As well as the updates to Wavelength, AWS also used re:Invent to announce a new ML-powered operations service called Amazon DevOps Guru. The service uses machine learning to help developers detect and solve operational problems with applications.

AWS’ Machine Learning VP Swami Sivasubramanian said that the idea behind DevOps Guru was borne from customer requests to “continue adding services around areas where we can apply our own expertise on how to improve application availability and learn from the years of operational experience that we have acquired running Amazon.com”.

“With Amazon, we have taken our experience and built specialised machine learning models that help customers detect, troubleshoot, and prevent operational issues while providing intelligent recommendations when issues do arise," he said.

"This enables teams to immediately benefit from operational best practices Amazon has learned from running Amazon.com, saving customers the time and effort that would otherwise be spent configuring and managing multiple monitoring systems,” he added.

The current state of IT infrastructure, with a sprawling network of systems and changing environments, complicates operations for DevOps and SRE teams. Paradoxically, it also determines a business’ success, because software itself is necessary for success today.

Observability is the key to solving this problem. This whitepaper shows by how proactively collecting, visualising, and applying intelligence to all metrics, your business can understand its entire software system.

Download it now to understand the essential capabilities of a true observability platform, and the four imperatives for today’s DevOps and SRE teams to achieve true observability.

The increasing complexity of enterprise cloud environments, and the rise of hybrid cloud in particular, is rapidly increasing IT workloads and fuelling a rising demand for automation, Puppet’s CTO has claimed.

During a time when many organisations are being asked to do more with less, the shift from mostly on-prem to a mixture of cloud environments in a relatively short space of time has radically complicated the workloads of CIOs.

Speaking exclusively with IT Pro on the launch of Puppet’s automated Comply platform, Puppet CTO Abby Kearns suggested the increasing complexity of IT infrastructures over as little as the last five years is serving as the main element driving demand for automation.

“The hybrid cloud, and managing across hybrid environments, is the number one driver, honestly, because it’s so complex,” Kearns said. “So many companies started about five years ago to move workloads to the cloud, so we started to see that slow migration, but the cloud wasn’t really set up to mimic the way we were managing on-prem environments.”

Enterprises now have an on-prem environment, a public cloud deployment or perhaps even multiple clouds, with different tools, different workloads and different approaches all in play. Businesses are also using more cloud-native applications and more microservices, so the landscape for IT standards compliance is becoming far more complex.

Puppet’s Comply automation platform is a system designed to cut out many of the traditionally manual processes IT teams and CIOs would manage when ensuring their hardware meets a range of compliance standards.

The product, which will be offered in addition to a compliance automation consultancy service the company already markets, would allow customers to manage their own automation programmes across their IT estate.

Alex Hin, Puppet’s principal product manager, told IT Pro the platform will raise IT visibility, identify compliance shortcomings and remediate these issues.

He explained the need for such software comes from small teams of three to five people suddenly being tasked with making configuration changes on hundreds of thousands of nodes, either on-prem or on the public cloud. This becomes a high investment for the company, requiring a lot of spend and a lot of expertise for all environments to move into compliance.

“That’s really where it comes into play,” Kearns continued, “the idea that automation is really the only route to be able to do that. Because this isn’t just something where you can assign more people to the work. You can’t just throw more people at the problem, you’re going to have to figure out how to automate this as you start to get into the hundreds of thousands of workloads. It’s just a different kind of scale.”

Puppet’s Comply platform will launch in the coming weeks with pre-integrated compatibility with the CIS benchmarks, and further plans to integrate a number of other compliance standards in future. These will extend to include many common standards from DISA, FedRAMP, SOX, HIPAA, and PCI DSS.

The drive to automate, Puppet hopes, will begin to free up time for many organisations that are trying to do more with less, particularly as a result of economic pressures due to COVID-19. One example of a process that Comply will automate is the ‘desired state configuration’ feature. This essentially automatically reverts any configuration changes to a ‘desired state’ if the system detects that the change has led the system to deviate from the particular standard to which it’s adhering to.

“For us, we’ve spent the last six months really investing in a platform-centric approach and the opportunity to really extend into compliance and really build on those capabilities are really powerful for us and our customers,” Kearns added.

“And that’s something we’re going to spend the next several years really continuing to expand on, and really continuing to drive innovation from an automation standpoint, but also from a compliance standpoint as we see those things go hand-in-hand for our customers. “

An ethical hacker has been awarded $3,000 (£2,300) for disclosing a security vulnerability that could have lead to the exposure of private GitLab groups.

Solutions architect Riccardo Padovani first encountered the vulnerability in November 2019. He promptly informed GitLab to the fact that private projects which were formerly public could have been accessed by other parties through vulnerable search APIs.

Although the issue was disclosed almost a year ago, the report was only made public on 6 October.

As explained by Padavani on the bug bounty platform HackerOne: "Alice creates the public group 'Example', and a public project named 'Example-project' inside the group. In the readme of the project, Alice writes 'Example'. Now, Alice creates a private group called 'private', and transfers all the 'Example' group to the 'private' group.

'If Bob (totally unrelated to Alice) searches for 'Example' instance-wide, he will not find anything [... but if he] uses APIs, he will receive the results back with the information that should be private,” he wrote, adding that the issue only arises when entire groups are transferred, as opposed to single projects.

GitLab software security expert Jeremy Matos verified this finding and escalated the issue to GitLab’s engineering team. The DevOps tool patched the vulnerability in GitLab version 12.5.4 and awarded Padavani with $3,000 for disclosing it.

Bug bounty hunting, which is a form of ethical hacking that focuses on finding and disclosing security issues, is becoming an increasingly popular source of income for security experts.

In August, it was reported that Microsoft paid out $13.7 million (roughly £10.5 million) across 15 bounty programmes during the previous 12 months, more than three times the amount paid out to researchers in the same period during 2018/19. The company rewarded 327 researchers for identifying bugs and flaws in Microsoft software, with 1,226 eligible vulnerability reports being filed during the period. The biggest single reward was $200,000.

CREST chairman Ian Glover previously told IT Pro that "the demand for high-quality individuals working for professional companies far outstrips supply."

"The UK is seen as one of the leaders in this area and the opportunity to work on international projects is increasing every day,” he said, adding that a registered level professional would expect to earn in the region of £55,000 and a team leader could be looking at more than £90,000.

Trend Micro has been heavily engaged with the DevOps initiatives of other companies for the last several years, helping them with cloud migration and cloud-native application designs, among other things.

Doug Cahill, ESG Vice President and Group Director for Cyber Security, and David Clement, Global Product Marketing Manager at Trend Micro, speak in this video about the importance of constantly updating and maintaining protocols and security measures to ensure your business runs smoothly and safely.

This video runs through the composition of cloud-native applications, challenges in securing cloud-native environments, and best practices for secure DevOps. Click now to watch the video for detailed research insights into DevOps security.

Although MTTR (mean time to resolution) is one of the most widely used metrics of systems reliability, it’s also one of the most misunderstood. Many DevOps teams find it difficult to define, use, or improve, and with increasing reliance on software for running businesses, a lack of understanding around MTTR can disrupt the digital customer experience and, ultimately, threaten the bottom line.

So how do you avoid these problems? You will need a progressive approach to defining and applying MTTR, combining comprehensive instrumentation and monitoring with a reliable incident-response process and a team with an in-depth understanding of MTTR.

This whitepaper walks you through 10 best practices for reducing MTTR the right way. If your team is in need of an improved incident-response plan and a wider understanding of MTTR in general, then this is an ideal whitepaper for you.

Like many other organisations, you may currently find yourself months into a DevOps initiative that is not delivering the results you expected to see. DevOps is increasingly shown to deliver more code deploys, reduced mean time to resolutions, and reduced lead time for changes—so why aren’t you seeing results?

The answer is measurement. Without measuring the right things effectively, you won’t be able to empower your team with data that shows their extra efforts have made a difference, and you won’t know to stop processes that aren’t working for you.

This whitepaper argues that DevOps without measurement is a fail and advises you on measurement strategies. Download this whitepaper to read about five critical drivers of DevOps success.

The COVID-19 pandemic has seen businesses increase their focus on DevOps initiatives, according to a survey commissioned by enterprise software delivery firm CloudBees.

More than half (52%) of respondents said their firms placed higher priorities on increasing their DevOps initiatives as a result of the pandemic. The most popular initiative is daily standup meetings, which are now practised by 56% of the 347 technology professionals surveyed by the Accelerated Strategies Group (ASG).

Others include using cross-functional teams, practised by 46%, and automating tasks, which was selected by 42% of respondents.

Pandemic restrictions, such as the closure of offices, have also seen 52% of businesses prioritise contracting with public cloud service providers, while almost two-thirds (63%) of respondents said that digital transformation objectives had been placed at a higher priority than before.

The CloudBees report also further proved that employees are confident in their ability to remain productive from home. However, despite 60% of respondents saying that software developer productivity has improved during the pandemic, the speed of software delivery varies. A third (33.8%) of respondents saw speed increases, 27% saw decreases, and 39% didn't perceive any change in speed.

According to Mitchell Ashley, CEO and managing analyst at ASG, software teams "have seen demonstrable benefit from working remotely". However, he added that "time will tell which changes are permanent and/or beneficial" in the future.

The findings were unveiled on the first day of CloudBees' DevOps World event, which also saw the company announce the general availability of the first two modules of its Software Delivery Management (SDM) solution.

The first module aims to solve higher-order feature management problems and provide product development teams with control over the features that get issued in software releases. Meanwhile, the second one focuses on engineering productivity and will provide leaders with insight into the prioritising abilities of their team members.

CloudBees also unveiled new DevSecOps solutions for CloudBees CI and CloudBees CD. The new security features aim to address the lack of integration when trying to meet audit and compliance requests.

They include enhanced granularity in Role-Based Access Control, audit-ready pipelines, a newly-developed hardened version of its CloudBees CI, feature flag integration, as well as proven integrations to security automation applications, such as Anchore, Alcide.io, CyberArk, Checkmarx, Contrast Security, FOSSA, RunSafe Security, Shiftleft.io, Snyk, Sonatype, Synopsys, WhiteSource Software, and Zimperium.

Speaking at the start of the event, CloudBees CEO and co-founder Sacha Labourey said: "When we originally came up with the theme for this year's DevOps World - that was back in January - we decided on Transforming the Future of Software Delivery. And it felt right.

"We understood what transformation meant to us at the time, but we had no clue how transformative this world would truly get."

In this eBook, you’ll learn how a hybrid application delivery strategy demands an ADC that manages both traditional IT applications and DevOps microservice apps. To guide you toward the right ADC for your infrastructure, this guide will show you:

• How a hybrid application delivery infrastructure improves agility
• Key implications for a hybrid application delivery infrastructure
• The three must-haves for an ADC to bridge traditional and microservice app delivery

Learn more about the right ADC for you.

DevOps has become a very popular methodology among digital-first companies, but it can be tricky to get right - especially if your thinking is still based on older models of software development. Join Puppet field CTO Nigel Kersten as he takes us through the process of moving away from multiple cross-functional DevOps teams towards a truly unified, platform-centric approach.

Nigel Kersten

Field CTO, Puppet

Nigel Kersten, Field CTO at Puppet, leads a team of engineers to build greater knowledge around how customers around the world are using Puppet to ensure the company is building pragmatic products for current and future customers. In addition, his team focuses on building out service strategies and finding new opportunities and geographies to expand Puppet’s commercial footprint. Nigel brings a worldwide perspective to the challenges some of the biggest companies on the plant have with infrastructure automation, continuous deployment, compliance and DevOps.

Nigel works with Puppet’s largest customers on the cultural and organizational changes necessary for large scale DevOps implementations. His deep expertise and research into DevOps give him invaluable experience and understanding of what it takes to make DevOps work at scale. Nigel has served in a range of executive roles at Puppet across product and engineering over the last 9 years, and came to Puppet from the Google SRE organization, where he was responsible for one of the largest Puppet deployments in the world.

DevOps engineers are one of the most sought-after job positions in the tech industry, with many companies experiencing a shortage of people with adequate skills for the role. This stems from the combined effects of the digital skills shortage as well as the growing digitisation and automation of business processes during the pandemic, and creates an environment where retraining as a DevOps engineer is one of the lucrative career moves of 2022.

The role is currently considered the fourth best job in the United States according to Glassdoor’s annual ranking. However, this isn’t limited to one part of the world – companies based in Singapore have seen salaries in the field increase by a fifth (22%) amid a wave of venture capital investment.

This high demand for new DevOps engineers is only expected to grow, with the DevOps market expected to reach $12.85 billion in the next three years, according to data from Grand View Research.

However, retraining into a completely new career is a significant life decision which can be considered overwhelming to many, leading to becoming stuck in a different role that offers little to no prospects and a lower salary – despite being more than suited to work in DevOps. In order to aid the process as well as clarify any uncertainties surrounding the DevOps engineer role, IT Pro has compiled this guide to tackling all elements of the process: from education and skills training to salary expectations and interview questions. Read on to find out how to become a DevOps engineer.

Who is a DevOps engineer?

A DevOps engineer is a tech expert who understands the entire software development lifecycle — a six-stage process that guides the software development process — and uses various engineering tools and techniques to develop pipelines and solve operational challenges.

DevOps engineers encapsulate the depth of knowledge and years of experience in a wide array of open-source technologies and tools. They’re masters in coding, integrating, scripting and testing, and they possess extensive data management skills.

For this reason, they’re rightly called ‘Special Forces’ who hold core attributes around open communication, collaboration, and reaching across functional borders.

Typically, DevOps engineers are senior developers or system administrators with business, configuration, operation, automation, and leadership skills.

Some people argue the DevOps engineer title is antithetical to the DevOps practice, as it creates another silo. Thus, some companies have people who perform DevOps management tasks without the DevOps engineer title.

What do DevOps engineers do?

As the term suggests, DevOps engineers facilitate collaboration between the development and operations teams to enhance the product quality and streamline workflow. They oversee the efficient release and use of new software code and products.

DevOps engineers are responsible for an application or piece of software from development through testing, infrastructure deployment, application deployment and end-user feedback. They also implement changes based on the input.

DevOps engineers typically spend much of their time researching new technologies that can improve efficiency. They deploy scalable applications and integrate infrastructure with the application deployment process.

Here are a few tasks and duties of a DevOps engineer:

• Designing tools and processes for managing infrastructure
• Working on multiple platforms across various programming languages
• Automating testing and deployment processes and workloads across cloud providers
• Supporting internal and external stakeholders on different platforms
• Creating tools and processes for increased user satisfaction
• Managing version control through Git or similar systems

How to become a DevOps engineer

To become a DevOps engineer, you’ll need technical and interpersonal skills, as you’ll act as a catalyst to break the communication and operation barriers between the development and operations teams in organizations.

The DevOps engineer role is typically a senior position that demands extensive industry experience. Thus, employers might prefer candidates with a tertiary qualification in computer science or related fields and industry experience.

Several career paths lead to becoming a DevOps engineer. Below is a common approach many DevOps engineers take.

Step 1: Earn a bachelor’s degree

Suppose you wish to become a DevOps engineer. In that case, the logical first step is earning a bachelor’s degree in a discipline related to math, computer science, or electronics engineering, with a focus on software development.

Step 2: Earn a master’s degree:

Completing a master’s degree in software development will help you advance your DevOps engineer career. You will learn skills in different aspects of software development, like coding and emerging technologies.

If you are a working professional, consider an online master’s program to help balance your career and education.

Step 3: Acquire the necessary skills

DevOps is a vast environment that covers almost all technologies and processes. Thus, technical skills are crucial to land a job.

Below, we’ll outline the complete DevOps engineer roadmap to acquiring the necessary skills.

Build your knowledge of industry-relevant coding language

As a DevOps engineer, you’ll use different software and plugins for a continuous integration and continuous delivery (CI/CD) pipeline. CI/CD pipeline automates the software delivery process by building code, running tests (CI) and securely deploying an updated version of the application (CD). Automated pipelines help remove manual errors and aid fast product iterations.

You can write versatile, scalable and error-proof code if you have a good grip on some of the popular programming and scripting languages. It will also help to brush up on Java, Python, JavaScript and Powershell for windows.

Understand different OS concept

A sound understanding of the OS concepts will help you be more productive in your programming, including writing codes faster, interacting with input-output devices and communicating with the other OS.

Try to learn at least process management, sockets, virtualisation, I/O management, file systems, and threads and concurrency. Since most DevOps engineers work in Linux, you should consider going through the Linux Administration Bootcamp to learn about Linux OS.

Learn about managing servers

You’ll have to manage a group of servers instead of individual servers. It’ll help to familiarize yourself with web servers like Nginx, which you can use as a reverse proxy, mail proxy, load balancer and HTTP cache.

Networking and security

The world is more connected than ever, making networking and security essential concepts to a career as a DevOps engineer. Learn basic networking concepts like DNS firewalls, IP addresses, and ports. Also, understand the standard rules of networking, such as TCP/IP, HTTP, SSH, and FTP.

Setting up your machine

As a DevOps engineer, you should know when to and how to set up your machine.

A DevOps engineer should be proficient in setting up web servers like Tomcat, IIS and Apache. They should also be knowledgeable about the caching server, reverse proxy, load balancer and firewall.

Learn infrastructure as code

Learning infrastructure as code is one of the most crucial aspects of becoming a DevOps engineer.

DevOps engineers should know about containers, like Kubernetes and Docker, infrastructure provisionings, like Terraform and Cloud formation, and configuration management tools, like Ansible, Puppet and Chef Salt.

Learn CI/CD tools

Setting up a pipeline for CI/CD is something DevOps engineers will always have to do. Though there are many tools, like Jenkins, Drone and TeamCity, in the CI/CD space, it’s best to learn at least Jenkins, which is the most widely used and likely the most sophisticated CI/CD tool available.

Learn Jenkins from Zero to Hero: The DevOps Jenkins Master.

Learn to monitor software and infrastructure

Besides setting up and deploying, DevOps engineers are also responsible for monitoring applications and software. Depending on your company’s preferred tool, learn to use web application performance and monitoring tools like Datadog, Monit, Nagios or New Relic.

Learn about cloud providers

Cloud is growing and will continue to become more prominent in the near term. Sooner or later, you’ll have to move your applications to the cloud. Hence, build your knowledge in one or more cloud technologies, including Amazon Web Services, Google Cloud Platform, Terraform, or Microsoft Azure.

DevOps engineer skills and experience employers are looking for

• Amazon Web Services
• Powershell Programming Language
• Microsoft Azure
• Communication Skills
• Python Programming Language
• Deployment
• Pipelines
• Ansible Configuration Management
• Linux
• Kubernetes

Useful DevOps certifications

Thanks to the massive demand for DevOps professionals, the number of DevOps engineers has increased exponentially over the years.

As you battle with other professionals to land jobs, a DevOps certification will help you stand out from the competition. Certificates show your knowledge of specific tools and your commitment and passion for the subject.

Key players in the industry, including Amazon Web Services and Google, offer certifications to educate users on using their tools and services. Besides the ability to gain valuable knowledge, DevOps certifications are also how they onboard new engineers to their platform.

Here are a few reasons you should consider DevOps certifications.

• They help boost your tech skills and stay updated about the latest DevOps trends.
• They demonstrate your expertise and enable recruiters to validate your skills through certification providers.
• They prove your passion and commitment and measure your DevOps-related skills.

There are hundreds of certification programmes on the market from various websites or organisations. Some are better than others, so compare your options and choose the right certification program.

Your best bet is to choose DevOps certifications accredited by prominent tech companies or relevant organisations.

Here are the best DevOps certification programmes you can apply to.

1. AWS Certified DevOps Engineer - Professional
2. Microsoft Azure Certifications
3. Google Cloud Certifications
4. Red Hat Certificate of Expertise in Ansible Automation
5. Kubernetes Administrator Certification

DevOps engineer salary

Payscale places the average DevOps engineer’s base salary at $94,660 per year, but salaries vary widely by company and location.

DevOps engineer salaries are the highest in San Francisco, California, where they can earn an average yearly salary of $123,748, according to PayScale. This is 30.7% higher than the national average.

DevOps Engineer Interview Questions

Currently, DevOps is one of the hottest buzzwords in the tech space. And in the past few years, the job listings for DevOps engineers have increased dramatically.

You’ll frequently find multiple open DevOps positions at multinational companies like Facebook, Google, and Amazon. However, the job market is highly competitive, and the interview can be quite challenging.

Finding answers to these common DevOps interview questions will help you land the job you want.

• How is DevOps different from agile methodology?
• Which are the top DevOps tools? Which tools have you worked on?
• Describe your ideal working environment for your DevOps team. What have you learned working closely with operations?
• What are the different phases in DevOps?
• Explain with a use case where DevOps can be used in industry/ real life.
• How do you judge your success and the DevOps team’s success as a whole?
• What is the difference between continuous delivery and continuous deployment?
• What are the anti-patterns of DevOps?
• Tell me about a few scripts you’ve developed in the past that have been critical to your DevOps success.
• How is laC implemented using AWS?

Final thoughts

If you wish to become a DevOps engineer, become proficient in various aspects of software engineering and business operations. Also, develop specific tech and interpersonal skills.

DevOps covers a range of technology and tools that will take some time to learn. Focus on your critical skills first and choose the right tools and technology to work with as you go.

Becoming a DevOps engineer takes hard work and dedication, but it opens the door to a wide range of opportunities to grow and succeed.

Historically, it has been very difficult to deliver code changes and new functionality in an agile way in SAP systems. SAP teams have been left behind doing waterfall-style development while other IT teams have moved to more modern approaches like DevOps which can deliver greater business value, more quickly.

That’s been frustrating for SAP professionals who are eager to adopt new ways of working, as well as for IT leaders who’d like to extend a DevOps approach across their entire IT landscape.

This whitepaper sets out ten steps that can help you to move away from old-school, waterfall-style change management to a fast, agile, responsive way of delivering meaningful, business-enhancing change in SAP. Download it to discover why DevOps is now a business imperative - even for SAP.