Met Office hails huge efficiency gains in first year of cloud supercomputing with Microsoft Azure

The Met Office has marked one year since it transitioned to cloud supercomputing hosted in Microsoft Azure to generate its critical weather data.

Best known for its weekly weather forecasts, the Met Office is a trading fund within the UK government. But beyond its role at a consumer level, it’s also a critical partner for government and industry as the unique provider of past and present weather data, as well as detailed projections for real-world decision making.

Charles Ewen, chief data and information officer at the Met Office, told assembled media that since partnering with Microsoft in 2024, the Met Office has worked to improve its scalability, flexibility, and resilience while also markedly increasing its compute capacity.

This is the Met Office’s first foray into enterprise-scale cloud computing, and as of last year, has seen the organization move away from on-premises computing for its forecasts.

Microsoft now hosts a cloud cluster for the Met Office, which delivers approximately 60 petaflops of computational performance for functions that could normally only be carried out on a scientific supercomputer, such as climate simulations and weather forecasting.

Ewen explained that cluster is one of the top five CPU clusters in the world, with 1.8 million cores operating at three-nines uptime.

The Met Office’s history of innovation

The Met Office has a storied history of supercomputing, having first used the EDSAC computer at Cambridge in 1952 and in 1959 adopting its first full-time forecasting computer in the form of a Ferranti Mercury computer dubbed ‘Meteor’.

In the decades since, the organization has been through numerous upgrades and reinventions. Even the past 18 years have marked major shifts.

When Ewen joined the Met Office, it was still authoring its own relational databases, he noted, adding that even then it was “an odd thing to be doing”.

The Met Office is now chasing collaboration, agility, scalability, and resilience across all its systems – a process that Ewen described as a significant overhaul.

Chasing resilience

Indeed, uptime and operational resilience were key drivers for the Met Office in partnering with Microsoft and migrating to its cloud supercomputer offering.

One year on from going live on Azure, the Met Office is now recording 99.95% availability for its integrated services, 99.77% supercomputer availability, and 100% critical workload availability.

Erin Chapple, CVP for Commercial Solution Areas at Microsoft, explained that the Met Office is making full use of Azure’s managed services, validated models, robust data workflows, and resilience.

“The platform is delivered with telemetry really built in as the standard and at the scale we operate, it allows us to focus on large-scale identification and resolution of incidents,” Chapple said.

“So the observability and the levels of automation that we have right really have eclipsed what we have been able to deliver in [the] past on premises, leading to identification of issues faster and quicker resolution. This also really enables us to support long running workflows in a consistent manner with minimal or zero impact on those jobs.”

This last point is critical for Met Office operations, Ewen explained, as many of its ongoing scientific research projects last for “ten or more years” and need constant variables to produce usable results.

“So the ability to maintain that direction of travel, to continue to develop pioneering science and understanding, to be able to implement that in next generation codes on new technologies, is vital.”

Ewen added that with the built-in telemetry possible on Azure, the Met Office can now identify and address large-scale incidents at a level not before possible.

“So the observability and the levels of automation that we have really have eclipsed what we have been able to deliver in the past on premises, leading to identification of issues faster and quicker resolution,” he explained.

For digital sovereignty purposes, Microsoft offers workloads based in the UK and provides dedicated protections for data sovereignty.

New frontiers

In February, the Met Office announced it had implemented a new and improved weather model, which would not have been possible without its Microsoft partnership.

The new model allows for more accurate local and global forecasts, driven by improved model physics, a greater range of aircraft data, and an overhaul of the Met Office’s prediction system.

Practically speaking, this means the Met Office is now able to produce more realistic rainfall forecasts, better cloud cover forecasts – hugely important for helping schedule aircraft take-offs, for example – and better UK temperature predictions.

The new model is also able to extend forecasting to 10 days, allowing for better medium-range predictions that feed directly into private sector and military decision-making.

Ewen said the Met Office and Microsoft have now formed a true partnership and that the two will continue to work together on meteorological and climate breakthroughs.

Although the bulk of the organization’s work is rooted in computational physics, Ewen acknowledged that AI is a potential catalyst for innovation. The Met Office will continue to assess the role a hybrid approach to AI and traditional compute can play, he explained, but he cautioned against focusing on it too much.

“People are struggling because they're looking through the narrow lens of, typically, AI at the moment and before that, it was cloud,” he said.

Going forward, Ewen said the Met Office will build on its trusted partnership with Microsoft to maintain its national reputation and deliver groundbreaking research.

“Of course, real world progress is measured in our outputs,” Ewen said.

FOLLOW US ON SOCIAL MEDIA

Make sure to follow ITPro on Google News to keep tabs on all our latest news, analysis, and reviews.

You can also follow ITPro on LinkedIn, X, Facebook, and BlueSky.