IT Pro is supported by its audience. When you purchase through links on our site, we may earn an affiliate commission. Learn more

Why the Met Office will keep on-premise kit despite cloudy forecast

While the cloud spurs innovation, sometimes on-premise is cheaper

Weather forecast

The Met Office has turned to the cloud to deal with ever increasing demand for weather data, but many of its core services will remain on-premise for the foreseeable future.

As the number of web apps and weather alert services using the Met Office’s data increase, the organisation decided to move part of its infrastructure into the cloud to ensure spikes in demand didn’t affect its services.

James Tomkins, enterprise architect at the Met Office, tells IT Pro: “It’s an issue of scale for us. The scale of consumption of the data we collect has driven us into the cloud, as well as the customer expectations around how they consume that data as well.”

The organisation’s supercomputer kicks out between one and two terabytes of weather forecasts everyday. “As the number of people who consume that data increases, we were running into bandwidth issues in our ability to transfer that data to the customers,” Tomkins explains.

Latency was also an issue, he added, saying: “If it takes us an hour to transfer that data, it may no longer be any use to the person receiving it. Our ability to transfer these large amounts of data to a large number of recipients quickly was one of the drivers [behind the move to cloud].”

When the weather observation body first took the decision to move into the cloud 18 months ago, it looked to some of the big public cloud providers, and found Amazon Web Services (AWS) to be its best fit after narrowing down its search to the public cloud giant and its nearest rival, Microsoft Azure.

“We’re predominantly a Java/Linux shop,” Tomkins explains. “Amazon had a much stronger ecosystem for that tech stack than Microsoft did and we felt there was more power to leverage in that stack at that time.

“We’ve not done that analysis again, I know Microsoft are making great gains in that space but we’ve never had cause to revisit that decision.”

The Met Office was storing its weather data on an on-premise file system, then using file transfer protocol (FTP) services to push that data to weather aggregation services like Accuweather and The Meteo Group, who would use that to deliver location-based weather forecasts to users.

But the risk was that this on-premise system would be unable to cope with the sheer number of queries from users checking their local weather forecast, and would choke access to the Met Office’s weather warning product, the Public Weather Media Service, which helps major media customers deliver crucial weather warnings to the general public.

With AWS, the Met Office started storing a lot of its weather data in an S3 bucket, allowing location-based weather services to ping this for relevant data, and scaling the bucket to deal with any spikes in demand.

On-premise, “we’d had have to had a server dedicated to hosting an application that could do that, but it might be doing nothing 99% of the time”, Tomkins says.

In the cloud, the Met Office has automated the entire process, so that any data pushed into S3 triggers Amazon SNS (Simple Notification Service) to ping customers with that data.

When the data arrives, AWS Lambda, Amazon’s serverless coding platform, caches it and puts it back into the Met Office’s API, so future requests for the same data can use the cache, rather than trigger a new query, easing demand on the infrastructure.

Featured Resources

Activation playbook: Deliver data that powers impactful, game-changing campaigns

Bringing together data and technology to drive better business outcomes

Free Download

In unpredictable times, a data strategy is key

Data processes are crucial to guide decisions and drive business growth

Free Download

Achieving resiliency with Everything-as-a-Service (XAAS)

Transforming the enterprise IT landscape

Free Download

What is contextual analytics?

Creating more customer value in HR software applications

Free Download

Recommended

What is Amazon S3?
Amazon S3

What is Amazon S3?

16 May 2022
EDB unveils world-first openly governed Kubernetes Postgres operator
Cloud

EDB unveils world-first openly governed Kubernetes Postgres operator

13 May 2022
How the cloud primed Markerstudy for an M&A spree
Cloud

How the cloud primed Markerstudy for an M&A spree

9 May 2022
Gaia-X: The last chance saloon for Europe’s visionary cloud project
Cloud

Gaia-X: The last chance saloon for Europe’s visionary cloud project

4 May 2022

Most Popular

16 ways to speed up your laptop
Laptops

16 ways to speed up your laptop

13 May 2022
Preparing for the 3G sunset
Network & Internet

Preparing for the 3G sunset

18 May 2022
(ISC)2 launches free scheme to get 100,000 UK citizens into cyber security
Careers & training

(ISC)2 launches free scheme to get 100,000 UK citizens into cyber security

17 May 2022