Met Office uses MongoDB to predict space weather

NoSQL database centralises data and images from NASA and NOAA.

The Met Office has chosen MongoDB as a central data warehouse for information that will be used to predict space weather, such as coronal mass ejections (CMEs), solar flares and solar wind.

The project has been undertaken as the affect space weather can have on our modern lives has become more apparent.

We have created an end-to-end open source solution incorporating Apache and MongoDB

"Natural disasters have been recognised as increasingly important by governments," James Tomkins, data services portfolio technical lead at the Met Office told IT Pro.

Tomkins gave the 1989 geomagnetic storm, which resulted from a CME and solar flare and knocked out power in the Canadian province of Quebec for nine hours, as an example of this.

In a worst case scenario power could be knocked out across an entire continent and take years to rebuild, Tomkins claimed.

More commonly, space weather can cause satellites to go offline temporarily or they may need to be replaced altogether.

"President Obama had been looking at space weather quite closely, following advice from NASA, and that made its way across to the UK, where it was put on the Natural Hazards National Risk Register.

"With it being on the Risk Register, they needed to find a risk owner, and the Met Office was deemed to be the best place to work with NASA and develop the kind of predictive capability that we have for terrestrial weather," said Tomkins.

Currently, space weather prediction is in its infancy and particularly with events such as solar flares, which travel at the speed of light, all that can be done is to notify governments and satellite operators that it is happening, how severe it is likely to be, and how long it is expected to last.

It is hoped the Met Office programme, which brings together data from NASA, NOAA, the UK Space Agency, and the British Geological Survey, will accelerate the development of improved space weather models and predictions.

Tomkins said he and his team chose MongoDB after realising their incumbent relational database was not fit for the purpose of collecting, housing and analysing the diverse range of data the project deals with, which can include images and videos as well as raw data.

"During our pilot of the programme, we encountered a few problems in terms of performance and the amount of development effort required to ingest all of that data and then serve it out again in a performant way to the forecasters," said Tomkins.

"So when we came to implementing the project for production, we reviewed our technology choices and while it was our first step into the world of NoSQL, the use case pattern seemed to fit what we wanted to do," he added.

Tomkins said he and his team had chosen MongoDB as it is an open source resource something the Government has been pushing departments to adopt.

"Whereas we might have gone with traditional vendors in the past, the steer from the Cabinet Office meant we did dip out toe in the water a lot more with open source technologies," Tomkins said.

"We have in fact created an end-to-end open source solution, incorporating the Apache Camel integration framework and Apache ActiveMQ messaging system, as well as MongoDB," he added.

With phase one of the project completed it is hoped the Met Office will soon be able to make space weather information available to the public alongside terrestrial weather information on its website.

However, it is still seeking funding for this endeavour.

Featured Resources

Edge-enabled mobility of the future

Turning vehicle data into value

Download now

Modern networking for the borderless enterprise

Five ways top organisations are optimising networking at the edge

Download now

Address multi-cloud configuration risks

Cloud security challenges and how to overcome them

Watch now

The total economic impact of IBM Security Verify

Cost savings and business benefits enabled by IBM Security Verify

Download now

Recommended

Best Linux distros 2021
operating systems

Best Linux distros 2021

26 Feb 2021
Golang XML parser vulnerability could enable SAML authentication bypass
vulnerability

Golang XML parser vulnerability could enable SAML authentication bypass

15 Dec 2020
How to automate your infrastructure with Ansible
automation

How to automate your infrastructure with Ansible

2 Dec 2020
Trump's election defeat isn't going to get Huawei back in the UK
Security

Trump's election defeat isn't going to get Huawei back in the UK

17 Nov 2020

Most Popular

UK gov flip-flops on remote work, wants it a standard for all jobs
flexible working

UK gov flip-flops on remote work, wants it a standard for all jobs

5 Mar 2021
Star Alliance passenger data stolen in SITA data breach
data breaches

Star Alliance passenger data stolen in SITA data breach

5 Mar 2021
How to find RAM speed, size and type
Laptops

How to find RAM speed, size and type

26 Feb 2021