How AI is helping Heathrow's connected vision take flight
Robot baggage handlers and facial scanners are just some of the ideas being trialled at the world's most frantic airport
Heathrow is one of the busiest airports in the world. A plane takes off or lands every 45 seconds, using just two runways, meaning 219,000 passengers and their luggage are processed every day. In 2018, some 80 million people used the airport.
As Heathrow operates at 99% capacity, there's no room for error, with data analytics, machine learning and more used across all areas of the business to improve consistency of landing times, speed up security checks and even hire more diverse staff more quickly. In the future, that could lead to robot greeters, automated biometrics for security and AI in air traffic control.
Stuart Birrell is the CIO at Heathrow. Alongside running the day-to-day IT, covering everything from back office to passenger Wi-Fi, he also leads the innovation team.
"They seek out longer-term technology trends to understand when they'll come to fruition," he tells us. "And they test them to see whether they are applicable to us." Alongside that team, Heathrow has also set up an "Innovation Hub", which examines next-generation technology as well as innovation in management. "It's not just an IT tech thing, it's a broad business change initiative." Because of those efforts, Birrell says innovation trials are happening in "pretty much every part of the business".
AI and air traffic control
Heathrow's air traffic controllers are already globally renowned innovators - anyone who has faced long delays at the airport may doubt this, but it's true. The firm behind the control tower, NATS, has a plane landing or taking off more than once a minute, achieved via a novel system of holding stacks, with planes joining 29 mile-long queues - this is Britain, after all - to ensure there's always a plane ready to take a spot on the runway.
Heathrow is installing 20 4K cameras, with the images then being fed back to the control tower and into an AI platform
That's only the beginning. In 2015, NATS introduced a new way to measure the distance between landing planes. Previously, planes were kept a set distance apart to avoid each others' wake; NATS pioneered the idea of time-based separation that takes into account prevailing winds. It's a lot of work to add an extra couple of flights per hour, but it also increases reliability in landing times.
Next up is trialling AI. The NATS control tower is 87m high, which means its view is sometimes obscured by cloud. In such cases, controllers rely on radar to see if planes have successfully taken off, but it means they need to give more time between each take-off and landing for safety, reducing capacity by 20%. That's why NATS is installing 20 4K cameras to watch the runways. "It'll be using the 4K cameras and image recognition to support the controllers in low visibility and areas that they can't see particularly well," explained Birrell. Controllers will watch those feeds, but they will also be fed into an AI platform that tracks when planes depart, for automated departure notifications.
There's the Heathrow you see, and the mirror version created in data. "Every passenger, every bag, we get messages and data about," said Birrell. It's not personal data, such as names, but a trail of how people and their luggage flow through the airport. "The data flows are huge, and from all over the world... it's a complete parallel, virtual world."
To make use of that data, Birrell's team has pulled it into a "data lake" using Microsoft Azure. Birrell's team helped train tech-savvy staff from across the airport to use that data via Microsoft Power BI visualisation tools. "We now have a fairly large community across the airport of power users," he said; people who know how to access that data and present it to colleagues. Security managers, for example, manage teams of 40 or 50 staff across ten lanes, scheduling them in 15-minute increments based on when passengers are expected to arrive. "Crew rosters used to be a bit of paper produced a week or two in advance," he explained. "Now it's done the day before... and we can do updates live as dramatics things happen throughout the day or flights are delayed."
That's real-time information, but next Birrell says the plan is to shift to predictive information. "We know when flights are going to be late, often several hours before they arrive," he said. Some 30% of passengers are transferring, so helping them know where to go and get there quickly is key. "Getting that data in front of the passengers and our colleagues across the organisation is really important."
Next up is letting staff build apps using Microsoft PowerApps. Some apps help with translation for overseas passengers, while others help staff track how they meet regulatory requirements. "We've had a couple [of] people come through from the front lines of security and customer service who are self-taught and are building these pilots to make their colleagues' lives simpler," said Birrell. For instance, one security officer built an app to collectelectronic signatures, which could save 30,000 bits of paper annually.
Heathrow is one of the few places you can hop in a driverless car today, although they're really just automated pods that are limited to a set track. However, driverless technology and other automation ideas are being trialled elsewhere at the airport. "We have trials on autonomous vehicles around the airport, around how to transport people and freight, and baggage, which has a lot going on around robotics, semi-automated solutions, and self-guided vehicles," said Birrell. "It's challenging the technology, but also how we think about providing the service."
The challenge isn't the vehicles themselves, but the communication between them. "We're working with various organisations about how to build a data standard so, irrespective of the supplier of a technology, they can all communicate," he said. "So we can bring vehicles and equipment on site and have it not really matter where it comes from, but we have to have visibility and oversight on the orchestration level."
Getting passengers through security is always a bottleneck at any airport, but Heathrow is trialling biometrics with automated gates to help smooth the way. The system is akin to the automated customs gates that use a facial scanner to compare you to the image stored on your passport's chip - what's different is the system currently being trialled isn't limited to the immigration hall.
Automated gates are a familiar sight in immigration halls, but a new variety will be used when you check in (below)
When you arrive at the airport to check in, the system will scan your face, look at the image on your passport chip to confirm it's you, and then tie that scan to your ticket. When you go to security, you need not pull out a passport or boarding card; simply let the cameras look at your face again. The same follows for the boarding gates. So far, the trials are optional, meaning people concerned about privacy can opt out.
When you get to security you won't even need to produce a boarding pass or passport - just look at the camera
"We've been doing biometrics here for ten or 12 years quite successfully," Birrell said. "That's actually relatively straightforward. But when you start attaching a biometric image to a trusted identity, so you can facilitate secure and trusted travel through an airport, that's a real challenge. There's a lot of work by us, the airlines and governments on how to do that seamlessly."
There are other areas where security could see improvement. Terminal 5 may feel relatively new still, but it's due a refresh - and that means new technology. Birrell couldn't say exactly what to expect, but there will be a focus on security, in particular biometrics and next-generation X-rays. "It will be a very different experience with fewer queues and less stress," he promised. And that's the real innovation: making airports less painful places for us humans to be.
Top 5 challenges of migrating applications to the cloud
Explore how VMware Cloud on AWS helps to address common cloud migration challengesDownload now
3 reasons why now is the time to rethink your network
Changing requirements call for new solutionsDownload now
All-flash buyer’s guide
Tips for evaluating Solid-State ArraysDownload now
Enabling enterprise machine and deep learning with intelligent storage
The power of AI can only be realised through efficient and performant delivery of dataDownload now