What is deep learning?
We look at the phenomenon behind some of today's most advanced AI
It fits into a larger family of machine learning techniques that aim to teach a machine to analyse data based on its own determinations, rather than use predefined algorithms built by humans for a specific task.
Deep learning methods are loosely based on the neocortex of the brain, arranging analytical nodes in a series of pathways for data to flow between, essentially connecting them in a net-like network of layered nodes although it's unable to fully replicate the intricate series of multi-layered connections that makes the brain such a powerfful computer.
The analytical capabilities this method provides is helping to power futuristic technology such as driverless cars, helping them to recognise road signs, or to differentiate between objects in their way.
Deep learning models can achieve high levels of accuracy, sometimes exceeding human-level performance, and are usually trained by using a large set of labelled data and neural network architectures that contain many layers.
Deep learning, AI, and its classifications
The idea of AI isn't new. In fact, there are records of artificial intelligence from as early as the 18th century, with the threat of machines becoming as intelligent as humans (or more so) widely popularized in films such as 2001: A Space Odyssey and The Terminator.
But these concepts are no longer ideas captured only in films. They're increasingly becoming part of everyday life and, with the introduction of technology such as super-intelligent chatbots and applications, there could come a time when machine intelligence surpasses human intelligence and it's not too far in the future.
The term "artificial intelligence" is useful for describing the broad idea of machines thinking for themselves, but, in reality, is far too loose of a term when we start to look at the technology. Artificial intelligence itself can be split into two distinct fields of study: general and narrow (or applied) AI.
General AI, as its name suggests, refers to the study and design of systems capable of performing any task that a human would otherwise be able to perform. It's perhaps the most common interpretation of AI and one that causes the most hysteria, given fears around mass automation and the rise of killer robots. As you might have realised already, success in this field has been fairly limited to date.
Narrow AI, on the other hand, has been far more successful. Rather than focusing on building a system that is capable of mimicking a human generally, this field looks at building machines that can perform a specific task or set of tasks far better than any human.
A real-life application of this technology is Replika, a chatbot designed by San Francisco-based AI company Luka, as well as its predecessor, Roman, named after Belarusian entrepreneur Roman Mazurenko.
IT Pro 20/20: Meet the companies leaving the office for good
The 15th issue of IT Pro 20/20 looks at the nature of operating a business in 2021DOWNLOAD NOW
Mazurenko, who had been a close friend of Luka co-founder Eugenia Kuyda, passed away following a hit-and-run incident in Moscow in 2015. In order to honour his memory and help his friends and family grief with the sudden death, Kuyda decided to develop a chatbot which would respond to messages in the same style as Mazurenko when he was still alive – including his memories, sense of humour, and even spelling mistakes caused by his dyslexia. This was done by making the chatbot analyse data including Mazurenko’s messages and social media interactions from the four years prior to his passing.
Unsurprisingly, the concept of the deceased being memorialised with the help of deep AI has also been explored in the media. A few years prior to Mazurenko’s passing and the inception of the Roman chatbot, a similar scenario was depicted in an episode of a popular American TV show Black Mirror, in which a woman deals with the death of her late boyfriend by communicating with an android which had been created to resemble him. In fact, Kuyda had watched the episode, titled Be Right Back, and has admitted to having used it as inspiration for creating the Roman chatbot. Not unlike other AI technology such as facial recognition, the concept raises significant questions about the ethics of such innovation, as well as society’s relationship with death, especially premature.
Admittedly, it's a fairly macabre example, but it shows that while narrow AI does not necessarily hold the same ambitions as general AI, and while it's certainly not as advanced as the killer robots of sci-fi dreams, it has none the less helped replicate some degrees of human intelligence.
That has largely been possible thanks to machine learning. Rather than machines only copying the actions of humans with preset instructions, algorithms built with machine learning principles are used to train narrow AI systems to learn from the data they process.
For example, in the case of a system trying to identify a picture of a birthday balloon, a machine may be taught to use pre-defined routines, such as one to detect shapes, one to identify numbers, and another to analyse colours. In early machine learning models, the system would take these human-coded routines and develop algorithms to help it learn to identify objects correctly.
While this was certainly groundbreaking for the development of AI, flaws in the model quickly surfaced. The biggest issue was the use of predefined analysis routines, which required far too much human input along the way. There were also problems when it came to photos that were difficult to process, such as blurred faces or objects.
Deep learning definition
Models since have drawn on our understanding of the human brain, something that today is known as deep learning.
The term 'deep' refers to the construction of a layered neural network, resembling the mesh of interconnected neurons that sit within the brain. Unlike the brain, which acts like a 3D net where any one neuron is able to talk to any other within its vicinity, these artificial networks operate a tiered structure, with layer upon layer of connected paths to allow for data to flow. A technique called backpropagation adjusts the weight between the nodes in these networks to ensure an incoming data point leads to the right output.
Researchers wanted to recreate the brain's sophisticated analysis process. Each layer is designed not only to analyse data, but also provide additional context each time. As the object passes through each layer, a more accurate picture and understanding of it becomes possible.
In the balloon example, the picture will be broken down into its constituent parts, whether that be its colouring, any numbering or lettering on its surface, the shape it holds, and whether it's being held or flying through the air. Each part is then analysed by the first layer of neurons, a judgement is made, and it's passed along to the next layer.
This could work particularly well in the fight against fraud. For example, a system could be designed to identify fraudulent account activity, involving neural networks that first take raw data, and then add contextual information as it passes through, such as transaction values and location data.
While some networks may have only a few layers, some programs, including Google's AlphaGo - which managed to defeat a champion player of Chinese board game Go in 2016 - have hundreds. Naturally, this requires vast computational power, and although neural networks have always been an ambition for early AI pioneers, until recently it has remained impractical.
Deep learning today
Many of today's most advanced machine learning systems use a neural network to process data. Recent successes in the driverless cars industry have been made possible because of deep learning, while the principles are also being deployed in the defence and aerospace sectors in order to identify objects from space.
While the potential of deep learning is vast, it has limitations when it comes to more human-like tasks. Deep learning excels at pattern recognition, like the complex but fixed rules of Go. But researchers point out the vast amount of training data required to teach a machine only a specific set of rules.
Pattern recognition is perhaps exemplified most prominently in conversation AI, with deep learning acting as the supporting network. Multimodal inputs including voice and recognition capabilities are processed alongside multimodal outputs such as images and synthesized voices. Enterprises from Starbucks to Apple are deploying this intelligence, giving customers the option to place orders through their applications via voice commands and the ease of logging onto their devices with sight alone.
At the current stage of development, it does not appear possible for deep learning to perform the same elaborate, adaptive thought processes of humans, however the technology continues to evolve at quite a rate.
Deep learning tomorrow
Deep learning may not result in killer robots anytime soon, but that isn't to say it won't fundamentally alter aspects of society in other ways.
The research group Google Brain demonstrated how its deep learning AI was thinking for itself. Without specifying any experimental parameters for cat identification, millions of cats were put forward to the 'Google Brain', and the network successfully identified the images without the help of labelled data.
Identifying cats may seem rudimentary, but it isn't difficult to see how such a breakthrough could be put to a more practical use.
In medicine, deep learning has been found to be on a par with human expertise when it comes to interpreting medical images. The study, carried out by the University of Birmingham, may pave the way for AI to play a greater role in the medical field going forwards, easing the strain on resources and allowing doctors to spend more time with patients.
Perhaps the most exciting area where deep learning is being touted as a possible springboard to discovery is in the cosmos. Researchers from ETH Zurich university recently released a paper in which they employed neural networks to study dark matter. When compared to the Hubble telescope, deep learning was found to deliver 30% more accurate values when breaking down the composites of the universe, apportioning baryonic matter, dark matter and dark energy. The researchers concluded by claiming that deep learning is a promising prospect for cosmological data analysis in the future.
What is certain, is that with funding being poured into AI, the Pentagon has allocated nearly $1 billion to AI for 2020 and specifically deep learning research studies, their influence will only grow.