Deep learning is a family of computer algorithms that enable computer systems to learn from data by discovering layers of increasingly abstract concepts about the data it’s trained on.
Contents
How is Machine learning different from traditional computer programming?
When engineers write code of a traditional computer program they have to define precisely all rules how to calculate all input data to output data.
When we use machine learning methods we don’t specify the relationship between input and output data with a long set of tedious rules. Instead, we provide some examples of input and output data and let the computer software figure out how to calculate the future output based on input by learning from the training sample. The machine improves the quality of calculated results as we continue to provide more examples of training data.
The input data source may come from any type of digital records: financial, digitized audio signals, digital photo images or video streams.
Artificial neural networks
Can machines learn like humans?
We use variety or different machine learning methods (algorithms) to train the system depending on a type of problem that we are trying to solve.
One of the well-known machine learning technics called Neural Networks.
Computer scientists tried to apply the concepts that they learned from studying the human brain to a new class of software called Artificial Neural Networks.
Researchers have been trying to understand how the human brain learns for a long time and if it is possible to use the similar concept to teach a machine.
Renaissance of the artificial neural networks
The concept of artificial neural network It is not a very new field of science and technology. We can find early references to the research of human nerves activity model in calculations published as far as the mid-40s of the 20th century. However artificial neural networks have been mostly subject of academic research and had very limited number of real world applications until the last decade.
The main reason is that that we need enough computing power to use successfully Neural Networks in practical applications. The exponential growth in available computer power made the utilization of neural network possible.
For example, Apple Watch in 2016 had twice more computer power than the most powerful computer in 1985 – Cray2. Available and affordable computer power makes the new applications of machine learning economically possible. We have access to machine learning in our daily lives. We use personal intelligent assistant built into our smartphones or smart home hubs, movie recommendation engines, e-commerce recommendation engines, computer security software, or face recognition systems.
Human neural wiring
The human brain action starts from a signal sent by sensory neurons. The signal flows to the human brain for processing. The brain neurons process the input signals and send the output signals to the next group of neurons. We know three types of neurons by their functions:
- Sensory neurons that carry signals from sense organs to the to the spinal cord and then to the brain for interpretation.
- Relay neurons operate as circuits to transmit signals from one layer of the central nervous system neurons to another.
- Motor neurons carry signals from the central nervous system to muscles. Motor neurons are connected to the relay neurons.
The human brain has about 100 billion neurons that represent about 10% of brain cells. Neurons form tiny gaps called synapses. Synapses pass signals from one neuron to the next. (1)
How does the Human Brain Learn?
Scientists found that brain neurons have brain fibers known as dendrites. Those brain tissues grow when humans acquire new knowledge. The fibers connect brain cells at synapses. When the brain fibers become bigger and connect more brain cells, the brain can store more information.
Brain fibers can grow only from the existing brain fibers. It means that our brain must somehow connect the new information to the existing knowledge stored earlier. The new fiber connections within the brain are not permanent. Practice in learning make the fiber connections thicker and help solidify the new skills or knowledge.
One of the important fundamentals of the human learning is the feedback mechanism. When we practice we make mistakes and receive feedback, then we make a correction. In other words – we learn from our mistakes.
How do we teach an artificial neural network system?
The principle we learn from our mistakes in one of the fundamental concepts behind the learning mechanism of neural network systems.
An artificial neural network system includes layers of interconnected nodes (artificial neurons) in a pattern that is similar to human brain neuron connections. Each node receives an input value and does a simple computation to produce an output value and then sends it to a node in the next layer.
When we train the neural network, we have to provide the fist set of input and output data, so that the system can execute its first learning lesson. During the lesson, the system calculates first output and compares the calculated results with expected training answers. Then it changes the computational rules of each node to fix errors in the output calculations. The lesson is complete when machine calculated output values match the correct training set of output values.
Now we can give the system a test set of inputs. The computer will calculate output, and we compare results with expected testing data. If we see 100% match, then we have a perfect student. In reality, a machine almost never produces 100% match after the first training lesson.
The process of neural network self-learning when the system uses the difference between calculated and expected values to adjust the computational parameters is known as backpropagation. You may find a good explanation with an example of how the backpropagation works in Matt Mazur’s article (2). The backpropagation process does not exist in the human neurons communication. The human brain uses only forward propagation.
The process when we provide data training sets with input and desired outputs to teach the system called Supervised Machine Learning.
Affordable computer power opened a new opportunity in improving machine learning systems and new algorithms development. One of the fast-growing types of machine learning methods during the past decade called Deep Learning.
Deep learning is not only about the number of layers
By its definition, the Deep Learning algorithms utilize multiple layers of processing nodes. The higher layer processes the input data and sends output to the lower layer. Another additional distinct characteristic of the deep learning systems is that each layer output has a higher level of abstraction than the input coming from the lower layer.
It has an analogy to how the human cognitive process works (3):
- Humans organize their ideas and concepts hierarchically.
- Humans first learn simpler concepts and then compose them to represent more abstract ones.
- Engineers break-up solutions into multiple levels of abstraction and processing
Deep learning applications include pattern analysis (unsupervised) and classification (supervised) types of machine learning.
How can we benefit from deep learning in our daily lives?
Researchers and engineers discovered that Deep Learning algorithms help solve the problem of automated pattern recognitions in digital representations of sounds, images, and other data. The automated pattern recognition becomes the foundation block of many practical applications such as speech recognition, image classification, face recognition, robotics vision, and self-driving cars.
Retailers also use deep learning to improve customer service, marketing, and overall customer experience.
- For instance, fashion retailers are trying to offer an application that helps to find products based on the photo of an item that person likes. (4)
- Another application helps people shop for outfits that they like in a movie or a TV show. The application analyses the database of fashion products photos and finds the most visually similar match that is available for purchase.
- The online recommendation engines also use deep learning to help shoppers find right product based on their past purchases, product views, or customer review and rating history.
- The software based on Deep learning helps to develop predictive models to improve demand and sales forecasting in retail. (5)
Who are better in learning humans or machines?
The significant advantage of machine learning that a computer system does not forget the acquired knowledge, while people quickly forget unless they practice multiple times.
However, the artificial neural network cannot yet match the power of human brain in the ability to learn more complex skills.
The human brain has about 100 billion neurons, while a neural network that capable to recognize visual patterns like handwritten text or images, may have less than 10 thousand nodes(artificial neurons).
The human brain neurons are much more sophisticated than a simple mathematical model of neuron function.
What developments can we expect to see in deep learning in the next 5 years?
Here are answers to this question by some of the leading experts in the field of deep learning (6):
- We should expect to see even more accurate and useful speech and visual recognition systems.
- We expect new application domains for deep learning, such as robotics, data mining, and knowledge discovery.
- Deep learning algorithms will become so efficient that they will be able to run on cheap mobile devices.
- Researchers and developers who build large neural systems will make the network architecture more modular similar to Lego blocks.
- Scientists predict significant advances in deep unsupervised learning and deep reinforcement learning.
- We don’t achieve the Holy Grail of human-level cognition within the next five years (though this will most probably happen in our lifetimes), we will see huge improvements in many additional domains. The most promising area will be unsupervised learning, as most of the data in the world are unlabeled.
- More focus will be given to unsupervised training and/or semi-supervised training algorithms.
- We will have neural networks that can summarize what happens in a video clip, and will be able to generate short videos.
- Over the next 5 years, we will increasingly see hybrid systems where deep learning is used to handle some hard perceptual tasks while other artificial intelligence (AI) and machine learning (ML) techniques are used to address other parts of the problem, e.g., reasoning.
- We also expect an increase in multimodal learning, and a stronger focus on learning that persists beyond individual datasets.
- In the next 5 years, harder and harder problems such as video recognition, medical imaging or text processing will be successfully tackled by deep learning algorithms. We can also expect deep learning algorithms to be ported to commercial products, much like how the face detector was incorporated into consumer cameras in the past 10 years.
References
- Neuron, https://en.wikipedia.org/wiki/Neuron
- A Step by Step Backpropagation Example, https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
- Introduction to Deep Learning Algorithms, http://www.iro.umontreal.ca/~pift6266/H10/notes/deepintro.html
- How Three Companies Are Using AI to Reinvent Fashion Retailing, https://blogs.nvidia.com/blog/2017/03/13/ai-fashion-industry/
- Demand Forecasting Using Neural Network for Supply Chain Management, http://www.ijmerr.com/v4n1/ijmerr_v4n1_10.pdf
- What To Expect from Deep Learning in 2016 and Beyond, http://www.kdnuggets.com/2016/01/deep-learning-2016-beyond.html
