If you’re reading this article, chances are you’ve just started exploring the vast realm of deep learning, and we hope to pave the way for you to uncover the logic behind implementing neural networks. As incredible pattern recognition machines, we humans hardly imagine how computers execute tasks nearly close to what we can naturally do. To illustrate, it takes no time for us to recognize people, find and classify objects in the images, and even decipher letters in messy handwriting (OK, this one may take a bit longer). Artificial neural networks (ANNs) are aimed at enabling computers to “think” and “see” by imitating how the human brain functions.
This article will introduce the basic ideas of deep learning and ANNs, the most popular types of neural networks, and their advantages and disadvantages.
A brief overview of deep learning
Deep learning is a subfield of machine learning that uses multiple layers to extract higher-level features from the raw input. Simply said, modern deep learning operates networks with multiple layers (and the more layers, the ‘deeper’ the network) where the output of one level is the input for the other. Deep learning has been around since the 1940s, and the approaches back then were relatively unpopular due to various shortcomings. However, the research has helped the field to advance, and some of the algorithms developed during those times are used widely today in machine learning and deep learning models.
Some popular applications of deep learning and neural networks involve object detection, facial detection, image recognition, and speech-to-text or text-to-speech detection and transcription. Still, there are numerous other opportunities ripe for exploration, and they are only expected to increase.
As already mentioned, most modern deep learning models are based on ANNs. So what exactly is a neural network?
What is a neural network in AI?
Inspired by how the human brain functions, ANNs form the foundation of deep learning. These algorithms take in data, train themselves to recognize the patterns in this data, and then predict the outputs for a new set of similar data. That's what makes neural networks and deep learning so exciting – they are designed to discover data patterns automatically with no human interference, and that's something no other method can do. Essentially, neural networks can act as a sorting and labeling system for data, although their accuracy is dependent on the quality and quantity of the data they are trained on.
Perceptron
Neural networks consist of neurons, and ANNs consist of similar smaller units, namely perceptrons. A perceptron contains one or more inputs, a bias, an activation function, and a single output. The perceptron receives inputs, multiplies them by weight, and then passes them into an activation function to produce an output. Adding the bias to the perceptron is essential so that no issues occur if all inputs are equal to zero.
Y = ∑ (weight * input) + bias
So the first thing we do is the calculations within a single perceptron: here, we calculate the weighted sum and pass it through the activation function. There are many possible activation functions, such as the logistic function, a trigonometric function, a step function, etc.
The structure of a neural network
To create a neural network, we simply begin to group layers of perceptrons together, creating a multi-layer perceptron model. The first layer is the input layer which directly takes in the feature inputs, whereas the last or the output layer creates the resulting outputs. Any layers in between are known as hidden layers because they don't directly "see" the feature inputs or outputs. Neurons of one layer are connected with neurons of the next layer by channels. The result of the activation function determines if the particular neuron will get activated or not. An activated neuron transmits data to the next layer over the channels. In this manner, the data is propagated through the network. Finally, in the output layer, the neuron with the highest value fires and determines the output. The mentioned value that neuron receives after propagation is a probability, meaning that based on the input that network got, it estimates the output via the highest probability value.
Once we have the output, we can compare it to a known label and adjust the weights accordingly because weights usually represent random initialization values. We keep repeating this process until we reach a maximum number of allowed iterations or an acceptable error rate.
Types of neural networks
The number of neural networks is excessive, and it grows annually. In the chart of neural networks by Asimov Institute, you can get some idea of the massive variety of neural network architectures. Here, each color represents various nodes, types of computation, and there's even a topology of how information is compressed in different neural networks. Let's look at some of the most popular types of neural networks:
Convolutional neural networks
Instead of the standard two-dimensional array, the convolutional neural networks (CNN) are comprised of a three-dimensional arrangement of neurons where the first layer is the convolutional layer. Each neuron in this layer processes only a small part of the information of the visual field. So, the network understands the images in parts and computes them multiple times to complete the whole picture. It's no surprise that CNNs are very useful for image recognition. Other applications of CNNs include speech recognition, machine translation, and computer vision tasks.
Recurrent neural networks
Brought about in the 1980s, recurrent neural networks (RNN) use sequential data or time-series data that is especially useful for ordinal or temporal solutions, such as language translation, natural language processing (NLP), speech recognition, and more. Similar to the CNNs, RNNs use training data to learn. What makes them different is the ability to memorize the output of one layer and feed it back to neurons of different layers. It's more like a feedback network where information gets re-processed, rather than just a forward propagation where data moves onward. There are various popular RNN architectures, including Bidirectional recurrent neural networks (BRNN), Gated recurrent units (GRUs), Long short-term memory (LSTM), and more.
Challenges of neural networks
In such industries as healthcare or construction, the performance of an AI model is crucial. However, most AI models are far from perfect. Likewise, building a neural network can be unpredictable and cumbersome, and below we'll introduce some disadvantages of neural networks:
Much data required
Neural networks typically require a great deal of training data and several parameters to function properly in production. These parameters include the number of hidden layers, the number of neurons per layer, learning rates, regularization parameters, etc. So when designing neural networks, we need to tune these different parameters to jointly minimize two objectives – the prediction error on some validation data and the prediction speed. However, the ultimate goal is to facilitate neural networks to correspond to human brain functions.
Black box
Because it's hard to establish how the hidden layers work, neural networks are entitled to this black-box nature. In case of an error, it's challenging and time-consuming to interpret the attributes of neural networks, and more importantly – it’s expensive. That is the reason most banks do not take advantage of neural networks to predict whether an individual is creditworthy. Banks have concrete credit terms and are expected to substantiate their decisions on client proposals with records on property ownership, revenue streams, etc. Even if you invest the time and resources for visualizing neural networks, they are still not transparent enough for banks to fully and quickly access the cause-and-effect relationship between the input and output data and communicate it to the end customer. Such a solution would not simplify the estimation of creditworthiness arousing potential deterrents instead.
Time-consuming development
Many computer vision libraries make the advancement of neural networks moderately straightforward. However, in some cases, developers require more control over the details of the algorithm, which is complicated and takes much longer to establish. A neural network is additionally computationally expensive because of the computational power and training data the network requires.
Key takeaways
ANNs indeed redefine the way deep learning develops. Understanding the fundamental nature of neural networks helps a great deal in apprehending deep learning-based AI projects, at large. In a nutshell, ANNs comprise smaller neurons - perceptrons, and each perceptron is responsible for a minor part of the computation that is processed throughout the network. ANNs stand behind some of the most major accomplishments such as self-driving cars, natural language processing, visual recognition, and many more. Though highly progressive and practical, ANNs still hold a lot of space for research and development. Considering the current activity in the field, it's safe to say the advancement of neural networks is very promising.
