AI basics: What are neural networks and why are they significant?
The release of new AI models is just a part of the picture when it comes to developments in the AI industry. But it can be quite daunting to follow what’s actually happening with each new development. That’s because the field of AI is filled with jargon — words such as LLMs, neural networks, and algorithms are thrown around casually without any explanation.
So, to better understand what is going on, here is a series of explainers, which will break down some of the most common terms used in AI, and why they are important. In the previous instalment, we decluttered two basic terms: artificial intelligence and machine learning (click here to read). In this instalment, the focus will be on neural networks, their significance in the AI industry, and how they work. It is essential to know about them as these networks form the foundation of much of modern AI.
What are neural networks?
Neural networks, also known as artificial neural networks (ANNs), are a method that teaches computers to process data. They are a subset of machine learning (ML), and mimic the way the human brain works.
The roots of ANN go back to the late 1950s when psychologist Frank Rosenblatt invented an AI program called perceptron, which was basically one of the earliest ANNs. The invention was inspired by how neurons process information. Neurons are essentially nerve cells that send and receive signals which allow humans to move their muscles, breathe, talk, eat, and think, among other things.
“Roughly speaking, a neuron sums up all the inputs it receives from other neurons, and if the total sum reaches a certain threshold level, the neuron fires. Importantly, different connections (synapses) from other neurons to a given neuron have different strengths; in calculating the sum of its inputs, the given neuron gives more weight to inputs from stronger connections than inputs from weaker connections,” wrote Melanie Mitchell in her book, ‘Artificial Intelligence: A Guide for Thinking Humans’. These adjustments to the strength of connections between neurons are a crucial part of how the human brain learns.
Rosenblatt used the same process to build his perceptron which received numerical inputs and gave one output. “Analogous to the neuron, the perceptron adds up its inputs, and if the resulting sum is equal to or greater than the perceptron’s threshold, the perception outputs the value 1 (it ‘fires’); otherwise it outputs the value 0 (it ‘does not fire’),” wrote Mitchell. Each of the perceptron’s inputs is assigned a numerical weight, and each input is multiplied by its weight before being added to the sum. The threshold is a number set by either the programmer or the perceptron itself.
Put simply, the perceptron makes a yes-or-no (1 or 0) decision based on whether the sum of its weighted inputs meets a threshold. This process helps the perceptron to learn from data and improve its prediction accuracy over time.
How do neural networks work?
Neural networks are just the extension of perceptrons. A typical neural network has three layers of artificial neurons which are called nodes. First, there is an inner layer which receives the data. Then, there is an inner or hidden layer which takes input from the input layer and processes it further. Finally, there is the output layer which gives the result of all the data processing by the neural network.
Just like in the human brain, each node here connects to others, and has its own associated weight and threshold. “If the output of any individual node is above the specified threshold value, that node is activated, sending data to the next layer of the network. Otherwise, no data is passed along to the next layer of the network,” according to an explanation by IBM on its website.
Neural networks require a lot of data to be trained for a specific task such as classification of data or making predictions (how researchers train these networks will be explained in subsequent explainers). Once a neural network has been fine-tuned for accuracy, it can be a powerful tool in computer science and AI.
Why are neural networks significant?
Neural networks help AI systems make decisions without a lot of human assistance. This takes place as these networks can learn and model the relationships between input and output data that are nonlinear and complex.
For instance, neural networks can recognise the inputs between sentences with similar meanings but requiring different actions. Take these two sentences:
- Where can I buy a new pair of sneakers?
- Where can I find quality snowshoes?
“Neural networks help AI applications understand similarities and differences in requests and take the correct action. None of that happens in a vacuum. You must put in time and effort to train neural networks to function as needed with AI applications,” according to a report by the New York Institute of Technology.