September 09, 2020

The Neural Network: A Machine Learns

by Sam Sandqvist

Nature is excellent at problem solving. One of the perennial problems is to ensure the organism survives in a hostile and ever-changing world. The answer? It learns to confront new situations, not always instinctively, but constructively, learning new skills and ways of coping. Welcome, intelligence.

An Artificial Neural Network, or ANN (or just NN), is an efficient computing system whose central theme is borrowed from the analogy of biological neural networks.

Blog img 7-2

ANN acquires a large collection of units that are interconnected in some pattern to allow communication between the units.

These units, also referred to as nodes, or neurons, are simple processors which operate in parallel.

Sometimes we see the term ‘perceptron’ for the whole net, or confusingly, just for the neurons.

What does then a NN look like? The figure in the left shows a very simple NN: two input nodes, one output node, and four hidden nodes between them. What can really be done with a simple structure like this?

The key is in what is happening during learning (i.e., teaching the network). Looking at the picture we notice that the connections between the nodes in the layers are of different width. This is intended to show that the weight of the information passing from one node to another is different. The idea is simple: a node will add all its inputs, but weighted according to the associated connection weight. Then, each node will use its activation function to determine its output that it sends to the next layer’s nodes. The function can provide a binary value, or anything the designer desires; very common is a value between 0 and 1 (or -1 and 1) providing the next layer with that node’s strength indicator of its inputs.

Machine learning model

It is important to understand the underlying model, the machine learning model we are encapsulating in the neural network.
Blog 7-3

A machine (an NN) will do a task, whatever the designer wants it to perform. It will of course achieve this with a certain fitness for that task; its performance. Recall that the machine initially doesn’t (in principle) know anything, so its performance is probably very bad. Thus the experience the machine has is that it performed badly, and it will attempt to modify itself (the weights between the nodes) in some fashion to improve its experience.

The result is a new iteration, a new generation of the NN.

Following this it will try anew. And continue until some stopping criterion is met, e.g., number of iterations, fitness for the task is good enough, or the whatever the designer of the network has determined.

In any case, the outcome is a new neural network (or model) that performs better on its assigned task than the initial model. The model is then frozen and saved for production use.

The data it is taught with, the training set, is then replaced with real data, and it is allowed to perform its task for real.

Of course, there are lots of challenges for the machine learning (ML) model on the journey to production. Some are listed below.

Challenge

Description

Quality of data

Good-quality data is crucial. Use of bad data leads to the problems in data preprocessing and feature extraction.

Time-consuming task

A challenge faced by ML is the consumption of time for data acquisition, feature extraction and retrieval.

Lack of specialists

As ML technology is still not widely used, availability of expert resources is a tough job.

No clear objective

Having no clear objective and well-defined goal for business problems is another key challenge for ML.

Over/underfitting

If the model is overfitting or underfitting, it cannot be represented well for the problem.

Dimensionality

Another challenge ML model faces is too many features or data points. This can be a real hindrance.

Deployment

Complexity of the ML model makes it quite difficult to be deployed in real life.

In the next article I will detail some of the misconceptions we have regarding neural networks. There are many, and they should be taken into consideration when talking about machine learning, and neural networks in particular.

Sam Sandqvist
AUTHOR

Sam Sandqvist

Dr Sam Sandqvist is our in-house Artificial Intelligence Guru. He holds a Dr. Sc. in Artificial Intelligence and is a published author. He is specialized in AI Theory, AI Models and Simulations. He also has industry experience in FinServ, Sales and Marketing.