Introduction to artificial neural networks

In this post I will give an introduction about what artificial neural networks are, how they work and your applications.

What they are?

Neural network is a computational method which learn with experience and imitate a neuron network. All neural network has nodes or neurons and are divided in layers, input layers, output and hidden. Hidden neurons can form more than one layer.

Neurons represent an activation function, the network receives information in the input nodes which are precessed in the hidden layers and produce an output. A non-trained neural network have all connections with equal weight, when it is learning, some connections obtain more weight than others, connections with bigger weight means they are more influent in the process of learning and task execution.

Artificial neuron

Here is the artificial neuron’s mathematical model.

The “X” are the input, “W” are the weights, Σ is the sum function and the block f(a) is the activation function with limiter t. The activity level is:

a=W_{1}X_{1}+W_{2}X_{2}+...+W_{p}X_{p}

The output y is equal to 1 if a>=t, 0 otherwise.

Training

One of the most common learning rules is the delta rule, it is for backpropagation neural networks. It is supervised learning rule, by that, an external agent define the desired output and the neural network must adjust the connection’s weights to produce the desired output. The Perceptron delta rule updates the weights with basis in the step function and the Adaline delta rule update based on linear activation.

Initially a standard is showed to the network and it produce an output, the distance between the ideal weight and the current weight is measured and adjusted to reduce the quadratic medium error until a minimum global value. The hyperboloidal represent the error margin of weights which reduce until reach a minimum acceptable error value. The delta rule creates a gradient vector in direction of the ideal weight’s vector.

In the learning for reinforcement, the network’s output is evaluated, if the answer is positive, the weights are adjusted to produce this output. There are situation where it is not possible to label great data quantities, so the non-supervised learning method is implemented, in this learning the data are divided in clusters.

Applications

There are many application with neural networks, here are some examples:

• Robotics (the reason why this post is in this theme);
• Analysis and processing of sound and images;
• Standard reconnaissance;
• Filters against EM noises;
• etc.