An insight into feed-forward networks.

Posted 25 FebruaryTagged artificial intelligence, master in artificial intelligence, master in artificial intelligence online

The development of technology over the last decades has reached different fields such as finance and monetary environments. The use of Artificial Intelligence increased in the last years, especially with the creation of artificial neural networks. But what is exactly a feed-forward network? In our Master of Artificial Intelligence you will discover the involvement of these neural networks in Artificial Intelligence. Keep reading if you want to know more about it!

What is a feed-forward network?

A feed-forward network is an artificial network whose connections between nodes do not form a cycle. This means that there aren’t feedback connections or loops in the network. Moreover, the information only moves forward through the input layer to the output layer with the hidden layers (if any), in between. It was the first and simplest form of artificial network created. Warren McCulloch and Walter Pitts metioned a similar neuron in the 1940s. This is inspired by the functioning of the human brain in which the dendrites recieve information from the neuron, processed in the body and then they pass it to the next neuron.

The two types of feed-forward network

The single-layer perceptron which is formed by the simplest type of this network, which is the so-called perceptron with no hidden units.  It only has an input layer and an output layer. These inputs are fed directly to the outputs with a series of weights. The limitation of this system is that the single-layer perceptrons are linear classifiers. This means that they can only learn linearly separable patterns.

Many perceptrons create the multi-layer perceptron. Furthermore, they can compute non-linearly separable functions. They are one of the primary Machine Learning techniques for regression and classification. The different layers connect in a feed-forward way. Each neuron in one layer connects to a neuron in the subsequent layer. In many applications a sigmoid function can appear as an activation function.

How does the network learn?

The samples pass through the network and they compare the output that they obtain with the actual output. To decrease this error, it is used to change the weights of the neurons. To do this, they use the Backpropagation algorithm . The practice of passing batches of data through the network and updating the weights to make the error decrease is a Stochastic gradient descend (SGD). The Learning rate is a parameter which utility is to determine the amount of weights changes.

In the backpropagation algorithm, the output values are compared with the correct answer to compute some predefined error-function. With this, as explained before, the algorithm adjusts the weight of each connection to reduce the number of the error function.  Pairs of input and output layers are fed into the network for many cycles until the system ‘learns’ the relationship between the input and the output. Some problems of this algorithm are the speed of convergence and the possibility of ending up in a local minimum of the error function.

The SDG is a method of non-linear optimization and the network calculates the derivative of the error function taking into account the weight. It changes them to make the error decrease so the machine converge to a state where the error of the calculations is small. Yet, there is still a problem in training the systems to operate well and requires additional techniques. This is important in cases where there is a limitation on the number of training samples.

Applications of the feed-forward network

Image processing and character recognition

 Artificial networks plays a big role in this area as they can take in several inputs. It can process them to infer both hidden and complex non-linear relationships. This character’s recognition as handwriting has several applications in the detection of fraud and even national security. Another field which is expanding is the use of facial recognition. Some uses are in the field of social media and smartphones or in cancer recognition in medicine. The research on artificial networks establish the further possibilities of deep networks. It allows innovations in areas such as computer vision, natural language processing, etc.


Business decisions such as sales or capacity utilization in finance and stock market require it. Usually, forecasting problems are as complex as the prediction of stock prices in the market. The traditional models of forecasting have limitations at considering the complex and non-linear relationships. Moreover, this traditional models do not have restrictions on input and residual distribution.

Finally, as you see, feed-forward networks and other types of artificial network impact on society nowadays. The development of new technologies will probably allow these systems to increase their capacities in Artificial Intelligence. If you want to learn more, join us in our Master in Artificial Intelligence and Deep Learning. Contact us to learn more!

 Anterior  Todos Siguiente