The post Tf: Regression tutorial appeared first on Predictive Modeler.

Neural Nets is also the first class of algorithm that I have made available within the AiXQL download package. The following video goes over how to use the AiXQL neural networks script. I think about AiXQL as a constantly improving algorithmic package. The video below may or may not be up to date with the latest version of the download package, however, I will upload a new video whenever the script changes significantly.

The post AiXQL: Neural Networks appeared first on Predictive Modeler.

This is an exciting post, because in this one we get to interact with a neural network! There is a download link to an excel file below, that you can use to go over the detailed functioning of a multilayer perceptron (or backpropagation or feedforward) neural network. The video below explains the various components at a high level. If I can explain something better – please let me know using the comment section below!

If you have any suggestions on how I can improve this page, please let me know.

Clarification (2021-04-18): The neuron output function used in the excel example is a logistic function.

The post An MS Excel Example of a Basic MLP appeared first on Predictive Modeler.

A perceptron is a basic unit of a neural network. It is simply a mathematical function that takes in one or more inputs, performs an operation, and produces an output. The following tutorial goes over the basic functioning of a perceptron.

We can arrange several perceptrons in layers to create a multilayer feedforward neural network. This type of arrangement is a back-propagation network. We call it feedforward because the input propagates sequentially through the layers of the network all the way forward to create an output (i.e. prediction or Ŷ). The prediction is compared to the actual output to calculate an error, which then propagates backwards through the network, tuning weights along the way (hence the back-propagation terminology).

There are a few key equations that give one all the mathematics necessary to create a back-propagation multilayer perceptron network (hereafter referred to as MLP in this post). We will describe these in terms of the forward and backward passes through the network.

The Forward Pass

When a signal propagates forward through an MLP, it creates or induces a field for the example of input data at neuron . This field is computed as:

(1)  

where is the total amount of incoming connections into neuron . is the input to neuron , and is the layer of the network (e.g. for a network with one hidden and an output layer,  will be 1 or 2 respectively). The weight connecting to neuron in layer is denoted by . Note that for the first hidden layer is the same as as the data input. For other layers, it will represent the output from the respective neurons.

Once the induced field is calculated, the output of neuron can be calculated according to the selected activation function. We will use the hyperbolic tangent function for this purpose². The output at neuron is then calculated as:

(2)  

where , are constants that are greater than zero. Practically useful values for these constants are and . for neuron and at the data example is calculated as in equation 1.

And that’s it! Equations 1 & 2 completely specify the forward computational pass through the MLP. Now, let’s look at the more complicated backwards pass.

The Backwards Pass

The first thing we have to do is to compute the error term, . We can calculate this term by comparing the output at the final neuron in the forward pass. This is the prediction from the neural network for the output for the example of input data, let’s call this  . We will compute the error as the difference of this prediction from the actual value, .

(3)  

Now, we need the equation to update the weights of the network The equations are not derived here. We use the following equation to update the connecting weights throughout the MLP:

(4)  

where is the change in the weight connecting neuron in layer to neuron in layer . is the learning rate constant (the heuristic value for it range from 0.01-5). is the output of neuron , or in the case of the input layer, it is just the input. There is a new term in the equation (4) above, and that is the local gradient function . This function for the output neuron is defined as:

(5)  

where and are the same constants as from equation (2), is the first order differential of equation (2).

Now that you have the local gradient for the output neuron from (5), you can use equation (4) to update the weights of the incoming connections to the output neuron. In order to update the connecting weights to any hidden neurons, you need to first calculate the local gradient at each of the hidden neurons using the following equation:

(6)  

where is over all of the outgoing connectors connecting neuron in layer to neuron in layer .

Using equation (6) and equation (4), you can now update the remaining weights in the neural network.

I know that was a lot of algebra! Fear not – in the next post I will provide a simple working model of this mathematics in Microsoft Excel. You will be able to see the workings of the forward and backwards pass in live action!

The post Theory: The Multi-Layer Perceptron appeared first on Predictive Modeler.