Exercise 5

Program:

• Introduction to Netlab toolbox
• Classification of the wedge dataset using Netlab
• Classification of the XOR dataset using Netlab

Downloads:

• The Netlab toolbox
• Wedge dataset
• XOR dataset

Likely, you will not manage to finish the implementation of all the functions during the excercise in the lab. Finish them as a home work.

Download the Netlab toolbox, unpack it to a directory, and set the path to the directory in MATLAB.

• What is the puprose of the following functions:
  • mlp()
  • netopt()
  • mlpfwd()
• Do you understand the meaning of their arguments?
• In what format does the toolbox expect the data? Columns or rows? Is it the same format we used in previous exercises?
• Do you see the analogy with functions you created last week?

Repeat the task from the last week: train a simple neural network (2,2,1) on the wedge dataset.

• What types of transfer functions are available in Netlab? Can they be used freely for all neurons? What setting is suitable for our data?
• Try changing the optimization algorithm in netopt(). Does the change affect the speed of learning? Does it affect the quality of learning?
• Try to visualize the results:
  • plot the training data and the decision boundary
  • plot the response of the network over the definition range

For plotting, you can use the following code snippets. For the decision boundary:

%% Plot the decision boundary
% Assuming the data are already plotted in current figure.
% Get the current axis limits
ax = axis;
% Prepare the mesh of points we will use to test the network
myx = ax(1):(ax(2)-ax(1))/100:ax(2);
myy = ax(3):(ax(4)-ax(3))/100:ax(4);
[xx,yy] = meshgrid(myx,myy);
% Prepare the data to the right format
data = [xx(:) yy(:)];
% Ask the NN for predictions
pred = mlpfwd(net, data);
% Reshape the predictions to the original matrix size
pred = reshape(pred, numel(myx), numel(myy));
% Draw the contour line - the decision boundary
[foo,c] = contour(myx, myy, pred, [0.5 0.5]);

And for the network response (assuming myx, myy, and pred come from the previous code snippet):

surf(myx, myy, pred);
shading flat;
view(-10,80);

Use the XOR dataset and explore the results of (2,2,1) network for this data set.

• Is the (2,2,1) network sufficient for this data?
• Is the resulting response surface always the same? How many different 'solutions' can this network provide?

Try to set higher number of units in the hidden layer.

• What is the minimal number of units in the hidden layer that allows us to split the data perfectly?
• Visualize the result. Did you expected it?
• Try to issue the command axis([-1 2 -1 2]) after plotting the data, but before plotting the decision boundary and the response surface. Did the picture change? Is that what you expected?
• Try to increase the number of units in hidden layer (e.g. to 10), and issue the command axis([-10 11 -10 11]). What does the response surface look like? Again, did you expected it?