====== Exercise 5 ======
Program:
  * Introduction to Netlab toolbox
  * Classification of the wedge dataset using Netlab
  * Classification of the XOR dataset using Netlab

Downloads:
  * [[http://www1.aston.ac.uk/eas/research/groups/ncrg/resources/netlab/downloads/|The Netlab toolbox]]
  * {{:courses:y33aui:cviceni:datawedge.txt|Wedge dataset}}
  * {{:courses:y33aui:cviceni:dataxor.txt|XOR dataset}}

**Likely, you will not manage to finish the implementation of all the functions during the excercise in the lab. Finish them as a home work.**

===== Netlab installation =====
Download the Netlab toolbox, unpack it to a directory, and set the path to the directory in MATLAB.

  * What is the puprose of the following functions:
    * ''mlp()''
    * ''netopt()''
    * ''mlpfwd()''
  * Do you understand the meaning of their arguments?
  * In what format does the toolbox expect the data? Columns or rows? Is it the same format we used in previous exercises? 
  * Do you see the analogy with functions you created last week?

===== The wedge dataset =====
Repeat the task from the last week: train a simple neural network (2,2,1) on the wedge dataset.

  * What types of transfer functions are available in Netlab? Can they be used freely for all neurons? What setting is suitable for our data?
  * Try changing the optimization algorithm in ''netopt()''. Does the change affect the speed of learning? Does it affect the quality of learning?
  * Try to visualize the results:
    * plot the training data and the decision boundary
    * plot the response of the network over the definition range

For plotting, you can use the following code snippets. For the decision boundary:

<code matlab>
%% Plot the decision boundary
% Assuming the data are already plotted in current figure.
% Get the current axis limits
ax = axis;
% Prepare the mesh of points we will use to test the network
myx = ax(1):(ax(2)-ax(1))/100:ax(2);
myy = ax(3):(ax(4)-ax(3))/100:ax(4);
[xx,yy] = meshgrid(myx,myy);
% Prepare the data to the right format
data = [xx(:) yy(:)];
% Ask the NN for predictions
pred = mlpfwd(net, data);
% Reshape the predictions to the original matrix size
pred = reshape(pred, numel(myx), numel(myy));
% Draw the contour line - the decision boundary
[foo,c] = contour(myx, myy, pred, [0.5 0.5]);
</code>

And for the network response (assuming ''myx'', ''myy'', and ''pred'' come from the previous code snippet):

<code matlab>
surf(myx, myy, pred);
shading flat;
view(-10,80);
</code>

===== The XOR dataset =====
Use the XOR dataset and explore the results of (2,2,1) network for this data set.
  * Is the (2,2,1) network sufficient for this data?
  * Is the resulting response surface always the same? How many different 'solutions' can this network provide?

Try to set higher number of units in the hidden layer.
  * What is the minimal number of units in the hidden layer that allows us to split the data perfectly?
  * Visualize the result. Did you expected it?
  * Try to issue the command ''axis([-1 2 -1 2])'' after plotting the data, but before plotting the decision boundary and the response surface. Did the picture change? Is that what you expected?
  * Try to increase the number of units in hidden layer (e.g. to 10), and issue the command ''axis([-10 11 -10 11])''. What does the response surface look like? Again, did you expected it?