The Minimax Task

The Bayesian formulation has some limits discussed at the lecture and for certain situation one may need to formulate the problem e.g. as a Neyman-Pearson task, Wald task or minimax task. Out of this spectrum of problem formulations we will examine the minimax task.

To continue with our simple OCR developed in the previous labs, lets consider a situation when we are trying to recognise licence plates. For simplicity we again stick to the simple two-letter alphabet and 10×10 already segmented letters. The situation in the Bayes task then corresponds to a problem of designing a recognition system for a particular crossroad where the a priori probability of the two letters is known or could be measured.

In today's assignment we aim at designing a recognition system which could be deployed anywhere in the Czech republic and we need to take into account that licence plates in different cities use different combination of the letters, so the a priori probabilities differ as well. Further, lets assume it is not practical to find the optimal Bayesian strategy for each used camera.

As you might have already guessed, the situation is ideal for the minimax strategy. Here, the a priori probability exists, but is not known. We will first study the task rather theoretically to demonstrate some principles from the lecture and then we apply the gained knowledge to the real task of recognising the letters on the licence plates.

As in the last labs, use the discrete and continuous measurements $x$ computed by compute_measurement_lr_discrete.m and compute_measurement_lr_cont.m.

For the experiments, choose two letters corresponding to your initials and use the distribution parameters given in the following variables (assuming your name is Chuck Norris) for the discrete measurements tasks:

and the following ones for the cases with continuous measurements:

(still assuming you are Chuck Norris)

1. Use the discrete distributions and for $p_K(C) \in \{0, 0.01, \ldots, 0.99, 1\}$ compute and plot into one figure the following:
   1. Risk of the Bayesian strategy optimal for each $p_K(C)$.
      
      Hint 1: Use your code from the last labs.
      
      Hint 2: In the minimax problem formulation, each error is penalised equally independent of the class (this corresponds to the 0-1 loss function).
      
      
   2. Find the optimal strategy for $p_K(C) = 0.25$. Then for this, now fixed strategy, alter the a priori probability and compute its Bayesian risk. For this complete the template function risk_fix_q_discrete.
      
      Hint 1: This is what happens, when we have assumed wrong a priori probabilities different from the real one.
      Hint 2: Equation (3) from [1] explains the shape of the observed curve.
      
      
   3. For each iterated $p_K(C)$ derive the optimal Bayesian strategy and compute its worst-case risk in case the true a priori probability is different. For this complete the template function worst_risk_discrete.
      
      Hint: You can again use the equation (3) from [1] and the fact that the a priori probability is bounded to the <0,1> interval.
      
      
2. Save the figure to plots_discrete.png.
   
   
   Assuming your name is Mirek Dušín. Chuck Norris initials refused to be used in this graph.
   
   
3. Repeat the steps 1 and 2 for the continuous distributions and save the figure to plots_cont.png. The implemented functions are the same but with _cont instead of _discrete suffix.
   
   
   Assuming your name is Mirek Dušín. Chuck Norris initials refused to be used in this graph.
   
   
4. Complete the template functions minmax_strategy_discrete.m and minmax_strategy_cont.m, so that they find the minmax strategy and its risk for discrete and continuous measurements respectively. Hint: The minimax strategy is the Bayesian strategy with minimal maximal Bayesian risk over all $p_K(C)$.
   
   Hint: Use following to find a minimizer of a function:

   x_minimizer = fminbnd(@(x) some_function(other_param1, other_param2, x), 0, 1)
   % where x is a variable parameter of your function some_function(other_param1, other_param2, x)
   % x_minimizer is x such that some_function returns local minimum
    
   % see 'doc fminbnd' section Arguments 

   In case of multiple minima there is a minimax strategy ambiguity. By using fminbnd you get the results as expected from the upload system.
   
   Python hint: use scipy.optimize.fminbound
   
   

    >> D1 = discrete.C;
   D2 = discrete.N;
   [q risk] = minmax_strategy_discrete(D1, D2)
    
   q =  1     1     1     1     1     1     1     1     1     2     2     2     2     2     1     1     1     1     1     1     1
    
   risk = 0.0286
    
   >> D1 = cont.C;
   D2 = cont.N;
   [q risk] = minmax_strategy_cont(D1, D2)
    
   q =       t1: -775.9832
             t2: 1.0080e+04
       decision: [1 2 1]
    
   risk = 0.0132

5. Plot bayesian risk for all values of $p_K(C)$ for the minimax strategy in both discrete and continuos cases.
   
   
6. Generate a test set for your initials using create_test_set.m.
7. Compute the error of the strategy on the generated test images, show the final classification in one figure and save it as minmax_classif_cont.png and minmax_classif_discrete.png.
   
   Hint: use the classification_error_discrete and classification_error_2normal functions from the bayes assignment.

   >> error_discrete = classification_error_discrete( images_test_set, labels_test_set, q_minimax_discrete)
    
   error_discrete =  0.0667
    
   >> error_cont = classification_error_2normal( images_test_set, labels_test_set, q_minimax_cont)
    
   error_cont =  0

   

Fill the correct answers to your answers.txt file.

This task is not compulsory.

Go through the chapter on non-Bayesian tasks in SH10 book [2], especially the parts discussing solution of the minimax task by linear programming (pages 25, 31, 35-38, 40-41, 46-47). Solve the above classification task using linear programming as described on page 47.

Hints:

• Work with the discrete measurements $x$.
• Represent the sought strategy (classification function) as a table α, with α(i,k) corresponding to the probability of classification of bin 'i' to class 'k' such that α(i,k) >= 0 and sum_k α(i,k) = 1.
• Reformulate the task in a matrix form according to equation 2.53 in SH10, page 47
• Solve the task numerically using linprog function (available in the MATLAB optimization toolbox installed in E132, E220, E230 or in MOSEK optimalization toolbox available under free academic license)
• Compare the obtained results with the results of the classification above.

• [1] Minimax Task (short support text for labs)
• [2] Michail I. Schlesinger, Vaclav Hlavac. Ten Lectures on Statistical and Structural Pattern Recognition. Kluwer Academic Publishers, 2002.
• [3] slides of prof. Matas