~~NOTOC~~
====== Homework 09 - Metric reconstruction =======

This homework is split into two weeks.
==== Part A: Essential matrix ====

Having a fundamental matrix ''F'' and camera calibration ''K'', the essential matrix ''E'' can be computed from the equation 

$F = K^{-\top} E K^{-1}\,.$

First, matrix ''E1'' is computed as

$E1 = K^{\top} F K\,.$
  
Due to errors in data used to estimate ''F'' and/or ''K'', the obtained matrix ''E1'' is not a true essential matrix, its two non-zero singular values are not equal. So the matrix must be modified in order to make these singular values equal. The matrix ''E1'' should be decomposed using the singular value decomposition, the matrix ''D'' replaced by diag(1,1,0) and correct esential matrix ''E'' is composed back.

$E1 = U D V^{\top}$

$E = U \left[\begin{array}{ccc}1&0&0\\0&1&0\\0&0&0\end{array}\right]V^{\top}$
  
Finally, ''K'' should be applied to compute a fundamental matrix ''Fe'' consistent with ''K'' from ''E'' (this matrix will differ from the original ''F'').

=== Steps ===

Find two essential matrices. A possibly bad ''Ex'' and the best ''E''
  - Compute essential matrix ''Ex'' using your best fundamental matrix ''F'' estimated in HW-08 and internal calibration from HW-04. Compute also the fundamental matrix ''Fx'' consistent with ''K'' from ''Ex'' and ''K''
   - Draw the 12 corresponding points IX (from HW-08) in different colour in the two images. Using ''Fx'', compute the corresponding epipolar lines and draw them into the images in corresponding colours. Export as ''09_egx.pdf''.
   - Draw graphs of epipolar errors ''d1_i'' and ''d2_i'' w.r.t ''Fx'' for all points. Draw both graphs into single figure (different colours) and export as ''09_errorsx.pdf''.
  - Find essential matrix ''E'' by minimizing the maximum epipolar error of the respective fundamental matrix ''Fe'' consistent with ''K'' using the same correspondences:
    - Generate all 7-tuples from the set of 12 correspondences and estimate fundamental matrix ''F'' for each of them.
    - For each tested ''F'', compute essential matrix ''E'' using internal calibration ''K''.
    - Compute fundamental matrix ''Fe'' consistent with ''K'' from ''E'' and ''K'' and **its** epipolar error over all matches.
    - Choose such ''Fe'' and ''E'' that minimize maximal epipolar error over all matches. 
  - Draw the 12 corresponding points in different colour in the two images. Using ''Fe'', compute the corresponding epipolar lines and draw them into the images in corresponding colours. Export as ''09_eg.pdf''.
  - Draw graphs of epipolar errors ''d1_i'' and ''d2_i'' w.r.t ''Fe'' for all points. Draw both graphs into single figure (different colours) and export as ''09_errors.pdf''.
  - Save ''F'', ''Ex'', ''Fx'', ''E'', ''Fe'' and ''u1'', ''u2'', ''point_sel_e'' (indices of seven points used for computing ''Fe'') as ''09a_data.mat''.

==== Part B: Cameras and reconstruction ====
  - Decompose  the best ''E'' into relative rotation ''R'' and translation ''C'' (four solutions). Choose such a solution that reconstructs (most of) the points in front of both (computed) cameras.
  - Construct projective matrices ''P1'', ''P2'' (including ''K'').
  - Compute scene points ''X''.
  - Display the images, draw the input points as blue dots and the scene points ''X'' projected by appropriate ''P_i'' as red circles. Draw also the edges, connecting the original points as yellow lines.  Export as ''09_reprojection.pdf''.
  - Draw graph of reprojection errors and export as ''09_errorsr.pdf''.
  - Draw the 3D point set (using 3D plotting facility) connected by the edges as a wire-frame model, shown from the top of the tower, from the side, and from some general view. Export as  ''09_view1.pdf'', ''09_view2.pdf'', and ''09_view3.pdf''.
  - Save ''Fe'', ''E'', ''R'', ''C'', ''P1'', ''P2'', ''X'', and ''u1'', ''u2'', ''point_sel_e'' as ''09b_data.mat''. 

^  Example of the corresponding points and edges   ^  Example of a wire-frame model reconstruction  ^
|  {{courses:gvg:labs:09_mesh.jpg?direct&400|}}  |  {{courses:gvg:labs:11_projection.gif|}}  |

==== Upload ====

The first part: upload an archive consisting of:
  - ''09_errorsx.pdf'', ''09_errors.pdf''
  - ''09_egx.pdf'', ''09_eg.pdf''
  - ''09a_data.mat''
  - ''hw09a.m'' or ''hw09a.py'' – your implementation
  - any other files required by your implementation

The second part: upload an archive consisting of:
  - ''09b_data.mat''
  - ''09_reprojection.pdf''
  - ''09_errorsr.pdf''
  - ''09_view1.pdf'', ''09_view2.pdf'', ''09_view3.pdf''
  - ''hw09b.m'' or ''hw09b.py'' – your implementation
  - other files required by your implementation