Homework 09 - Metric reconstruction

This homework is split into two weeks.

Having a fundamental matrix F and camera calibration K, the essential matrix E can be computed from the equation

$F = K^{-\top} E K^{-1}\,.$

First, matrix E1 is computed as

$E1 = K^{\top} F K\,.$

Due to errors in data used to estimate F and/or K, the obtained matrix E1 is not a true essential matrix, its two non-zero singular values are not equal. So the matrix must be modified in order to make these singular values equal. The matrix E1 should be decomposed using the singular value decomposition, the matrix D replaced by diag(1,1,0) and correct esential matrix E is composed back.

$E1 = U D V^{\top}$

$E = U \left[\begin{array}{ccc}1&0&0\\0&1&0\\0&0&0\end{array}\right]V^{\top}$

Finally, K should be applied to compute a fundamental matrix Fe consistent with K from E (this matrix will differ from the original F).

Find two essential matrices. A possibly bad Ex and the best E

1. Compute essential matrix Ex using your best fundamental matrix F estimated in HW-08 and internal calibration from HW-04. Compute also the fundamental matrix Fx consistent with K from Ex and K
2. Draw the 12 corresponding points IX (from HW-08) in different colour in the two images. Using Fx, compute the corresponding epipolar lines and draw them into the images in corresponding colours. Export as 09_egx.pdf.
3. Draw graphs of epipolar errors d1_i and d2_i w.r.t Fx for all points. Draw both graphs into single figure (different colours) and export as 09_errorsx.pdf.
4. Find essential matrix E by minimizing the maximum epipolar error of the respective fundamental matrix Fe consistent with K using the same correspondences:
   1. Generate all 7-tuples from the set of 12 correspondences and estimate fundamental matrix F for each of them.
   2. For each tested F, compute essential matrix E using internal calibration K.
   3. Compute fundamental matrix Fe consistent with K from E and K and its epipolar error over all matches.
   4. Choose such Fe and E that minimize maximal epipolar error over all matches.
5. Draw the 12 corresponding points in different colour in the two images. Using Fe, compute the corresponding epipolar lines and draw them into the images in corresponding colours. Export as 09_eg.pdf.
6. Draw graphs of epipolar errors d1_i and d2_i w.r.t Fe for all points. Draw both graphs into single figure (different colours) and export as 09_errors.pdf.
7. Save F, Ex, Fx, E, Fe and u1, u2, point_sel_e (indices of seven points used for computing Fe) as 09a_data.mat.

1. Decompose the best E into relative rotation R and translation C (four solutions). Choose such a solution that reconstructs (most of) the points in front of both (computed) cameras.
2. Construct projective matrices P1, P2 (including K).
3. Compute scene points X.
4. Display the images, draw the input points as blue dots and the scene points X projected by appropriate P_i as red circles. Draw also the edges, connecting the original points as yellow lines. Export as 09_reprojection.pdf.
5. Draw graph of reprojection errors and export as 09_errorsr.pdf.
6. Draw the 3D point set (using 3D plotting facility) connected by the edges as a wire-frame model, shown from the top of the tower, from the side, and from some general view. Export as 09_view1.pdf, 09_view2.pdf, and 09_view3.pdf.
7. Save Fe, E, R, C, P1, P2, X, and u1, u2, point_sel_e as 09b_data.mat.

Example of the corresponding points and edges	Example of a wire-frame model reconstruction

The first part: upload an archive consisting of:

1. 09_errorsx.pdf, 09_errors.pdf
2. 09_egx.pdf, 09_eg.pdf
3. 09a_data.mat
4. hw09a.m or hw09a.py – your implementation
5. any other files required by your implementation

The second part: upload an archive consisting of:

1. 09b_data.mat
2. 09_reprojection.pdf
3. 09_errorsr.pdf
4. 09_view1.pdf, 09_view2.pdf, 09_view3.pdf
5. hw09b.m or hw09b.py – your implementation
6. other files required by your implementation