Homework 06 - Panorama

Input Images

Consider the images were captured with the following EXIF info (focal length is in milimeters):

Tag Name Value
0xa002 ExifImageWidth 2400
0xa003 ExifImageHeight 1800
0xa20e FocalPlaneXResolution 2160000/225
0xa20f FocalPlaneYResolution 1611200/168
0xa210 FocalPlaneResolutionUnit inch
0x920a FocalLength 7400/1000

The images were then resized.

07 06 05 04 - reference 03 02 01
bridge_07.jpgbridge_06.jpgbridge_05.jpg bridge_04.jpg bridge_03.jpgbridge_02.jpgbridge_01.jpg

Steps

  1. Download the images above.
  2. Download point correspondences between every pair of adjacent images (10 matches each) from the upload system (InputData). The correspondences is stored in a single array (cell matrix in matlab), such that for image pair i-j, the points in the image i are in U[i-1,j-1] (python) or U{i,j} (matlab) and in the image j are in U[j-1,i-1] or U{j,i}.
  3. Find homographies H_ij for every pair of adjacent images by the same method (optimizing 4 over 10) as in HW-05; use your u2h_optim function for each homography.
  4. Draw graph of transfer errors, sorted from lower to higher, of all neighboring pairs into a single image. Use different colour for every image pair, export as 06_errors.pdf.
  5. Compute homographies H_i4 for every i ∈ <1,7> that maps the images above into the reference image 04 (thus H_44 is identity). Also construct inverse homographies H_4i.
  6. Plot the image borders of the images 02 to 06 transformed by appropriate homography to the image plane of the image 04. Export as 06_borders.pdf.
  7. Construct a projective panoramic image from the images 03, 04, and 05, in the image plane of the image 04. Save as 06_panorama.png.
  8. Construct calibration matrix K using the actual image size and the original EXIF. All the images share the same calibration. Store it in 06_data.mat
  9. Establish a transformation between each image plane and the coordinate system of the cylinder, described below.
  10. Plot the image borders of all the images mapped onto the cylinder (06_borders_c.pdf).
  11. Construct a panoramic image from all the images mapped onto the cylinder (06_panorama_c.png).
Example plot of errors
Example of image borders (origin not shifted)
Example of projective panorama (images 3-4-5)
Example of cylindrical panorama (all images)

The cylinder

The cylinder is defined in the coordinate system γ (blue) of the image 04 (green) such that:

  • cylinder axis is equal to c2 axis of γ,
  • image plane of the image 04 is tangential to the cylinder,
  • image coordinate system defined on the cylinder surface has square pixels and equal horizontal resolution in the line where it touches the image 04.

This leads that the cylinder has radius = 1 in the γ system. The cylinder surface is a set of points [x_{\gamma} y_{\gamma} z_{\gamma}]^\top such that

x_{\gamma}^2 + z_{\gamma}^2 = 1

For parametrisation of the cylinder surface we define two coordinate systems: the first (magenta) is based on γ, second (red) is pixel-based system of panoramic image.

First, the surface of the cylinder is parametrised by circumference length (equal to angle since radius=1) and by the y_{\gamma} coordinate. The ray \lambda [x_{\gamma} y_{\gamma} z_{\gamma}]^\top intersects the cylinder in the point

\lambda=\frac{1}{\sqrt{x_{\gamma}^2 + z_{\gamma}^2}}

[ a_{c\gamma}; y_{c\gamma} ] = [ \mbox{the angle given by } x_{\gamma} \mbox{ and } z_{\gamma}; \frac{y_{\gamma}}{\sqrt{x_{\gamma}^2 + z_{\gamma}^2}} ]

Second, the pixel size must be applied and image origin adjusted. Since the horizontal pixel size is 1/K(1,1) measured in units given by γ system, this leads to

[ x_c; y_c ] = K_{11}  [ a_{c\gamma}; y_{c\gamma} ] + [x_{0c}; y_{0c} ]

where the origin [x_{0c}; y_{0c}] must be found such that all pixels from the input images will fit into the resulting panorama.

Upload

Upload an archive consisting of:

  1. 06_data.mat
  2. 06_histograms.pdf
  3. 06_borders.pdf
  4. 06_panorama.png
  5. 06_borders_c.pdf
  6. 06_panorama_c.png
  7. hw06.m or hw06.py script solving the task, and any other files needed by the script
courses/gvg/labs/hw-06.txt · Last modified: 2020/03/31 22:07 by xmatousm