===== Lab 08 : Image Registration =====

Main topic of today's lab is the problem of image fusion by means of //image registration//. There are often multiple images acquired for a single patient -- this can be MR, CT at one session or a control MR few months after a previous scan, etc. For (clinical) evaluation of these data, we want the corresponding structures in the images to overlap in the best possible way. But the patient is not completely fixed in the scanner, so the position (in //world coordinate system//(!)) will in general be different for different scans. Restoring overlap of two image $I, J$ can be seen as finding a transform $T$ for one of the images such that an //overlap error// $E$ is minimized
$$ T^{*} = \mathop{\arg\min}_T E(I, T(J))$$

Thus, the //key concepts// in image registration are the choice of **transformation** and the choice of the error function, typically called **(image) metric**. 

** Transformation ** Knowing the simplest type of transformation needed for aligning the two input images helps us to restrict the search only to a //class// of transformations. We may search only for 
  * //translation// if the images are only shifted to each other, 
  * //tr. and rotation// (rigid transform) for transforming rigid objects (objects not changing in size)
  * //tr., rot. and scaling, shearing// (affine transform) for arbitrary linear transforms
  * //deformation vector field// the most general case, the image may be locally deformed

** Metric ** Optimal image registration metric $E(I, J)$ has its global minimum when the image $I, J$ overlap perfectly. If both $I$ and $J$ are of same modality and have similar intensity values, the //sum of squared distances (SSD)// between pixels pairs $(\mathbf{x}, t(\mathbf{x}))$ for all pixels $\mathbf{x}$ of image $I$ is such optimal metric. 
/*
 $$ SSD(I, J, t) = \sum_{\mathbf{x} \in \Omega(I)} [I(\mathbf{x}) - J(t(\mathbf{x})) ]^2$$
*/

The SSD metric will not work in case the images $I, J$ have different intensities, which occurs often in medical imaging. For instance when $I$ -- CT image, $J$ -- MR image, or when $I, J$ are acquired with different MR sequences (T1-weighted, T2-weighted, FLAIR, etc). In this cases, the (normalized) //mutual information metric// can be applied.

If the intensity / color information is not suitable for defining an image metric, we can formulate the registration problem as alignment of two set of points (corresponding //landmarks// in both images). The criterion in this case is usually the mean distance between the transformed moving image landmarks and the fixed image landmarks. 

In today's homework, we will try out different registration techniques on several pairs of images -- download them in a {{ :courses:zsl:registration_images.zip | zip-archive}}.

==== Homework ====

** [1.5 pt] Landmark registration (Matlab)** We will use MATLAB for this part. Load the images ''histology.HEStain.tif'' and ''histology.PanCytokeratin.tif'' that show similar histology slices in two different stain colouring. The task is to find the best transform, that will map one image on each other. Due to the differences in the image intensities, we will use manually set landmarks to compute the transform.

   - call the {{https://www.mathworks.com/help/images/ref/cpselect.html| cpselect tool}} and set corresponding landmarks, when closing the tool, you will be prompted to store the points -- we will need them for computing the transformation. Let the PanCytokreatin image ''Ipc'' be the moving, HEStain image ''Ihe'' the fixed one.
   - compute the transformed moving image by first estimating the transform ''fitgeotrans'' and then warping the moving image with ''imwarp'' <code matlab> 
    nrefsim_transform = fitgeotrans(moving_points, fixed_points, 'nonreflectivesimilarity');
    imwarp(Ipc, hist_nrfsim, 'OutputView', imref2d(size(Ihe)))
</code>
   - repeat step 2 for (at least 3) different choices of transforms available in ''fitgeotrans''
   - use {{https://www.mathworks.com/help/images/ref/imshowpair.html| imshowpair}} function or the provided {{courses:zsl:image_checkerboard.m|image_checkerboard}} to make joint visualization of the fixed and the transformed (warped) moving image. Place the visualizations for all tested transform types into your report.
   - Comment on the results and decide which of the tested transforms provided the best alignment of the two images

** [2 pt] Multi-modal registration (MITK)** Align the images ''T1.nii'' and ''FLAIR.nii'', visualize the result and evaluate the histogram of intensities in image ''T1'' in the lesion area (marked by ''LesionSeg-labels.nii''). The segmentation is aligned with the FLAIR image so pay attention to which image is set as fixed/moving. Put screenshots into your report. 

Due to contrast differences of the two MR sequences, we need to use one of the MultiModal algorithms. To also correct for acquisition artifacts, we want to search for ''affine'' transform, even though the images are from the same subject and the ''rigid'' transform would seem to be enough.

  * **Pipeline**
    - Load images, create a joint visualization before registration, put screenshot into report
    - Select the proper MultiModal registration algorithm and perform registration
    - Visualize the result by a method of your choice (checkerboard, color blend, ...) and insert screenshot into report
    - Compute the value histogram of T1 image in the lesion region (MITK plug-in //Statistics//)
 

** [1.5 pt] Deformable registration (MITK)** The most complex registration task is the mapping of images with different geometries. The transform in such case is a deformation (vector) field. The task is to applyit on two pairs of images. 

  * **Input data**
    - take two of the ''US_heart'' images, which show two different time-steps of a 3D+time acquisition of the mitral valve
  * **Pipeline**
    - Load images and create a joint visualization before registration (screenshot into report)
    - Select the deformable registration algorithm (''Demons'') and apply to the images
    - Visualize the result with the joint visualization
    - Inspect the result: Did the registration work perfectly? Are there some areas with high error (mismatch)? Does the resulting image
    - Create a visualization of the deformation: open the ''MatchPoint Registration Visualizer'' plugin, select the registration object and visualize also the 3D deformation (either as grid or as vector field glyphs)
{{ :courses:zsl:reggrid.png?600 |}}



==== Help: Registration in MITK ====

We will use the //MatchPoint// plug-ins from MITK Workbench for registration. We start with loading the two images and visualizing their differences with the ''MatchPoint Registration Evaluator''. After opening the plugin, select the two images and hit ''Start Evaluation''. You can select different visualization styles -- ''Blend'', ''Checkerboard'', ''Color blend'' etc.{{ :courses:zsl:regeval.png?600 |}}

We then need to select the registration algorithm. 
  - Open the ''MatchPoint Algorithm Browser'' plugin, select a suitable registration algorithm from the list. 
  - Open the ''MatchPoint Algorithm Control'' plugin, the ''Selection'' tab shows the algorithm selected in previous step, hit ''Load selected algorithm''. The plugin switches automatically to the tab ''Execution'' after the algorithm is loaded.{{ :courses:zsl:regload.png?600 |}}
  - Select the two images in ''Data Manager''. The first selected image is set as ''moving'', the second as ''fixed''.
  - Now hit ''Start algorithm'' button to run the registration {{ :courses:zsl:regstart.png?600 |}}
  - Two new data objects should appear, the //registration object// ''Reg #n'' and the //transformed moving image// ''Reg #n mapped moving data''

Inspect the registration results in the ''MatchPoint Registration Evaluator'' plugin. This time, select the registration object and ''Start Evaluation''.