Main topic of today's lab is the problem of image fusion by means of image registration. There are often multiple images acquired for a single patient – this can be MR, CT at one session or a control MR few months after a previous scan, etc. For (clinical) evaluation of these data, we want the corresponding structures in the images to overlap in the best possible way. But the patient is not completely fixed in the scanner, so the position (in world coordinate system(!)) will in general be different for different scans. Restoring overlap of two image $I, J$ can be seen as finding a transform $T$ for one of the images such that an overlap error $E$ is minimized $$ T^{*} = \mathop{\arg\min}_T E(I, T(J))$$
Thus, the key concepts in image registration are the choice of transformation and the choice of the error function, typically called (image) metric.
Transformation Knowing the simplest type of transformation needed for aligning the two input images helps us to restrict the search only to a class of transformations. We may search only for
Metric Optimal image registration metric $E(I, J)$ has its global minimum when the image $I, J$ overlap perfectly. If both $I$ and $J$ are of same modality and have similar intensity values, the sum of squared distances (SSD) between pixels pairs $(\mathbf{x}, t(\mathbf{x}))$ for all pixels $\mathbf{x}$ of image $I$ is such optimal metric.
The SSD metric will not work in case the images $I, J$ have different intensities, which occurs often in medical imaging. For instance when $I$ – CT image, $J$ – MR image, or when $I, J$ are acquired with different MR sequences (T1-weighted, T2-weighted, FLAIR, etc). In this cases, the (normalized) mutual information metric can be applied.
If the intensity / color information is not suitable for defining an image metric, we can formulate the registration problem as alignment of two set of points (corresponding landmarks in both images). The criterion in this case is usually the mean distance between the transformed moving image landmarks and the fixed image landmarks.
In today's homework, we will try out different registration techniques on several pairs of images – download them in a zip-archive.
[1.5 pt] Landmark registration (Matlab) We will use MATLAB for this part. Load the images histology.HEStain.tif
and histology.PanCytokeratin.tif
that show similar histology slices in two different stain colouring. The task is to find the best transform, that will map one image on each other. Due to the differences in the image intensities, we will use manually set landmarks to compute the transform.
Ipc
be the moving, HEStain image Ihe
the fixed one.
fitgeotrans
and then warping the moving image with imwarp
nrefsim_transform = fitgeotrans(moving_points, fixed_points, 'nonreflectivesimilarity'); imwarp(Ipc, hist_nrfsim, 'OutputView', imref2d(size(Ihe)))
fitgeotrans
[2 pt] Multi-modal registration (MITK) Align the images T1.nii
and FLAIR.nii
, visualize the result and evaluate the histogram of intensities in image T1
in the lesion area (marked by LesionSeg-labels.nii
). The segmentation is aligned with the FLAIR image so pay attention to which image is set as fixed/moving. Put screenshots into your report.
Due to contrast differences of the two MR sequences, we need to use one of the MultiModal algorithms. To also correct for acquisition artifacts, we want to search for affine
transform, even though the images are from the same subject and the rigid
transform would seem to be enough.
[1.5 pt] Deformable registration (MITK) The most complex registration task is the mapping of images with different geometries. The transform in such case is a deformation (vector) field. The task is to applyit on two pairs of images.
US_heart
images, which show two different time-steps of a 3D+time acquisition of the mitral valve
Demons
) and apply to the images
MatchPoint Registration Visualizer
plugin, select the registration object and visualize also the 3D deformation (either as grid or as vector field glyphs)
We will use the MatchPoint plug-ins from MITK Workbench for registration. We start with loading the two images and visualizing their differences with the MatchPoint Registration Evaluator
. After opening the plugin, select the two images and hit Start Evaluation
. You can select different visualization styles – Blend
, Checkerboard
, Color blend
etc.
We then need to select the registration algorithm.
MatchPoint Algorithm Browser
plugin, select a suitable registration algorithm from the list. Data Manager
. The first selected image is set as moving
, the second as fixed
.Reg #n
and the transformed moving image Reg #n mapped moving data
Inspect the registration results in the MatchPoint Registration Evaluator
plugin. This time, select the registration object and Start Evaluation
.