Search
Main topic of today's lab is the problem of image fusion by means of image registration. There are often multiple images acquired for a single patient – this can be MR, CT at one session or a control MR few months after a previous scan, etc. For (clinical) evaluation of these data, we want the corresponding structures in the images to overlap in the best possible way. But the patient is not completely fixed in the scanner, so the position (in world coordinate system(!)) will in general be different for different scans. Restoring overlap of two image $I, J$ can be seen as finding a transform $T$ for one of the images such that an overlap error $E$ is minimized $$ T^{*} = \mathop{\arg\min}_T E(I, T(J))$$
Thus, the key concepts in image registration are the choice of transformation and the choice of the error function, typically called (image) metric.
Transformation Knowing the simplest type of transformation needed for aligning the two input images helps us to restrict the search only to a class of transformations. We may search only for
Metric Optimal image registration metric $E(I, J)$ has its global minimum when the image $I, J$ overlap perfectly. If both $I$ and $J$ are of same modality and have similar intensity values, the sum of squared distances (SSD) between pixels pairs $(\mathbf{x}, t(\mathbf{x}))$ for all pixels $\mathbf{x}$ of image $I$ is such optimal metric.
The SSD metric will not work in case the images $I, J$ have different intensities, which occurs often in medical imaging. For instance when $I$ – CT image, $J$ – MR image, or when $I, J$ are acquired with different MR sequences (T1-weighted, T2-weighted, FLAIR, etc). In this cases, the (normalized) mutual information metric can be applied.
If the intensity / color information is not suitable for defining an image metric, we can formulate the registration problem as alignment of two set of points (corresponding landmarks in both images). The criterion in this case is usually the mean distance between the transformed moving image landmarks and the fixed image landmarks.
In today's homework, we will try out different registration techniques on several pairs of images – download them in a zip-archive.
[1.5 pt] Landmark registration (Matlab) We will use MATLAB for this part. Load the images histology.HEStain.tif and histology.PanCytokeratin.tif that show similar histology slices in two different stain colouring. The task is to find the best transform, that will map one image on each other. Due to the differences in the image intensities, we will use manually set landmarks to compute the transform.
histology.HEStain.tif
histology.PanCytokeratin.tif
Ipc
Ihe
fitgeotrans
imwarp
nrefsim_transform = fitgeotrans(moving_points, fixed_points, 'nonreflectivesimilarity'); imwarp(Ipc, hist_nrfsim, 'OutputView', imref2d(size(Ihe)))
[2 pt] Multi-modal registration (MITK) Align the images T1.nii and FLAIR.nii, visualize the result and evaluate the histogram of intensities in image T1 in the lesion area (marked by LesionSeg-labels.nii). The segmentation is aligned with the FLAIR image so pay attention to which image is set as fixed/moving. Put screenshots into your report.
T1.nii
FLAIR.nii
T1
LesionSeg-labels.nii
Due to contrast differences of the two MR sequences, we need to use one of the MultiModal algorithms. To also correct for acquisition artifacts, we want to search for affine transform, even though the images are from the same subject and the rigid transform would seem to be enough.
affine
rigid
[1.5 pt] Deformable registration (MITK) The most complex registration task is the mapping of images with different geometries. The transform in such case is a deformation (vector) field. The task is to applyit on two pairs of images.
US_heart
Demons
MatchPoint Registration Visualizer
We will use the MatchPoint plug-ins from MITK Workbench for registration. We start with loading the two images and visualizing their differences with the MatchPoint Registration Evaluator. After opening the plugin, select the two images and hit Start Evaluation. You can select different visualization styles – Blend, Checkerboard, Color blend etc.
MatchPoint Registration Evaluator
Start Evaluation
Blend
Checkerboard
Color blend
We then need to select the registration algorithm.
MatchPoint Algorithm Browser
MatchPoint Algorithm Control
Selection
Load selected algorithm
Execution
Data Manager
moving
fixed
Start algorithm
Reg #n
Reg #n mapped moving data
Inspect the registration results in the MatchPoint Registration Evaluator plugin. This time, select the registration object and Start Evaluation.