Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Abstract This paper presents a texture segmentation algorithm inspired by the multi-channel filtering theory for visual information processing in the early stages of human visual system. The channels are characterized by a bank of Gabor filters that nearly uniformly covers the spatial-frequency domain, and a systematic filter selection scheme is proposed, which is based on reconstruction of the input image from the filtered images. Texture features are obtained by subjecting each (selected) filtered image to a nonlinear transformation and computing a measure of “energy” in a window around each pixel. A square-error clustering algorithm is then used to integrate the feature images and produce a segmentation. A simple procedure to incorporate spatial information in the clustering process is proposed. A relative index is used to estimate the “true” number of texture categories. <s> BIB001 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> A new information-theoretic approach is presented for finding the registration of volumetric medical images of differing modalities. Registration is achieved by adjustment of the relative position and orientation until the mutual information between the images is maximized. In our derivation of the registration procedure, few assumptions are made about the nature of the imaging process. As a result the algorithms are quite general and can foreseeably be used with a wide variety of imaging devices. This approach works directly with image data; no pre-processing or segmentation is required. This technique is, however, more flexible and robust than other intensity-based techniques like correlation. Additionally, it has an efficient implementation that is based on stochastic approximation. Experiments are presented that demonstrate the approach registering magnetic resonance (MR) images with computed tomography (CT) images, and with positron-emission tomography (PET) images. Surgical applications of the registration method are described. <s> BIB002 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> The information theoretic measure of mutual information has been successfully applied to multi-modality medical image registration for several applications. There remain however; modality combinations for which mutual information derived from the occurrence of image intensities alone does not provide a distinct optimum at true registration. The authors propose an extension of the technique through the use of an additional information channel supplying region labelling information. These labels which can specify simple regional connectivity or express higher level anatomical knowledge, can be derived from the images being registered. The authors show how the mutual information measure can be extended to include an additional channel of region labelling, and demonstrate the effectiveness of this technique for the registration of MR and PET images of the pelvis. <s> BIB003 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> A new approach to the problem of multimodality medical image registration is proposed, using a basic concept from information theory, mutual information (MI), or relative entropy, as a new matching criterion. The method presented in this paper applies MI to measure the statistical dependence or information redundancy between the image intensities of corresponding voxels in both images, which is assumed to be maximal if the images are geometrically aligned. Maximization of MI is a very general and powerful criterion, because no assumptions are made regarding the nature of this dependence and no limiting constraints are imposed on the image content of the modalities involved. The accuracy of the MI criterion is validated for rigid body registration of computed tomography (CT), magnetic resonance (MR), and photon emission tomography (PET) images by comparison with the stereotactic registration solution, while robustness is evaluated with respect to implementation issues, such as interpolation and optimization, and image content, including partial overlap and image degradation. Our results demonstrate that subvoxel accuracy with respect to the stereotactic reference solution can be achieved completely automatically and without any prior segmentation, feature extraction, or other preprocessing steps which makes this method very well suited for clinical applications. <s> BIB004 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Over the last five years, new “voxel-based” approaches have allowed important progress in multimodal image registration, notably due to the increasing use of information-theoretic similarity measures. Their wide success has led to the progressive abandon of measures using standard image statistics (mean and variance). Until now, such measures have essentially been based on heuristics. In this paper, we address the determination of a new measure based on standard statistics from a theoretical point of view. We show that it naturally leads to a known concept of probability theory, the correlation ratio. In our derivation, we take as the hypothesis the functional dependence between the image intensities. Although such a hypothesis is not as general as possible, it enables us to model the image smoothness prior very easily. We also demonstrate results of multimodal rigid registration involving Magnetic Resonance (MR), Computed Tomography (CT), and Positron Emission Tomography (PET) images. These results suggest that the correlation ratio provides a good trade-off between accuracy and robustness. <s> BIB005 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> The registration of multimodal medical images is an important tool in surgical applications, since different scan modalities highlight complementary anatomical structures. We present a method of computing the best rigid registration of pairs of medical images of the same patient. The method uses prior information on the expected joint intensity distribution of the images when correctly aligned, given a priori registered training images. We discuss two methods of modeling the joint intensity distribution of the training data, mixture of Gaussians and Parzen windowing. The fitted Gaussians roughly correspond to various anatomical structures apparent in the images and provide a coarse anatomical segmentation of a registered image pair. Given a novel set of unregistered images, the algorithm computes the best registration by maximizing the log likelihood of the two images, given the transformation and the prior joint intensity model. Results aligning SPGR and dual-echo MR scans demonstrate that this algorithm is a fast registration method with a large region of convergence and sub-voxel registration accuracy. <s> BIB006 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Abstract This paper is concerned with the development of entropy-based registration criteria for automated 3D multi-modality medical image alignment. In this application where misalignment can be large with respect to the imaged field of view, invariance to overlap statistics is an important consideration. Current entropy measures are reviewed and a normalised measure is proposed which is simply the ratio of the sum of the marginal entropies and the joint entropy. The effect of changing overlap on current entropy measures and this normalised measure are compared using a simple image model and experiments on clinical image data. Results indicate that the normalised entropy measure provides significantly improved behaviour over a range of imaged fields of view. <s> BIB007 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> This paper introduces two important issues of image registration. At first we want to recall the very general definition of mutual information that allows the choice of various feature spaces to perform image registration. Second we discuss the problem of finding the global maximum in an arbitrary feature space. We used a very general parallel, distributed memory, genetic optimization which turned out to be very robust. We restrict the examples to the context of multi-modal medical image registration but we want to point out that the approach is very general and therefore applicable to a wide range of other applications. The registration algorithm was analysed on a LINUX cluster. <s> BIB008 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Non-rigid registration of multi-modality images is an important tool for assessing temporal and structural changesbetween images. For rigid registration, voxel similarity measures like mutual information have been shown to alignimages from different modalities accurately and robustly. For non-rigid registration, mutual information can besensitive to local variations of intensity which in MR images may be caused by RF inhomogeneity. The reasonfor the sensitivity of mutual information towards intensity variations stems from the fact that mutual informationignores any spatial information. In this paper we propose an extension of the mutual information framework whichincorporates spatial information about higher-order image structure into the registration process and has the potentialto improve the accuracy and robustness of non-rigid registration in the presence of intensity variations. We haveapplied the non-rigid registration algorithm to a number of simulated MR brain images of a digital phantom whichhave been degraded by a simulated intensity shading and a known deformation. In addition, we have applied thealgorithm for the non-rigid registration of eight pre- and post-operative brain MR images which were acquired withan interventional MR scanner and therefore have substantial intensity shading due to RF field inhomogeneities. Inall cases the second-order estimate of mutual information leads to robust and accurate registration. <s> BIB009 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Mutual information has developed into an accurate measure for rigid and affine monomodality and multimodality image registration. The robustness of the measure is questionable, however. A possible reason for this is the absence of spatial information in the measure. The present paper proposes to include spatial information by combining mutual information with a term based on the image gradient of the images to be registered. The gradient term not only seeks to align locations of high gradient magnitude, but also aims for a similar orientation of the gradients at these locations. Results of combining both standard mutual information as well as a normalized measure are presented for rigid registration of three-dimensional clinical images [magnetic resonance (MR), computed tomography (CT), and positron emission tomography (PET)]. The results indicate that the combined measures yield a better registration function does mutual information or normalized mutual information per se. The registration functions are less sensitive to low sampling resolution, do not contain incorrect global maxima that are sometimes found in the mutual information function, and interpolation-induced local minima can be reduced. These characteristics yield the promise of more robust registration measures. The accuracy of the combined measures is similar to that of mutual information-based methods. <s> BIB010 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> A new approach is presented for elastic registration of medical images, and is applied to magnetic resonance images of the brain. Experimental results demonstrate remarkably high accuracy in superposition of images from different subjects, thus enabling very precise localization of morphological characteristics in population studies. There are two major novelties in the proposed algorithm. First, it uses an attribute vector, i.e. a set of geometric moment invariants that is defined on each voxel in an image, to reflect the underlying anatomy at different scales. The attribute vector, if rich enough, can distinguish between different parts of an image, which helps establish anatomical correspondences in the deformation procedure. This is a fundamental deviation of our method, referred to as HAMMER, from other volumetric deformation methods, which are typically based on maximizing image similarity. Second, in order to avoid being trapped by local minima, i.e. suboptimal poor matches, HAMMER uses a successive approximation of the energy function being optimized by lower dimensional energy functions, which are constructed to have significantly fewer local minima. This is achieved by hierarchically selecting features that have distinct attribute vectors, thus drastically reducing ambiguity in finding correspondence. A number of experiments in this paper have demonstrated excellent performance. <s> BIB011 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> We present a new image-based technique to rigidly register intraoperative three-dimensional ultrasound (US) with preoperative magnetic resonance (MR) images. Automatic registration is achieved by maximization of a similarity measure which generalizes the correlation ratio, and whose novelty is to incorporate multivariate information from the MR data (intensity and gradient). In addition, the similarity measure is built upon a robust intensity-based distance measure, which makes it possible to handle a variety of US artifacts. A cross-validation study has been carried out using a number of phantom and clinical data. This indicates that the method is quite robust and that the worst registration errors are of the order of the MR image resolution. <s> BIB012 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Abstract Multimodal medical images are often of too different a nature to be registered on the basis of the image grey values only. It is the purpose of this paper to construct operators that extract similar structures from these images that will enable rigid registration by simple grey value based methods, such as maximization of cross-correlation. These operators can be constructed using only basic morphological tools such as erosion and dilation. Simple versions of these operators are easily implemented on any computer system. We will show that accurate registration of images of various modalities (MR, CT, SPECT and PET) can be obtained using this approach. <s> BIB013 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Automatic registration of multimodal images involves algorithmically estimating the coordinate transformation required to align the data sets. Most existing methods in the literature are unable to cope with registration of image pairs with large nonoverlapping field of view (FOV). We propose a robust algorithm, based on matching dominant local frequency image representations, which can cope with image pairs with large nonoverlapping FOV. The local frequency representation naturally allows for processing the data at different scales/resolutions, a very desirable property from a computational efficiency view point. Our algorithm involves minimizing-over all rigid/affine transformations--the integral of the squared error (ISE or L2 E) between a Gaussian model of the residual and its true density function. The residual here refers to the difference between the local frequency representations of the transformed (by an unknown transformation) source and target data. We present implementation results for image data sets, which are misaligned magnetic resonance (MR) brain scans obtained using different image acquisition protocols as well as misaligned MR-computed tomography scans. We experimently show that our L2E-based scheme yields better accuracy over the normalized mutual information. <s> BIB014 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Image registration requires the specification of a class of discriminatory image features and an appropriate image dissimilarity measure. Entropic spanning graphs produce a consistent estimator of feature entropy and divergence. Direct estimators with non-parametric "plug-in" density estimators, on single pixels and independent image component feature vectors are compared. A technique for minimum spanning tree construction with significantly lower memory and time complexity is investigated. On the basis of misregistration errors with decreasing SNR, the minimal graph entropy estimator can have better performance than indirect estimators. In general, misregistration errors are lower with higher dimensional ICA feature vectors as compared to single pixels. <s> BIB015 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> An overview is presented of the medical image processing literature on mutual-information-based registration. The aim of the survey is threefold: an introduction for those new to the field, an overview for those working in the field, and a reference for those searching for literature on a specific application. Methods are classified according to the different aspects of mutual-information-based registration. The main division is in aspects of the methodology and of the application. The part on methodology describes choices made on facets such as preprocessing of images, gray value interpolation, optimization, adaptations to the mutual information measure, and different types of geometrical transformations. The part on applications is a reference of the literature available on different modalities, on interpatient registration and on different anatomical objects. Comparison studies including mutual information are also considered. The paper starts with a description of entropy and mutual information and it closes with a discussion on past achievements and some future challenges. <s> BIB016 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Entropy-based divergence measures have shown promising results in many areas of engineering and image processing. We define a new generalized divergence measure, namely, the Jensen-Renyi (1996, 1976) divergence. Some properties such as convexity and its upper bound are derived. Based on the Jensen-Renyi divergence, we propose a new approach to the problem of image registration. Some appealing advantages of registration by Jensen-Renyi divergence are illustrated, and its connections to mutual information-based registration techniques are analyzed. As the key focus of this paper, we apply Jensen-Renyi divergence for inverse synthetic aperture radar (ISAR) image registration. The goal is to estimate the target motion during the imaging time. Our approach applies Jensen-Renyi divergence to measure the statistical dependence between consecutive ISAR image frames, which would be maximal if the images are geometrically aligned. Simulation results demonstrate that the proposed method is efficient and effective. <s> BIB017 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> A variational method for nonrigid registration of multimodal image data is presented. A suitable deformation will be determined via the minimization of a morphological, i.e., contrast invariant, matching functional along with an appropriate regularization energy. The aim is to correlate the morphologies of a template and a reference image under the deformation. Mathematically, the morphology of images can be described by the entity of level sets of the image and hence by its Gauss map. A class of morphological matching functionals is presented which measure the defect of the template Gauss map in the deformed state with respect to the deformed Gauss map of the reference image. The problem is regularized by considering a nonlinear elastic regularization energy. Existence of homeomorphic, minimizing deformation is proved under assumptions on the class of admissible deformations. With respect to actual medical applications, suitable generalizations of the matching energies and the boundary conditions are pre... <s> BIB018 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Abstract We present a method to register a preoperative MR volume to a sparse set of intraoperative ultrasound slices. Our aim is to allow the transfer of information from preoperative modalities to intraoperative ultrasound images to aid needle placement during thermal ablation of liver metastases. The spatial relationship between ultrasound slices is obtained by tracking the probe using a Polaris optical tracking system. Images are acquired at maximum exhalation and we assume the validity of the rigid body transformation. An initial registration is carried out by picking a single corresponding point in both modalities. Our strategy is to interpret both sets of images in an automated pre-processing step to produce evidence or probabilities of corresponding structure as a pixel or voxel map. The registration algorithm converts the intensity values of the MR and ultrasound images into vessel probability values. The registration is then carried out between the vessel probability images. Results are compared to a “bronze standard” registration which is calculated using a manual point/line picking algorithm and verified using visual inspection. Results show that our starting estimate is within a root mean square target registration error (calculated over the whole liver) of 15.4 mm to the “bronze standard” and this is improved to 3.6 mm after running the intensity-based algorithm. <s> BIB019 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> The ordinary sample correlation coefficient is a popular similarity measure for aligning images from the same or similar modalities. However, this measure can be sensitive to the presence of "outlier" objects that appear in one image but not the other, such as surgical instruments, the patient table, etc., which can lead to biased registrations. This paper describes an intensity-based image registration technique that uses a robust correlation coefficient as a similarity measure. Relative to the ordinary sample correlation coefficient, the proposed similarity measure reduces the influence of outliers. We also compared the performance of the proposed method with the mutual information-based method. The robust correlation-based method should be useful for image registration in radiotherapy (KeV to MeV X-ray images) and image-guided surgery applications. We have investigated the properties of the proposed method by theoretical analysis, computer simulations, a phantom experiment, and with functional magnetic resonance imaging (MRI) data. <s> BIB020 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Finding point correspondence in anatomical images is a key step in shape analysis and deformable registration. This paper proposes an automatic correspondence detection algorithm for intramodality MR brain images of different subjects using wavelet-based attribute vectors (WAVs) defined on every image voxel. The attribute vector (AV) is extracted from the wavelet subimages and reflects the image structure in a large neighborhood around the respective voxel in a multiscale fashion. It plays the role of a morphological signature for each voxel, and our goal is, therefore, to make it distinctive of the respective voxel. Correspondence is then determined from similarities of AVs. By incorporating the prior knowledge of the spatial relationship among voxels, the ability of the proposed algorithm to find anatomical correspondence is further improved. Experiments with MR images of human brains show that the algorithm performs similarly to experts, even for complex cortical structures. <s> BIB021 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> A new information-theoretic approach is presented for finding the pose of an object in an image. The technique does not require information about the surface properties of the object, besides its shape, and is robust with respect to variations of illumination. In our derivation few assumptions are made about the nature of the imaging process. As a result the algorithms are quite general and may foreseeably be used in a wide variety of imaging situations. ::: ::: Experiments are presented that demonstrate the approach registering magnetic resonance (MR) images, aligning a complex 3D object model to real scenes including clutter and occlusion, tracking a human head in a video sequence and aligning a view-based 2D object model to real images. ::: ::: The method is based on a formulation of the mutual information between the model and the image. As applied here the technique is intensity-based, rather than feature-based. It works well in domains where edge or gradient-magnitude based methods have difficulty, yet it is more robust than traditional correlation. Additionally, it has an efficient implementation that is based on stochastic approximation. <s> BIB022 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> A measure for registration of medical images that currently draws much attention is mutual information. The measure originates from information theory, but has been shown to be successful for image registration as well. Information theory, however, offers many more measures that may be suitable for image registration. These all measure the divergence of the joint distribution of the images' grey values from the joint distribution that would have been found had the images been completely independent. This paper compares the performance of mutual information as a registration measure with that of other f-information measures. The measures are applied to rigid registration of positron emission tomography(PET)/magnetic resonance (MR) and MR/computed tomography (CT) images, for 35 and 41 image pairs, respectively. An accurate gold standard transformation is available for the images, based on implanted markers. The registration performance, robustness and accuracy of the measures are studied. Some of the measures are shown to perform poorly on all aspects. The majority of measures produces results similar to those of mutual information. An important finding, however, is that several measures, although slightly more difficult to optimize, can potentially yield significantly more accurate results than mutual information. <s> BIB023 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Mutual information (MI) has emerged in recent years as an effective similarity measure for comparing images. One drawback of MI, however, is that it is calculated on a pixel by pixel basis, meaning that it takes into account only the relationships between corresponding indi- vidual pixels and not those of each pixel's respective neighborhood. As a result, much of the spatial information inherent in images is not utilized. In this paper, we propose a novel extension to MI called regional mutual information (RMI). This extension efficiently takes neighborhood regions of corresponding pixels into account. We demonstrate the usefulness of RMI by applying it to a real-world problem in the medical domain— intensity-based 2D-3D registration of X-ray projection images (2D) to a CT image (3D). Using a gold-standard spine image data set, we show that RMI is a more robust similarity meaure for image registration than MI. <s> BIB024 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> We propose a new voxel similarity measure which utilises local image structure and intensity information. Gaussian scale space derivatives provide the structural information. Each derivative is assigned an information channel of N-D normalised mutual information. We illustrate the behaviour of the measure for a simulated signal and 2D medical brain images and demonstrate its potential for non-rigid, inter-subject registration of 3D thorax MR images as a proof of concept. <s> BIB025 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> A method for processing images prior to normalized mutual information based registration and an enhancement of the registration measure are presented. The method relies on k-means clustering of the intensity distribution and a gray-value remapping of spatially unconnected segments of similar intensity. Registration experiments using binary images from the MPEG-7 dataset and retinal images show our preprocessing to cause a significant reduction of the number of misregistrations without loss of accuracy or an increase in optimization steps. <s> BIB026 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> The need for non-rigid multi-modal registration is becoming increasingly common for many clinical applications. To date, however, existing proposed techniques remain as largely academic research effort with very few methods being validated for clinical product use. It has been suggested by Crum et al. [1] that the context-free nature of these methods is one of the main limitations and that moving towards context-specific methods by incorporating prior knowledge of the underlying registration problem is necessary to achieve registration results that are accurate and robust enough for clinical applications. In this paper, we propose a novel non-rigid multi-modal registration method using a variational formulation that incorporates a prior learned joint intensity distribution. The registration is achieved by simultaneously minimizing the Kullback-Leibler divergence between an observed and a learned joint intensity distribution and maximizing the mutual information between reference and alignment images. We have applied our proposed method on both synthetic and real images with encouraging results. <s> BIB027 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Automatic multi-modal image registration is central to numerous tasks in medical imaging today and has a vast range of applications e.g., image guidance, atlas construction, etc. In this paper, we present a novel multi-modal 3D non-rigid registration algorithm where in 3D images to be registered are represented by their corresponding local frequency maps efficiently computed using the Riesz transform as opposed to the popularly used Gabor filters. The non-rigid registration between these local frequency maps is formulated in a statistically robust framework involving the minimization of the integral squared error a.k.a. L2E (L2 error). This error is expressed as the squared difference between the true density of the residual (which is the squared difference between the non-rigidly transformed reference and the target local frequency representations) and a Gaussian or mixture of Gaussians density approximation of the same. The non-rigid transformation is expressed in a B-spline basis to achieve the desired smoothness in the transformation as well as computational efficiency. ::: ::: The key contributions of this work are (i) the use of Riesz transform to achieve better efficiency in computing the local frequency representation in comparison to Gabor filter-based approaches, (ii) new mathematical model for local-frequency based non-rigid registration, (iii) analytic computation of the gradient of the robust non-rigid registration cost function to achieve efficient and accurate registration. The proposed non-rigid L2E-based registration is a significant extension of research reported in literature to date. We present experimental results for registering several real data sets with synthetic and real non-rigid misalignments. <s> BIB028 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> This paper presents a learning-based method for deformable registration of magnetic resonance (MR) brain images. There are two novelties in the proposed registration method. First, a set of best-scale geometric features are selected for each point in the brain, in order to facilitate correspondence detection during the registration procedure. This is achieved by optimizing an energy function that requires each point to have its best-scale geometric features consistent over the corresponding points in the training samples, and at the same time distinctive from those of nearby points in the neighborhood. Second, the active points used to drive the brain registration are hierarchically selected during the registration procedure, based on their saliency and consistency measures. That is, the image points with salient and consistent features (across different individuals) are considered for the initial registration of two images, while other less salient and consistent points join the registration procedure later. By incorporating these two novel strategies into the framework of the HAMMER registration algorithm, the registration accuracy has been improved according to the results on simulated brain data, and also visible improvement is observed particularly in the cortical regions of real brain data <s> BIB029 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Registration of 3D volumetric data to 2D X-ray images has many applications in image-guided surgery, varying from verification of patient position to working projection searching. In this work, we propose a learning-based method that incorporates the prior information on the expected joint intensity histogram for robust real-time 2D/3D registration. Jensen-Shannon divergence (JSD) is used to quantify the statistical (dis)similarity between the observed and expected joint histograms, and is shown to be superior to Kullback-Leibler divergence (KLD) in its symmetry, being theoretically upper-bounded, and well-defined with histogram non-continuity. A nonlinear histogram mapping technique is proposed to handle the intensity difference between the observed data and the training data so that the learned prior can be used for registration of a wide range of data subject to intensity variations. We applied the proposed method on synthetic, phantom and clinical data. Experimental results demonstrated that a combination of the prior knowledge and the low-level similarity measure between the images being registered led to a more robust and accurate registration in comparison with the cases where either of the two factors was used alone as the driving force for registration. <s> BIB030 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> We present a deformable registration algorithm for multi-modality images based on information theoretic similarity measures at the scale of individual image voxels. We derive analytical expressions for the mutual information, the joint entropy, and the sum of marginal entropies of two images over a small neighborhood in terms of image gradients. Using these expressions, we formulate image registration algorithms maximizing local similarity over the whole image domain in an energy minimization framework. This strategy produces highly elastic image alignment as the registration is driven by voxel similarities between the images, the algorithms are easily implementable using the closed-form expressions for the derivative of the optimization function with respect to the deformation, and avoid estimation of joint and marginal probability densities governing the image intensities essential to conventional information theoretic image registration methods. <s> BIB031 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> This paper is motivated by the analysis of serial structural magnetic resonance imaging (MRI) data of the brain to map patterns of local tissue volume loss or gain over time, using registration-based deformation tensor morphometry. Specifically, we address the important confound of local tissue contrast changes which can be induced by neurodegenerative or neurodevelopmental processes. These not only modify apparent tissue volume, but also modify tissue integrity and its resulting MRI contrast parameters. In order to address this confound we derive an approach to the voxel-wise optimization of regional mutual information (RMI) and use this to drive a viscous fluid deformation model between images in a symmetric registration process. A quantitative evaluation of the method when compared to earlier approaches is included using both synthetic data and clinical imaging data. Results show a significant reduction in errors when tissue contrast changes locally between acquisitions. Finally, examples of applying the technique to map different patterns of atrophy rate in different neurodegenerative conditions is included. <s> BIB032 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Mutual information has been successfully used as an effective similarity measure for multimodal image registration. However, a drawback of the standard mutual information-based computation is that the joint histogram is only calculated from the correspondence between individual voxels in the two images. In this paper, the normalized mutual information measure is extended to consider the correspondence between voxel blocks in multimodal rigid registration. The ambiguity and high-dimensionality that appears when dealing with the voxel neighborhood is solved using uniformly distributed random lines and reducing the number of bins of the images. Experimental results show a significant improvement with respect to the standard normalized mutual information. <s> BIB033 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> A particular problem in image registration arises for multi-modal images taken from different imaging devices and/or modalities. Starting in 1995, mutual information has shown to be a very successful distance measure for multi-modal image registration. However, mutual information has also a number of well-known drawbacks. Its main disadvantage is that it is known to be highly non-convex and has typically many local maxima. ::: ::: This observation motivate us to seek a different image similarity measure which is better suited for optimization but as well capable to handle multi-modal images. In this work we investigate an alternative distance measure which is based on normalized gradients and compare its performance to Mutual Information. We call the new distance measure Normalized Gradient Fields (NGF). <s> BIB034 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> A new type of divergence measure for the registration of medical images is introduced that exploits the properties of the modified Bessel functions of the second kind. The properties of the proposed divergence coefficient are analysed and compared with those of the classic measures, including Kullback-Leibler, Renyi, and Iinfinity, divergences. To ensure its effectiveness and widespread applicability to any arbitrary set of data types, the performance of the new measure is analysed for Gaussian, exponential, and other advanced probability density functions. The results verify its robustness. Finally, the new divergence measure is used in the registration of CT to MR medical images to validate the improvement in registration accuracy. <s> BIB035 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> We present a new method for the fast and robust computation of information theoretic similarity measures for alignment of multi-modality medical images. The proposed method defines a non-uniform, adaptive sampling scheme for estimating the entropies of the images, which is less vulnerable to local maxima as compared to uniform and random sampling. The sampling is defined using an octree partition of the template image, and is preferable over other proposed methods of non-uniform sampling since it respects the underlying data distribution. It also extends naturally to a multi-resolution registration approach, which is commonly employed in the alignment of medical images. The effectiveness of the proposed method is demonstrated using both simulated MR images obtained from the BrainWeb database and clinical CT and SPECT images. <s> BIB036 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> We propose conditional mutual information (cMI) as a new similarity measure for nonrigid image registration. We start from a 3D joint histogram incorporating, besides the reference and floating intensity dimensions, also a spatial dimension expressing the location of the joint intensity pair in the reference image. cMI is calculated as the expectation value of the conditional mutual information between the reference and floating intensities given the spatial distribution. ::: ::: Validation experiments were performed comparing cMI and global MI on artificial CT/MR registrations and registrations complicated with a strong bias field; both a Parzen window and generalised partial volume kernel were used for histogram construction. In both experiments, cMI significantly outperforms global MI. Moreover, cMI is compared to global MI for the registration of three patient CT/MR datasets, using overlap and centroid distance as validation measure. The best results are obtained using cMI. <s> BIB037 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> One of the most challenging problems in modern neuroimaging is detailed characterization of neurodegeneration. Quantifying spatial and longitudinal atrophy patterns is an important component of this process. These spatiotemporal signals will aid in discriminating between related diseases, such as frontotemporal dementia (FTD) and Alzheimer's disease (AD), which manifest themselves in the same at-risk population. Here, we develop a novel symmetric image normalization method (SyN) for maximizing the cross-correlation within the space of diffeomorphic maps and provide the Euler-Lagrange equations necessary for this optimization. We then turn to a careful evaluation of our method. Our evaluation uses gold standard, human cortical segmentation to contrast SyN's performance with a related elastic method and with the standard ITK implementation of Thirion's Demons algorithm. The new method compares favorably with both approaches, in particular when the distance between the template brain and the target brain is large. We then report the correlation of volumes gained by algorithmic cortical labelings of FTD and control subjects with those gained by the manual rater. This comparison shows that, of the three methods tested, SyN's volume measurements are the most strongly correlated with volume measurements gained by expert labeling. This study indicates that SyN, with cross-correlation, is a reliable method for normalizing and making anatomical measurements in volumetric MRI of patients and at-risk elderly individuals. <s> BIB038 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> In Studholme et al. introduced normalized mutual information (NMI) as an overlap invariant generalization of mutual information (MI). Even though Studholme showed how NMI could be used effectively in multimodal medical image alignment, the overlap invariance was only established empirically on a few simple examples. In this paper, we illustrate a simple example in which NMI fails to be invariant to changes in overlap size, as do other standard similarity measures including MI, cross correlation (CCorr), correlation coefficient (CCoeff), correlation ratio (CR), and entropy correlation coefficient (ECC). We then derive modified forms of all of these similarity measures that are proven to be invariant to changes in overlap size. This is done by making certain assumptions about background statistics. Experiments on multimodal rigid registration of brain images show that 1) most of the modified similarity measures outperform their standard forms, and 2) the modified version of MI exhibits superior performance over any of the other similarity measures for both CT/MR and PET/MR registration. <s> BIB039 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> We provide a detailed analysis of the use of minimal spanning graphs as an alignment method for registering multimodal images. This yields an efficient graph theoretic algorithm that, for the first time, jointly estimates both an alignment measure and a viable descent direction with respect to a parameterized class of spatial transformations. We also show how prior information about the interimage modality relationship from prealigned image pairs can be incorporated into the graph-based algorithm. A comparison of the graph theoretic alignment measure is provided with more traditional measures based on plug-in entropy estimators. This highlights previously unrecognized similarities between these two registration methods. Our analysis gives additional insight into the tradeoffs the graph-based algorithm is making and how these will manifest themselves in the registration algorithm's performance. <s> BIB040 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> To make up for the lack of concern on the spatial information in the conventional mutual information based image registration framework, this paper designs a novel spatial feature field, namely the maximum distance-gradient (MDG) vector field, for registration tasks. It encodes both the local edge information and globally defined spatial information related to the intensity difference, the distance, and the direction of a voxel to a MDG source point. A novel similarity measure is proposed as the combination of the multi-dimensional mutual information and an angle measure on the MDG vector field. This measure integrates both the magnitude and orientation information of the MDG vector field into the image registration process. Experimental results on clinical 3D CT and T1-weighted MR image volumes show that, as compared with the conventional mutual information based method and two of its adaptations incorporating spatial information, the proposed method can give longer capture ranges at different image resolutions. This leads to more robust registrations. Around 2000 randomized rigid registration experiments demonstrate that our method consistently gives much higher success rates than the aforementioned three related methods. Moreover, it is shown that the registration accuracy of our method is high. <s> BIB041 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> The fusion of tracked ultrasound with CT has benefits for a variety of clinical applications, however extensive manual effort is usually required for correct registration. We developed new methods that allow one to simulate medical ultrasound from CT in real-time, reproducing the majority of ultrasonic imaging effects. They are combined with a robust similarity measure that assesses the correlation of a combination of signals extracted from CT with ultrasound, without knowing the influence of each signal. This serves as the foundation of a fully automatic registration, that aligns a 3D ultrasound sweep with the corresponding tomographic modality using a rigid or an affine transformation model, without any manual interaction. These techniques were evaluated in a study involving 25 patients with indeterminate lesions in liver and kidney. The clinical setup, acquisition and registration workflow is described, along with the evaluation of the registration accuracy with respect to physician-defined Ground Truth. Our new algorithm correctly registers without any manual interaction in 76% of the cases, the average RMS TRE over multiple target lesions throughout the liver is 8.1mm. <s> BIB042 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Abstract The hierarchical subdivision strategy which decomposes a non-rigid matching problem into numerous local rigid transformations is a very common approach in image registration. While mutual information (MI) has proven to be a very robust and reliable similarity measure for intensity-based matching of multi-modal images, numerous problems have to be faced if it is applied to small-sized images, compromising its usefulness for such subdivision schemes. We examine and explain the loss of MI’s statistical consistency along the hierarchical subdivision. Information theoretical measures are proposed to identify the problematic regions in order to overcome the MI drawbacks. This does not only improve the accuracy and robustness of the registration, but also can be used as a very efficient stopping criterion for the further subdivision of nodes in the hierarchy, which drastically reduces the computational cost of the entire registration procedure. Moreover, we present a new intensity mapping technique allowing to replace MI by more reliable measures for small patches. Integrated into the hierarchical framework, this mapping can locally transform the multi-modal images into an intermediate pseudo-modality. This intensity mapping uses the local joint intensity histograms of the coarsely registered sub-images and allows the use of the more robust and computationally more efficient cross-correlation coefficient (CC) for the matching at lower levels of the hierarchy. <s> BIB043 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Radiation therapy for cervical cancer can benefit from image registration in several ways, for example by studying the motion of organs, or by (partially) automating the delineation of the target volume and other structures of interest. In this paper, the registration of cervical data is addressed using mutual information (MI) of not only image intensity, but also features that describe local image structure. Three aspects of the registration are addressed to make this approach feasible. First, instead of relying on a histogram-based estimation of mutual information, which poses problems for a larger number of features, a graph-based implementation of alpha-mutual information (alpha-MI) is employed. Second, the analytical derivative of alpha-MI is derived. This makes it possible to use a stochastic gradient descent method to solve the registration problem, which is substantially faster than nonderivative-based methods. Third, the feature space is reduced by means of a principal component analysis, which also decreases the registration time. The proposed technique is compared to a standard approach, based on the mutual information of image intensity only. Experiments are performed on 93 T2-weighted MR clinical data sets acquired from 19 patients with cervical cancer. Several characteristics of the proposed algorithm are studied on a subset of 19 image pairs (one pair per patient). On the remaining data (36 image pairs, one or two pairs per patient) the median overlap is shown to improve significantly compared to standard MI from 0.85 to 0.86 for the clinical target volume (CTV, p = 2 x 10(-2)), from 0.75 to 0.81 for the bladder (p = 8 x 10(-6)), and from 0.76 to 0.77 for the rectum (p = 2 x 10(-4)). The registration error is improved at important tissue interfaces, such as that of the bladder with the CTV, and the interface of the rectum with the uterus and cervix. <s> BIB044 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Multi-modal image registration is a challenging problem in medical imaging. The goal is to align anatomically identical structures; however, their appearance in images acquired with different imaging devices, such as CT or MR, may be very different. Registration algorithms generally deform one image, the floating image, such that it matches with a second, the reference image, by maximizing some similarity score between the deformed and the reference image. Instead of using a universal, but a priori fixed similarity criterion such as mutual information, we propose learning a similarity measure in a discriminative manner such that the reference and correctly deformed floating images receive high similarity scores. To this end, we develop an algorithm derived from max-margin structured output learning, and employ the learned similarity measure within a standard rigid registration algorithm. Compared to other approaches, our method adapts to the specific registration problem at hand and exploits correlations between neighboring pixels in the reference and the floating image. Empirical evaluation on CT-MR/PET-MR rigid registration tasks demonstrates that our approach yields robust performance and outperforms the state of the art methods for multi-modal medical image registration. <s> BIB045 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> A new feature based nonrigid image registration method for magnetic resonance (MR) brain images is presented in this paper. Each image voxel is represented by a rotation invariant feature vector, which is computed by passing the input image volumes through a new bank of symmetric alpha stable (SalphaS) filters. There are three main contributions presented in this paper. First, this work is motivated by the fact that the frequency spectrums of the brain MR images often exhibit non-Gaussian heavy-tail behavior which cannot be satisfactorily modeled by the conventional Gabor filters. To this end, we propose the use of SalphaS filters to model such behavior and show that the Gabor filter is a special case of the SalphaS filter. Second, the maximum response orientation (MRO) selection criterion is designed to extract rotation invariant features for registration tasks. The MRO selection criterion also significantly reduces the number of dimensions of feature vectors and therefore lowers the computation time. Third, in case the segmentations of the input image volumes are available, the Fisher's separation criterion (FSC) is introduced such that the discriminating power of different feature types can be directly compared with each other before performing the registration process. Using FSC, weights can also be assigned automatically to different voxels in the brain MR images. The weight of each voxel determined by FSC reflects how distinctive and salient the voxel is. Using the most distinctive and salient voxels at the initial stage to drive the registration can reduce the risk of being trapped in the local optimum during image registration process. The larger the weight, the more important the voxel. With the extracted feature vectors and the associated weights, the proposed method registers the source and the target images in a hierarchical multiresolution manner. The proposed method has been intensively evaluated on both simulated and real 3-D datasets obtained from BrainWeb and Internet Brain Segmentation Repository (IBSR), respectively, and compared with HAMMER, an extended version of HAMMER based on local histograms (LHF), FFD, Demons, and the Gabor filter based registration method. It is shown that the proposed method achieves the highest registration accuracy among the five widely used image registration methods. <s> BIB046 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Accurate definition of the similarity measure is a key component in image registration. Most commonly used intensity-based similarity measures rely on the assumptions of independence and stationarity of the intensities from pixel to pixel. Such measures cannot capture the complex interactions among the pixel intensities, and often result in less satisfactory registration performances, especially in the presence of spatially-varying intensity distortions. We propose a novel similarity measure that accounts for intensity nonstationarities and complex spatially-varying intensity distortions in mono-modal settings. We derive the similarity measure by analytically solving for the intensity correction field and its adaptive regularization. The final measure can be interpreted as one that favors a registration with minimum compression complexity of the residual image between the two registered images. One of the key advantages of the new similarity measure is its simplicity in terms of both computational complexity and implementation. This measure produces accurate registration results on both artificial and real-world problems that we have tested, and outperforms other state-of-the-art similarity measures in these cases. <s> BIB047 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> The registration of multi-modal images is the process of finding a transformation which maps one image to the other according to a given similarity metric. In this paper, we introduce a novel approach for metric learning, aiming to address highly non functional correspondences through the integration of statistical regression and multi-label classification. We developed a position-invariant method that models the variations of intensities through the use of linear combinations of kernels that are able to handle intensity shifts. Such transport functions are considered as the singleton potentials of a Markov Random Field (MRF) where pair-wise connections encode smoothness as well as prior knowledge through a local neighborhood system. We use recent advances in the field of discrete optimization towards recovering the lowest potential of the designed cost function. Promising results on real data demonstrate the potentials of our approach. <s> BIB048 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Visual understanding is often based on measuring similarity between observations. Learning similarities specific to a certain perception task from a set of examples has been shown advantageous in various computer vision and pattern recognition problems. In many important applications, the data that one needs to compare come from different representations or modalities, and the similarity between such data operates on objects that may have different and often incommensurable structure and dimensionality. In this paper, we propose a framework for supervised similarity learning based on embedding the input data from two arbitrary spaces into the Hamming space. The mapping is expressed as a binary classification problem with positive and negative examples, and can be efficiently learned using boosting algorithms. The utility and efficiency of such a generic approach is demonstrated on several challenging applications including cross-representation shape retrieval and alignment of multi-modal medical images. <s> BIB049 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> A general-purpose deformable registration algorithm referred to as "DRAMMS" is presented in this paper. DRAMMS bridges the gap between the traditional voxel-wise methods and landmark/feature-based methods with primarily two contributions. First, DRAMMS renders each voxel relatively distinctively identifiable by a rich set of attributes, therefore largely reducing matching ambiguities. In particular, a set of multi-scale and multi-orientation Gabor attributes are extracted and the optimal components are selected, so that they form a highly distinctive morphological signature reflecting the anatomical and geometric context around each voxel. Moreover, the way in which the optimal Gabor attributes are constructed is independent of the underlying image modalities or contents, which renders DRAMMS generally applicable to diverse registration tasks. A second contribution of DRAMMS is that it modulates the registration by assigning higher weights to those voxels having higher ability to establish unique (hence reliable) correspondences across images, therefore reducing the negative impact of those regions that are less capable of finding correspondences (such as outlier regions). A continuously-valued weighting function named "mutual-saliency" is developed to reflect the matching uniqueness between a pair of voxels implied by the tentative transformation. As a result, voxels do not contribute equally as in most voxel-wise methods, nor in isolation as in landmark/feature-based methods. Instead, they contribute according to the continuously-valued mutual-saliency map, which dynamically evolves during the registration process. Experiments in simulated images, inter-subject images, single-/multi-modality images, from brain, heart, and prostate have demonstrated the general applicability and the accuracy of DRAMMS. <s> BIB050 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Mutual information (MI) registration including spatial information has been shown to perform better than the traditional MI measures for certain nonrigid registration tasks. In this work, we first provide new insight to problems of the MI-based registration and propose to use the spatially encoded mutual information (SEMI) to tackle these problems. To encode spatial information, we propose a hierarchical weighting scheme to differentiate the contribution of sample points to a set of entropy measures, which are associated to spatial variable values. By using free-form deformations (FFDs) as the transformation model, we can first define the spatial variable using the set of FFD control points, and then propose a local ascent optimization scheme for nonrigid SEMI registration. The proposed SEMI registration can improve the registration accuracy in the nonrigid cases where the traditional MI is challenged due to intensity distortion, contrast enhancement, or different imaging modalities. It also has a similar computation complexity to the registration using traditional MI measures, improving up to two orders of magnitude of computation time compared to the traditional schemes. We validate our algorithms using phantom brain MRI, simulated dynamic contrast enhanced mangetic resonance imaging (MRI) of the liver, and in vivo cardiac MRI. The results show that the SEMI registration significantly outperforms the traditional MI registration. <s> BIB051 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> We propose a method to efficiently compute mutual information between high-dimensional distributions of image patches. This in turn is used to perform accurate registration of images captured under different modalities, while exploiting their local structure otherwise missed in traditional mutual information definition. We achieve this by organizing the space of image patches into orbits under the action of Euclidean transformations of the image plane, and estimating the modes of a distribution in such an orbit space using affinity propagation. This way, large collections of patches that are equivalent up to translations and rotations are mapped to the same representative, or "dictionary element". We then show analytically that computing mutual information for a joint distribution in this space reduces to computing mutual information between the (scalar) label maps, and between the transformations mapping each patch into its closest dictionary element. We show that our approach improves registration performance compared with the state of the art in multimodal registration, using both synthetic and real images with quantitative ground truth. <s> BIB052 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Defining a suitable metric is one of the biggest challenges in deformable image fusion from different modalities. In this paper, we propose a novel approach for multi-modal metric learning in the deformable registration framework that consists of embedding data from both modalities into a common metric space whose metric is used to parametrize the similarity. Specifically, we use image representation in the Fourier/Gabor space which introduces invariance to the local pose parameters, and the Hamming metric as the target embedding space, which allows constructing the embedding using boosted learning algorithms. The resulting metric is incorporated into a discrete optimization framework. Very promising results demonstrate the potential of the proposed method. <s> BIB053 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> There are two main issues that make nonrigid image registration a challenging task. First, voxel intensity similarity may not be necessarily equivalent to anatomical similarity in the image correspondence searching process. Second, during the imaging process, some interferences such as unexpected rotations of input volumes and monotonic gray-level bias fields can adversely affect the registration quality. In this paper, a new feature-based nonrigid image registration method is proposed. The proposed method is based on a new type of image feature, namely, uniform spherical region descriptor (USRD), as signatures for each voxel. The USRD is rotation and monotonic gray-level transformation invariant and can be efficiently calculated. The registration process is therefore formulated as a feature matching problem. The USRD feature is integrated with the Markov random field labeling framework in which energy function is defined for registration. The energy function is then optimized by the α-expansion algorithm. The proposed method has been compared with five state-of-the-art registration approaches on both the simulated and real 3-D databases obtained from the BrainWeb and Internet Brain Segmentation Repository, respectively. Experimental results demonstrate that the proposed method can achieve high registration accuracy and reliable robustness behavior. <s> BIB054 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> Deformable registration of images obtained from different modalities remains a challenging task in medical image analysis. This paper addresses this important problem and proposes a modality independent neighbourhood descriptor (MIND) for both linear and deformable multi-modal registration. Based on the similarity of small image patches within one image, it aims to extract the distinctive structure in a local neighbourhood, which is preserved across modalities. The descriptor is based on the concept of image self-similarity, which has been introduced for non-local means filtering for image denoising. It is able to distinguish between different types of features such as corners, edges and homogeneously textured regions. MIND is robust to the most considerable differences between modalities: non-functional intensity relations, image noise and non-uniform bias fields. The multi-dimensional descriptor can be efficiently computed in a dense fashion across the whole image and provides point-wise local similarity across modalities based on the absolute or squared difference between descriptors, making it applicable for a wide range of transformation models and optimisation algorithms. We use the sum of squared differences of the MIND representations of the images as a similarity metric within a symmetric non-parametric Gauss-Newton registration framework. In principle, MIND would be applicable to the registration of arbitrary modalities. In this work, we apply and validate it for the registration of clinical 3D thoracic CT scans between inhale and exhale as well as the alignment of 3D CT and MRI scans. Experimental results show the advantages of MIND over state-of-the-art techniques such as conditional mutual information and entropy images, with respect to clinically annotated landmark locations. <s> BIB055 </s> Deformable Medical Image Registration: A Survey <s> 1) Mono-Modal Registration: <s> The standard approach to multi-modal registration is to apply sophisticated similarity metrics such as mutual information. The disadvantage of these metrics, in comparison to measuring the intensity difference with, e.g. L1 or L2 distance, is the increase in computational complexity and consequently the increase in runtime of the registration. An alternative approach, which has not yet gained much attention in the literature, is to find image representations, so called structural representations, that allow for the application of the L1 and L2 distance for multi-modal images. This has not only the advantage of a faster similarity calculation but enables also the application of more sophisticated optimization strategies. In this article, we theoretically analyze the requirements for structural representations. Further, we introduce two approaches to create such representations, which are based on the calculation of patch entropy and manifold learning, respectively. While the application of entropy has practical advantages in terms of computational complexity, the usage of manifold learning has theoretical advantages, by presenting an optimal approximation to one of the theoretical requirements. We perform experiments on multiple datasets for rigid, deformable, and groupwise registration with good results with respect to both, runtime and quality of alignment. <s> BIB056
In the mono-modal case, the same imaging device is used to capture the same type of information for both volumes. a) Intensity-Based Methods: Different matching criteria can be devised depending on the assumptions about the intensity relationship between the images. In the case that the same anatomical structures are assumed to correspond to similar intensity values, the sum of squared or absolute differences (SSD and SAD, respectively) can be used as a matching criterion. The choice between the two depends on the assumption regarding the noise that corrupts the image intensities. In the case that a linear relation is assumed between the signal intensities, the optimal criterion is cross correlation (CCor) and correlation coefficient (CCoef)  , BIB038 , BIB020 . b) Attribute-Based Methods: Intensity information may lead to ambiguous matching and local minima in the objective function when pixels of the same anatomical structure take similar intensity values BIB011 . A number of researchers have proposed to increase the dimensionality of the feature space in order to cope with this shortcoming. A way to augment the feature space is by introducing local information through the use of attributes that represent the geometric structure of the underlying anatomy. These approaches are referred to as feature-or attribute-based ones. These approaches focus on a different way to represent image information, while they use standard similarity measures. Shen and Davatzikos BIB011 proposed the use of an attribute vector including geometric moment invariants in an attempt to capture local anatomical information at different spatial scales. The motivation was that a rich enough attribute vector would be able to differentiate voxels that would be considered the same based only on their intensity information. Thus, fewer local minima would be present and better accuracy may be achieved. To further reduce the effect of the local minima, they proposed a hierarchical scheme that successively approximated the objective function by progressively increasing the number of voxels where the matching criterion was evaluated. The previous method requires a presegmentation step in order to introduce local spatial information. Xue et al. proposed the use of Daubechies wavelets to populate the attribute vector in order to remove the requirement for segmentation BIB021 . The attribute vector was constructed in a multiscale fashion to be translation and rotation invariant. Shen proposed to tackle the above shortcoming by using local histograms and boundary information as attributes  . Wu et al. BIB029 proposed to use a learning approach in two ways. First, the authors proposed to learn the optimal scale for the geometric features for each voxel. Second, they proposed to learn which voxels should be used to drive the registration process. They proposed to take into account the saliency and the consistency of the description of the voxels across the training data. Local information may also be incorporated by exploiting the local frequency representations obtained as response to Gabor filters BIB050 , BIB014 . Gabor features have proven successful for both mono-modal and multi-modal image registration as they are able to capture information across different scales and orientations. Ou et al. BIB050 optimized the Gabor features to be more distinctive and employed the notion of mutual saliency to let the most reliable points drive the registration process. However, Liao and Chung BIB046 argued that frequency spectra of MRI brain images often exhibit non-Gaussian behavior and thus the choice of Gabor filters is not optimal. They proposed the use of symmetric alpha stable filters and showed experimentally that they outperform Gabor features in nonrigid MRI brain registration. Liao and Chung proposed a new feature for nonrigid registration in BIB054 . The feature is a uniform spherical region descriptor and is invariant with respect to rotation as well as monotonic gray-level transformation. Thus, it is able to account for the presence of a bias field. Myronenko and Song proposed to use residual complexity (RC) to account for complex spatially-varying intensity distortions BIB047 . This method attempts to register two images by minimizing the number of basis functions that are required to code the residual image. 2) Multi-Modal Registration: Multi-modal registration is more challenging as the choice of an appropriate matching criterion is a harder task. Two main approaches have been proposed to solve the problem (see also Fig. 2 ): 1) use of information theoretic measures, and 2) reduction of the multi-modal problem to a mono-modal problem. The latter can be achieved by either 1) simulating one modality from another, or 2) mapping both modalities to a common domain. Here, we are going to focus primarily on information theoretic approaches as they constitute the most frequently used way to tackle the challenges posed by multi-modal registration. Reduction techniques will also be briefly discussed. a) Information Theoretic Approaches: Information theoretic approaches were popularized by two different groups, one in US BIB002 , BIB022 , and one in Belgium  , BIB004 . Both teams investigated the use of mutual information (MI) in multi-modal image registration. The difference between their approaches is the way entropy is estimated. Wells et al. BIB002 and Viola and Wells BIB022 used a nonparametric estimator. Collignon et al.  and Maes et al. BIB004 used histograms instead. An important property of MI is its generality. MI does not assume any relationship between the image intensities. For a survey on MI-based registration methods, the interested reader is referred to the review by Pluim et al. BIB016 . The widespread use and study of MI has revealed some of its shortcomings. Primarily, it is not overlap invariant. Thus, in certain cases it may be possible for mutual information to be maximized when the images get misaligned. Studholme et al. proposed a Normalized version of mutual information (NMI) in order to remedy this shortcoming BIB007 . Recently, Cahill et al. elaborated upon the idea of overlap invariance and showed that neither NMI, MI, CR, CCor, nor CCoef are invariant to changes of overlap and proposed appropriate invariant versions of the previous similarity measures in BIB039 . The success of MI paved the way for the introduction of an important number of statistical criteria in image registration. Roche et al. BIB005 argued that the generality of mutual information can be a drawback when a reasonable hypothesis can be made regarding the relationship between the intensities. They proposed to use the correlation ratio (CR) as the appropriate similarity measure when the assumption of functional dependence between the image intensities is valid. Pluim et al. BIB023 compared the performance of a number of -information measures (including MI) in medical image registration. In the context of registration, -measures quantify the difference between the joint distribution of the intensities and the joint distribution that would arise if images were independent. The most important finding of the study was that there are -measures that were able to perform better than MI at the cost of more difficult inference. The idea to use divergence measures to compare joint intensity distributions has attracted significant attention and different divergence measures have been proposed for multi-modal image registration. Chung et al.  and Guetter et al. BIB027 used Kullback-Leibler divergence (KLD) to register multi-modal images. The joint intensity distribution was either learned from aligned pairs of images or by segmenting corresponding anatomical structures. Images got aligned by minimizing the divergence between the observed and estimated distributions. Liao et al. used Jensen-Shannon divergence (JSD) to compare learned distributions in BIB030 . JSD is symmetric, bounded and true metric. Another family of information theoretic approaches is built upon Renyi Entropy (RE)  . Let be a random variable with possible outcomes, then RE is defined as and . denotes the probability of the outcome . Based on this entropy, the JensenRenyi divergence can be defined. It is symmetric, convex for and is maximum when the distributions are degenerate. He et al. proposed its use for image registration BIB017 . Neemuchwala et al. BIB015 used a minimum spanning tree (MST) to estimate the RE. Spanning graphs were also used by Sabuncu and Ramadge BIB040 . Martin and Durrani introduced a generalization of KLD in BIB035 . The new divergence measure was based on modified Bessel functions of the second kind and allowed for an efficient recursive computation. The generalization of KLD was shown to perform better than the standard measures of divergence. Most of the aforementioned approaches share a common drawback; they are based on a single pixel joint probability model. As a consequence, by changing the positions of the pixels in a random way and evaluating the statistical criterion, the same similarity is obtained BIB009 . This extreme case demonstrates that when spatial information is ignored, registration may fail because the matching criterion is not able to quantify the difference between the two images. Shading artifacts pose a more reasonable challenge where information theoretic measures may fail BIB009 . To rectify this shortcoming, local context can be introduced in the used criterion. One way to relax the way statistical criteria are globally taken into account consists of computing them locally, thus coping with the fact that the relation between the two image intensities is nonstationary. This approach was investigated by Hermosillo et al.  and Karaçali BIB031 . Hermossilo et al. derived the Euler-Lagrange equations for MI, CR, and CCoef based on locally estimated probability distribution functions. Karaçali followed a deterministic rationale to express the mutual information, joint entropy and the sum of marginal entropies over small spherical regions in closed form. Local evaluation of mutual information has also been proposed by other researchers. For instance, Studholme et al. BIB032 investigated the use of Regional Mutual Information (RMI). The proposed similarity function is a linear weighted sum of local evaluations of MI and aims to reduce the error caused by local intensity changes. Sundar et al. proposed a robust way to compute MI BIB036 . The authors proposed to adaptively parcel late the image using octrees. The size of the octants is proportional to the homogeneity of the underlying image. These octant define the sampling strategy that is used to estimate the entropy. More samples are taken in regions where the density of octants is higher. In this case, the octree parcelation changes when objects move in the image and thus the estimation of the entropy changes. The method was applied in rigid registration. Loeckx et al. BIB037 proposed to condition the evaluation of MI upon the position. More recently, Zhuang et al. used locally evaluated MI in combination with standard global MI BIB051 . Under their approach, the local evaluation of the probability distribution function assesses pixels relatively to their distance from the FFD control points. An alternative way to introduce local context is by inserting spatial information. This has been mainly achieved by incorporating additional features that capture local geometric information, which results in higher order entropic measures. Pluim et al. BIB010 used the intensity image gradient as an additional cue. The proposed algorithm sought not only to maximize NMI but also intensity gradient information. This was simply achieved by multiplying NMI with a measure that takes into consideration both the intensity gradient magnitude and its orientation. This measure encourages the alignment of strong intensity gradients. Rueckert et al. BIB009 proposed to use second-order MI to encode local information by considering co-occurrences of intensities between neighboring voxels. That approach requires a -histogram to estimate the information measures. To account for the high dimension of the histogram and the curse of dimensionality, the number of bins was kept reasonably small. Russakoff et al. BIB024 proposed Regional Mutual Information that pushed forward the previous idea by taking into ac-count co-occurrences between regions. Moreover, an efficient way to deal with the curse of dimensionality was presented. Assuming a high-dimensional distribution, the data points were transformed so that they were independent in each dimension. Then, the entropy was estimated by summing the distributed entropies. Bardera et al. BIB033 investigated NMI between blocks of image elements. The high-dimensional NMI was estimated using random lines and a reduced number of bins. Recently, a similar approach was presented by Yi and Soatto BIB052 . Their approach is based upon learning a dictionary of image patches. Each image patch is represented by the label of its closest dictionary element. Then, higher-order mutual information can be estimated by using this label representation while accounting for the euclidean transformation that maps the patch to the label. Instead of explicitly taking into account neighboring voxels, another way to consider local information is by extracting features that concisely describe regional characteristics. Holden et al. BIB025 employed Gaussian scale space derivatives and incorporated them as an additional information channel in a higher dimensional MI criterion. Gan et al. also employed a multi-dimensional NMI criterion BIB041 . The authors proposed to construct a new feature field by considering the average rate of intensity change between any two points in the images. The proposed feature, named maximum distance-gradient, is calculated for a set of special points placed in important gradient areas by finding for which point the average rate of intensity change is maximized. This feature contains information regarding the local edge content, the maximum rate of intensity change as well as the direction in which this change happens. The magnitude of the MDG vector field formed the supplementary channel, while its orientation was used as a second element in the similarity function. The approaches that employ a higher dimensional statistical criterion are troubled by the curse of dimensionality. There are not enough samples to accurately calculate higher dimensional statistical criteria. To be able to handle such calculations, most researchers resort to crude implementation approximations such as limiting the number of histogram bins. Nevertheless, ways to estimate high dimensional entropies have been proposed and used to perform image registration. Sabuncu and Ramadge  introduced spatial information through the construction of feature vectors. The resulting high dimensional entropy was estimated with the use of the MST estimator. Neemuchwala et al. used entropic graphs to tackle high dimensional -MI registration of ultrasound images  . Both approaches coped with global linear registration. Staring et al. tackled deformable registration of Cervical MRI using highdimensional MI in BIB044 . Features were used to describe local geometric information and a -nearest neighbor graph was used to estimate the multi-dimensional MI. Spatial information is not the only type of information that can be used to endow registration with increased robustness and accuracy. Assuming that a prior step of segmentation has been performed, tissue classification information may also help disambiguate between voxels that belong to different tissues but share common appearance properties. Studholme et al. BIB003 segmented regions by thresholding and labeling connected components. The labels were used as an additional image and the matching criterion took into account the difference between the entropies of each image and the joint entropy. Knops et al. BIB026 performed a -means clustering before registration. Based on this clustering, voxels that shared similar intensity profiles but belonged to different anatomical structures were mapped to different intensity bins during the construction of the histogram. The new remapped intensities along with the initial one contributed to an NMI-based similarity criterion. D'Agostino et al.  took into account voxel class probabilities in the matching criterion in order to tackle the labeled-tolabeled and intensity-to-labeled image registration. For the labeled-to-labeled case, KLD was used to compare the distribution of the joint classes. For the intensity-to-labeled registration, a version of MI was used with the difference that one of the features is a class probability and not intensity. b) Reduction to Mono-Modal Registration: An alternative way to proceed with multi-modal registration is to reduce the problem to a mono-modal one. By this reduction, one aims to simplify the problem and facilitate its solution. There are two possible ways to perform such a task. First, one can simulate one modality from another so that at the end both images come from the same modality. Second, one can map both images to a third domain where the registration will take place. Simulating one modality from another can be achieved by taking advantage of the available knowledge regarding the physical properties of the imaging device. In this case, the goal is to model the imaging process. An alternative way is to exploit available co-registered pairs of images. In this case, machine learning techniques can be used to capture the relation between the intensities. Roche et al. BIB012 tackled ultrasound (US) to MR rigid registration by simulating an US image from the MR one. The authors exploited MR intensities and MR gradient magnitude information in order to predict US intensities. Complex phenomena such as US signal attenuation and speckle were neglected. As a consequence, the simulated images roughly resembled actual US images. Wein et al. BIB042 simulated an US image in order to tackle the problem of CT-to-US rigid/affine registration. In order to simulate the US image, the authors employed a model that was based on the physical principles of ultrasound. A locally evaluated statistical criterion was used to drive the registration. Michel and Paragios BIB048 used the mixture of experts methods to learn the conditional probability of the target intensity given a source patch. The conditional probability was then used to drive a Markov Random Field to regularize the simulated image. In the second case, both modalities are mapped to a common space. As both modalities image the same anatomical structure, the assumption can be made that the local geometry would be helpful to establish meaningful correspondences. Thus, in principle, most methods apply filters to extract geometrical information. This information is subsequently used in a mono-modal registration setting. Maintz et al. BIB013 tackled rigid multi-modal registration by using morphological tools to create new gray-value intensity images. The proposed method applied morphological opening and closing to extract edge information and then cross-correlation to align the images. It resembles a surface registration with the difference that instead of having binary values, real ones were used. Droske and Rumpf BIB018 proposed an approach that was motivated by the mathematical morphology theory that states that an image can be characterized uniquely by the entity of its level sets. The common space was defined by mapping every point to its normalized intensity gradient. The registration was formulated in a variational framework where a morphological, contrast invariant, matching criterion was minimized under the influence of an appropriate regularization term. Haber and Modersitzki BIB034 assumed that borders of anatomical structures correspond to intensity changes and thus opted to exploit intensity gradient information. An intermediate image domain was created by taking into account the normalized intensity gradient field. This field conveys purely geometric information and accounts for the fact that the gradient magnitude may vary among different modalities. The similarity function was based on the difference in angles between the normalized gradient vectors. Butz and Thiran BIB008 investigated the use of edge related information to cope with affine multi-modal registration. They used an edgeness operator that takes into account the local edge variance to map both images to a common space. Mutual information driven registration was then performed coupled with a multi-scale genetic optimization. Depending on the nature of the images, other operators may be applied. Penney et al. used the probability of vessel presence along with normalized cross-correlation to rigidly register MRI with ultrasound images BIB019 . Gabor filtering has also been used to map the images to a common domain because of their ability to capture local edge and texture information BIB001 . Liu et al. used local frequency representations to tackle rigid/affine multi-modal registration BIB014 . These representations are robust to edge strength and contrast differences. They were estimated by calculating the local phase gradient of the most significant Gabor filter response. Then, the integral squared error was chosen as the matching criterion. Jian et al. used local frequency maps for deformable registration BIB028 . The authors used the Riesz transform to estimate the local frequency information. Ou et al. used Gabor filters in deformable image registration BIB050 . Specifically, the responses of the filters were used to construct a rich vector descriptor. The images were aligned by minimizing a weighted sum of the vector differences. Andronache et al. tackled the problems related to the estimation of MI in small patches in BIB043 . Their strategy consisted of identifying the patches where the estimation of MI becomes unreliable and then mapping them to a common pseudo-modality. The pseudo-modality depicted only common structures in both images and was constructed by decreasing the variance of the mapped intensities. In the intermediate domain, simpler criteria may be used to drive registration. Recently, Heinrich et al. BIB055 presented a new descriptor for multi-modal registration. The driving idea behind the new descriptor is the use of similarities between neighboring patches as features. This idea is borrowed from the image denoising literature. Once the descriptor is constructed a vector-difference can be used as a matching criterion. Wachinger et al. BIB056 proposed two techniques that derive from information theory and manifold learning to create the intermediate structural representation. The first one used the entropy of a patch centered around the voxel to assign a new intensity value. The second method used Laplacian Eigenmaps to embed the patches in a lower-dimensional manifold that preserves local distances. Lee et al. presented a supervised technique to learn the similarity measure for multi-modal image registration BIB045 . The approach was formulated in a discriminative setting where the goal is to optimize a similarity function so that correct correspondences are assigned high values and erroneous ones low. Support vector machine regression was employed to learn the metric. Bronstein et al. presented a supervised technique whose aim was to learn a similarity metric that discerns between corresponding and noncorresponding points BIB049 . This technique maps both modalities to a Hamming metric space where true correspondences are likely to have the same code, while wrong ones are not. The embedding was constructed by using AdaBoost. Michel et al. investigated the application of the previous method to the problem of deformable registration BIB053 . It should also be noted that some of the techniques that were previously presented under the information theoretic class of methods learn a similarity measure. The difference is that a generative framework is employed. Given co-registered data, the joint distribution of the intensities is learned. Then, either a maximum likelihood approach BIB006 or a divergence criterion  - BIB030 , BIB040 is used to compare the estimated and learned distributions.