The author proposes a method for simultaneous registration and segmentation of multi-source images, using the multivariate mixture model (MvMM) and maximum of log-likelihood (LL) framework. Specifically, the method is applied to the myocardial segmentation combining the complementary information from multi-sequence (MS) cardiac magnetic resonance (CMR) images. For the image misalignment and incongruent data, the MvMM is formulated with transformations and is further generalized for dealing with the hetero-coverage multi-modality images (HC-MMIs). The segmentation of MvMM is performed in a virtual common space, to which all the images and misaligned slices are simultaneously registered. Furthermore, this common space can be divided into a number of sub-regions, each of which contains congruent data, thus the HC-MMIs can be modeled using a set of conventional MvMMs. Results show that MvMM obtained significantly better performance compared to the conventional approaches and demonstrated good potential for scar quantification as well as myocardial segmentation. The generalized MvMM has also demonstrated better robustness in the incongruent data, where some images may not fully cover the region of interest, and the full coverage can only be reconstructed combining the images from multiple sources.
In the literature, there is a research topic referred to as multivariate image analysis, which has been proposed to deal with images that have more than one measurement per pixel @cite_7 @cite_3 . This research has particularly focused on the three-channel (RGB) color images or multispectral and hyperspectral images, and the methods have been designed to efficiently compress the highly correlated data and project them onto a reduced dimensional subspace, for example using the principal component analysis @cite_1 . Multivariate image analysis considers only data, meaning for each pixel in one image there should be a corresponding pixel in the other images @cite_3 . However, in medical imaging the multi-source images commonly do not have this congruency, as the images acquired from different sources can be misaligned to each other, and the imaging field-of-views and resolution can vary greatly across different acquisitions. The images having such differences are referred to as images in this work.
@cite_3 @cite_1 @cite_7
2068324025 2394774286 2612585477 2339752168
Abstract Nowadays, image analysis is becoming more important because of its ability to perform fast and non-invasive low-cost analysis on products and processes. Image analysis is a wide denomination that encloses classical studies on gray scale or RGB images, analysis of images collected using few spectral channels (sometimes called multispectral images) or, most recently, data treatments to deal with hyperspectral images, where the spectral direction is exploited in its full extension. Pioneering data treatments in image analysis were applied to simple images mainly for defect detection, segmentation and classification by the Computer Science community. From the late 80s, the chemometric community joined this field introducing powerful tools for image analysis, which were already in use for the study of classical spectroscopic data sets and were appropriately modified to fit the particular characteristics of image structures. These chemometric approaches adapt to images of all kinds, from the simplest to the hyperspectral images, and have provided new insights on the spatial and spectroscopic information of this kind of data sets. New fields open by the introduction of chemometrics on image analysis are exploratory image analysis, multivariate statistical process control (monitoring), multivariate image regression or image resolution. This paper reviews the different techniques developed in image analysis and shows the evolution in the information provided by the different methodologies, which has been heavily pushed by the increasing complexity of the image measurements in the spatial and, particularly, in the spectral direction. The development of multisensor systems in recent years has led to great increase in the amount of available remote sensing data. Image fusion techniques aim at inferring high quality images of a given area from degraded versions of the same area obtained by multiple sensors. This paper focuses on pansharpening, which is the inference of a high spatial resolution multispectral image from two degraded versions with complementary spectral and spatial resolution characteristics: 1) a low spatial resolution multispectral image and 2) a high spatial resolution panchromatic image. We introduce a new variational model based on spatial and spectral sparsity priors for the fusion. In the spectral domain, we encourage low-rank structure, whereas in the spatial domain, we promote sparsity on the local differences. Given the fact that both panchromatic and multispectral images are integrations of the underlying continuous spectra using different channel responses, we propose to exploit appropriate regularizations based on both spatial and spectral links between panchromatic and fused multispectral images. A weighted version of the vector total variation norm of the data matrix is employed to align the spatial information of the fused image with that of the panchromatic image. With regard to spectral information, two different types of regularization are proposed to promote a soft constraint on the linear dependence between the panchromatic and fused multispectral images. The first one estimates directly the linear coefficients from the observed panchromatic and low-resolution multispectral images by linear regression while the second one employs the principal component pursuit to obtain a robust recovery of the underlying low-rank structure. We also show that the two regularizers are strongly related. The basic idea of both regularizers is that the fused image should have low-rank and preserve edge locations. We use a variation of the recently proposed split augmented Lagrangian shrinkage algorithm to effectively solve the proposed variational formulations. Experimental results on simulated and real remote sensing images show the effectiveness of the proposed pansharpening method compared with the state-of-the-art. Magnetic Resonance Imaging (MRI) offers high-resolution in vivo imaging and rich functional and anatomical multimodality tissue contrast. In practice, however, there are challenges associated with considerations of scanning costs, patient comfort, and scanning time that constrain how much data can be acquired in clinical or research studies. In this paper, we explore the possibility of generating high-resolution and multimodal images from low-resolution single-modality imagery. We propose the weakly-supervised joint convolutional sparse coding to simultaneously solve the problems of super-resolution (SR) and cross-modality image synthesis. The learning process requires only a few registered multimodal image pairs as the training set. Additionally, the quality of the joint dictionary learning can be improved using a larger set of unpaired images. To combine unpaired data from different image resolutions modalities, a hetero-domain image alignment term is proposed. Local image neighborhoods are naturally preserved by operating on the whole image domain (as opposed to image patches) and using joint convolutional sparse coding. The paired images are enhanced in the joint learning process with unpaired data and an additional maximum mean discrepancy term, which minimizes the dissimilarity between their feature distributions. Experiments show that the proposed method outperforms state-of-the-art techniques on both SR reconstruction and simultaneous SR and cross-modality synthesis. Imaging is a powerful tool being used in many disciplines such as engineering, physics, biology and medicine to name a few. Recent years have seen a trend that imaging modalities have been combined to create multi-modality imaging tools where different modalities acquire complementary information. For example, in medical imaging, positron emission tomography (PET) and magnetic resonance imaging (MRI) are combined to image structure and function of the human body. Another example is spectral imaging where each channel provides information about a different wave length, e.g. information about red, green and blue (RGB). Most imaging modalities do not acquire images directly but measure a quantity from which we can reconstruct an image. These inverse problems require a priori information in order to give meaningful solutions. Assumptions are often on the smoothness of the solution but other information is sometimes available, too. Many multi-modality images show a strong inter-channel correlation as they are acquired from the same anatomy in medical imaging or the same scenery in spectral imaging. However, images from different modalities are usually reconstructed separately. In this thesis we aim to exploit this correlation using the data from all modalities, that are present in the acquisition, in a joint reconstruction process with the assumption that similar structures in all channels are more likely. We propose a framework for joint reconstruction where modalities are coupled by additional information about the solution we seek. A family of priors -- called parallel level sets -- allows us to incorporate structural a priori knowledge into the reconstruction. We analyse the parallel level set priors in several aspects including their convexity and the diffusive flow generated by their variation. Several numerical examples in RGB colour imaging and in PET-MRI illustrate the gain of joint reconstruction and in particular of the parallel level set priors.