Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> (HMI). <s> There has been an ever-increasing interest in multi-disciplinary research on multisensor data fusion technology, driven by its versatility and diverse areas of application. Therefore, there seems to be a real need for an analytical review of recent developments in the data fusion domain. This paper proposes a comprehensive review of the data fusion state of the art, exploring its conceptualizations, benefits, and challenging aspects, as well as existing methodologies. In addition, several future directions of research in the data fusion community are highlighted and described. <s> BIB001 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> (HMI). <s> Abstract Multimodal fusion is an effective approach to better understand brain diseases. However, most such instances have been limited to pair-wise fusion; because there are often more than two imaging modalities available per subject, there is a need for approaches that can combine multiple datasets optimally. In this paper, we extended our previous two-way fusion model called “multimodal CCA + joint ICA”, to three or N-way fusion, that enables robust identification of correspondence among N data types and allows one to investigate the important question of whether certain disease risk factors are shared or distinct across multiple modalities. We compared “mCCA + jICA” with its alternatives in a 3-way fusion simulation and verified its advantages in both decomposition accuracy and modal linkage detection. We also applied it to real functional Magnetic Resonance Imaging (fMRI)–Diffusion Tensor Imaging (DTI) and structural MRI fusion to elucidate the abnormal architecture underlying schizophrenia (n = 97) relative to healthy controls (n = 116). Both modality-common and modality-unique abnormal regions were identified in schizophrenia. Specifically, the visual cortex in fMRI, the anterior thalamic radiation (ATR) and forceps minor in DTI, and the parietal lobule, cuneus and thalamus in sMRI were linked and discriminated between patients and controls. One fMRI component with regions of activity in motor cortex and superior temporal gyrus individually discriminated schizophrenia from controls. Finally, three components showed significant correlation with duration of illness (DOI), suggesting that lower gray matter volumes in parietal, frontal, and temporal lobes and cerebellum are associated with increased DOI, along with white matter disruption in ATR and cortico-spinal tracts. Findings suggest that the identified fractional anisotropy changes may relate to the corresponding functional/structural changes in the brain that are thought to play a role in the clinical expression of schizophrenia. The proposed “mCCA + jICA” method showed promise for elucidating the joint or coupled neuronal abnormalities underlying mental illnesses and improves our understanding of the disease process. <s> BIB002
Example V-A.1: Use of Features in Cosmological Inference From CMB Observations: In the inference of cosmological parameters from CMB observations (Example II-C.3), the raw data consist of detector readouts as well as other auxiliary information that amounts to several terabytes, or Oð10 12 Þ, of observations  . The scientific products are usually provided in several levels of multivariate ''features,'' as follows: 1) full-sky maps, of CMB and non-CMB emissions, amounting to roughly Oð10 8 Þ pixels; 2) CMB power spectrum computed from the CMB spatial map, at Oð10 3 Þ spectral multipoles; and 3) six cosmological parameters that represent the best fit of the CMB power spectrum to the LCDM model. It is clear that each level represents a strong compression of the data w.r.t. the preceding one. Each level of features is useful for a different type of inference. High-resolution component maps are the first useful outcome from the component separation procedure  . Apart from providing valuable information about the sky, they are useful, for instance, for consistency checks between instruments, experiments, and methods  ,  . Power spectra are useful to compare outcomes of different experiments that measure the CMB, e.g., Planck and BICEP2/Keck  , whereas cosmological parameters form the link, via the LCDM model, with data sets that do not involve astrophysical observations, e.g., high-energy physics at CERN  . Order selection and dimension reduction. Related to the open issue of choosing the most appropriate strategy of data fusion is order selection. As in nonmultimodal analysis, a dimension reduction step may be required in order to avoid overfitting the data, as well as a form of compression BIB001 . In a data fusion framework, this step must take into consideration the possibly different representations of the latent variables across data sets. As an example, a solution that maximally retains the joint information while also ensuring that the extracted sources are independent from each other, in the context of a ''joint ICA''-based approach, is proposed in BIB002 . Dimension reduction may be performed locally, at each sensor or modality, or at a central processing unit BIB001 .