Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Data-Driven Versus Model-Driven Methods <s> Simple structure and other common principles of factor rotation do not in general provide strong grounds for attributing explanatory significance to the factors which they select. In contrast, it is shown that an extension of Cattell's principle of rotation to Proportional Profiles (PP) offers a basis for determining explanatory factors for three-way or higher order multi-mode data. Conceptual models are developed for two basic patterns of multi-mode data variation, systemand object-variation, and PP analysis is found to apply in the system-variation case. Although PP was originally formulated as a principle of rotation to be used with classic two-way factor analysis, it is shown to embody a latent three-mode factor model, which is here made explicit and generalized frown two to N "parallel occasions". As originally formulated, PP rotation was restricted to orthogonal factors. The generalized PP model is demonstrated to give unique "correct" solutions with oblique, non-simple structure, and even non-linear factor structures. A series of tests, conducted with synthetic data of known factor composition, demonstrate the capabilities of linear and non-linear versions of the model, provide data on the minimal necessary conditions of uniqueness, and reveal the properties of the analysis procedures when these minimal conditions are not fulfilled. In addition, a mathematical proof is presented for the uniqueness of the solution given certain conditions on the data. Three-mode PP factor analysis is applied to a three-way set of real data consisting of the fundamental and first three formant frequencies of 11 persons saying 8 vowels. A unique solution is extracted, consisting of three factors which are highly meaningful and consistent with prior knowledge and theory concerning vowel quality. The relationships between the three-mode PP model and Tucker's multi-modal model, McDonald's non-linear model and Carroll and Chang's multi-dimensional scaling model are explored. <s> BIB001 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Data-Driven Versus Model-Driven Methods <s> This paper links the direct-sequence code-division multiple access (DS-CDMA) multiuser separation-equalization-detection problem to the parallel factor (PARAFAC) model, which is an analysis tool rooted in psychometrics and chemometrics. Exploiting this link, it derives a deterministic blind PARAFAC DS-CDMA receiver with performance close to non-blind minimum mean-squared error (MMSE). The proposed PARAFAC receiver capitalizes on code, spatial, and temporal diversity-combining, thereby supporting small sample sizes, more users than sensors, and/or less spreading than users. Interestingly, PARAFAC does not require knowledge of spreading codes, the specifics of multipath (interchip interference), DOA-calibration information, finite alphabet/constant modulus, or statistical independence/whiteness to recover the information-bearing signals. Instead, PARAFAC relies on a fundamental result regarding the uniqueness of low-rank three-way array decomposition due to Kruskal (1977, 1988) (and generalized herein to the complex-valued case) that guarantees identifiability of all relevant signals and propagation parameters. These and other issues are also demonstrated in pertinent simulation experiments. <s> BIB002 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Data-Driven Versus Model-Driven Methods <s> Simultaneous electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) provide complementary noninvasive information of brain activity, and EEG/fMRI fusion can achieve higher spatiotemporal resolution than each modality separately. This focuses on independent component analysis (ICA)-based EEG/fMRI fusion. In order to appreciate the issues, we first describe the potential and limitations of the developed fusion approaches: fMRI-constrained EEG imaging, EEG-informed fMRI analysis, and symmetric fusion. We then outline some newly developed hybrid fusion techniques using ICA and the combination of data-/model-driven methods, with special mention of the spatiotemporal EEG/fMRI fusion (STEFF). Finally, we discuss the current trend in methodological development and the existing limitations for extrapolating neural dynamics. <s> BIB003 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Data-Driven Versus Model-Driven Methods <s> The separation of speech signals measured at multiple microphones in noisy and reverberant environments using only the audio modality has limitations because there is generally insufficient information to fully discriminate the different sound sources. Humans mitigate this problem by exploiting the visual modality, which is insensitive to background noise and can provide contextual information about the audio scene. This advantage has inspired the creation of the new field of audiovisual (AV) speech source separation that targets exploiting visual modality alongside the microphone measurements in a machine. Success in this emerging field will expand the application of voice-based machine interfaces, such as Siri, the intelligent personal assistant on the iPhone and iPad, to much more realistic settings and thereby provide more natural human?machine interfaces. <s> BIB004 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Data-Driven Versus Model-Driven Methods <s> The widespread use of multisensor technology and the emergence of big data sets have highlighted the limitations of standard flat-view matrix models and the necessity to move toward more versatile data analysis tools. We show that higher-order tensors (i.e., multiway arrays) enable such a fundamental paradigm shift toward models that are essentially polynomial, the uniqueness of which, unlike the matrix methods, is guaranteed under very mild and natural conditions. Benefiting from the power of multilinear algebra as their mathematical backbone, data analysis techniques using tensor decompositions are shown to have great flexibility in the choice of constraints which match data properties and extract more general latent components in the data than matrix-based methods. <s> BIB005
Roughly, and for the sake of the discussion that follows, approaches to the problem in Section III-A can be divided into two groups: model driven and data driven. Modeldriven approaches rely on an explicit realistic model of the underlying processes  , BIB003 Sec. 3.3] ,  , and are generally successful if the assumptions are plausible and the model holds. However, model-driven methods may not always be the best choice, for example, when the underlying model of the signals or the medium in which they propagate is too complicated, varying rapidly, or simply unknown. In the context of multimodal data sets that are generated by complex systems as those mentioned in Sections I and II, very little is known about the underlying relationships between modalities. The interactions between data sets and data types are not always known or sufficiently understood. Therefore, we focus on and advocate a data-driven approach. In practice, this means making the fewest assumptions and using the simplest models, both within and across modalities BIB001 . ''Simple'' means, for example, linear relationships between variables, avoiding model-dependent parameters, and/or use of model-independent priors such as sparsity, nonnegativity, statistical independence, low rank, and smoothness, to name a few. As its name implies, a data-driven approach is self-contained in the sense that it relies only on the observations and their assumed model: it avoids external input BIB001 . For this reason, and especially in the signal processing community, data-driven methods are sometimes termed ''blind.'' In the rest of this section, we give a more concrete meaning to these ideas. Data-driven methods, both single modal and multimodal, have already proven successful in a broad range of problems and applications. A noncomprehensive list includes astrophysics  , biomedics  , telecommunications BIB002 , audiovision BIB004 , chemometrics  , and more. For further examples, see, e.g.,  - BIB005 and references therein, as well as the numerous models mentioned in the rest of this paper. In the rest of this section, we discuss and explain the role of diversity in achieving uniqueness in data-driven models. In particular, we demonstrate how the presence of multiple data sets can be exploited as a new form of diversity.