Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> Why <s> This survey provides an overview of higher-order tensor decompositions, their applications, and available software. A tensor is a multidimensional or $N$-way array. Decompositions of higher-order tensors (i.e., $N$-way arrays with $N \geq 3$) have applications in psycho-metrics, chemometrics, signal processing, numerical linear algebra, computer vision, numerical analysis, data mining, neuroscience, graph analysis, and elsewhere. Two particular tensor decompositions can be considered to be higher-order extensions of the matrix singular value decomposition: CANDECOMP/PARAFAC (CP) decomposes a tensor as a sum of rank-one tensors, and the Tucker decomposition is a higher-order form of principal component analysis. There are many other tensor decompositions, including INDSCAL, PARAFAC2, CANDELINC, DEDICOM, and PARATUCK2 as well as nonnegative variants of all of the above. The N-way Toolbox, Tensor Toolbox, and Multilinear Engine are examples of software packages for working with tensors. <s> BIB001 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> Why <s> Tensor decompositions are at the core of many blind source separation (BSS) algorithms, either explicitly or implicitly. In particular, the canonical polyadic (CP) tensor decomposition plays a central role in the identification of underdetermined mixtures. Despite some similarities, CP and singular value decomposition (SVD) are quite different. More generally, tensors and matrices enjoy different properties, as pointed out in this brief introduction. <s> BIB002 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> Why <s> The widespread use of multisensor technology and the emergence of big data sets have highlighted the limitations of standard flat-view matrix models and the necessity to move toward more versatile data analysis tools. We show that higher-order tensors (i.e., multiway arrays) enable such a fundamental paradigm shift toward models that are essentially polynomial, the uniqueness of which, unlike the matrix methods, is guaranteed under very mild and natural conditions. Benefiting from the power of multilinear algebra as their mathematical backbone, data analysis techniques using tensor decompositions are shown to have great flexibility in the choice of constraints which match data properties and extract more general latent components in the data than matrix-based methods. <s> BIB003 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> Why <s> We present structured data fusion (SDF) as a framework for the rapid prototyping of knowledge discovery in one or more possibly incomplete data sets. In SDF, each data set—stored as a dense, sparse, or incomplete tensor—is factorized with a matrix or tensor decomposition. Factorizations can be coupled, or fused, with each other by indicating which factors should be shared between data sets. At the same time, factors may be imposed to have any type of structure that can be constructed as an explicit function of some underlying variables. With the right choice of decomposition type and factor structure, even well-known matrix factorizations such as the eigenvalue decomposition, singular value decomposition and QR factorization can be computed with SDF. A domain specific language (DSL) for SDF is implemented as part of the software package Tensorlab, with which we offer a library of tensor decompositions and factor structures to choose from. The versatility of the SDF framework is demonstrated by means of four diverse applications, which are all solved entirely within Tensorlab’s DSL. <s> BIB004
matrices need not be full rank: identifiability of R ! 1 rank-1 terms is possible even if no factor matrix A; B; C; . . ., is of full rank. (4) Rank-1 terms are identifiable up to permutation: when a tensor decomposition is interpreted as joint analysis of lower order tensors, the arbitrary individual permutation that arises if each decomposition is done separately becomes common to all decompositions. (5) Increasing N allows uniqueness for higher R: more types of observational diversity allow to resolve more latent sources. (6) There is no need for structural constraints or assumptions such as statistical independence, nonnegativity, sparsity, or smoothness in order to achieve a unique decomposition. And yet, multilinear structures readily admit such additional types of diversity that can further contribute to interpretability, robustness, uniqueness, and other desired properties; see the end of Section III-D for examples. More properties of tensor decompositions and their uses in various engineering applications can be found, for example, in BIB001 , BIB003 , BIB002 , and references therein. Concluding Section III-C, Sections III-C1 and III-C2 presented two ways to look at matrices or tensors as data fusion structures. We have shown that matrix or tensor decompositions provide a natural framework to incorporate multiple types of observational diversity BIB004 on top of structural ones. We have shown that matrices and higher order tensors can be regarded as ways to jointly analyze multiple observations of the same data, when data sets share the same underlying structure BIB004 . It is thus no surprise that many multimodal data fusion models use matrix or tensor decompositions as their underlying analytical engine. Until now, we focused on decompositions in sum of rank-1 factors and statistical independence. In fact, these constraints can be regarded as too strong. Indeed, there exist other factorizations that may represent more flexible underlying relationships; see the end of Section III-D for examples. It is only for the sake of simplicity and limited space that we restrict our discussion to one type of decomposition.