Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> This paper is concerned with the study of similarities and differences in factor structures between different groups. A common situation occurs when a battery of tests has been administered to samples of examinees from several populations.A very general model is presented, in which any parameter in the factor analysis models (factor loadings, factor variances, factor covariances, and unique variances) for the different groups may be assigned an arbitrary value or constrained to be equal to some other parameter. Given such a specification, the model is estimated by the maximum likelihood method yielding a large samplex2 of goodness of fit. By computing several solutions under different specifications one can test various hypotheses.The method is capable of dealing with any degree of invariance, from the one extreme, where nothing is invariant, to the other extreme, where everything is invariant. Neither the number of tests nor the number of common factors need to be the same for all groups, but to be at all interesting, it is assumed that there is a common core of tests in each battery that is the same or at least content-wise comparable. <s> BIB001 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> In most data fusion systems, the information extracted from each image or sensor (either numerical or symbolic) is represented as a degree of belief in an event with real values, generally in [0,1], taking in this way into account the imprecise, uncertain and incomplete nature of information. The combination of such degrees of belief is performed through numerical fusion operators. A very large variety of such operators has been proposed in the literature. We propose a classification of these operators issued from different data fusion theories (probabilities, fuzzy sets, possibilities, Dempster- Shafer, etc.) w.r.t. their behaviour. Three classes are thus defined: context independent constant behaviour operators, which have the same behaviour (compromise, disjunction or conjunction) whatever the values to be combined, context independent variable behaviour operators (whose behaviour depends on the combined values), and context dependent operators (where the result depends also on some global knowledge like conflict, reliability of sources). This classification provides a guide for choosing an operator in a given problem. This choice can then be refined from the desired properties of the operators, from their decisiveness, and by examining how they deal with conflictual situations. These aspects are illustrated on simple examples in multispectral satellite imaging. In particular, we stress the interest of the third class for classification problems. <s> BIB002 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> We present a new approach to modeling and processing multimedia data. This approach is based on graphical models that combine audio and video variables. We demonstrate it by developing a new algorithm for tracking a moving object in a cluttered, noisy scene using two microphones and a camera. Our model uses unobserved variables to describe the data in terms of the process that generates them. It is therefore able to capture and exploit the statistical structure of the audio and video data separately, as well as their mutual dependencies. Model parameters are learned from data via an EM algorithm, and automatic calibration is performed as part of this procedure. Tracking is done by Bayesian inference of the object location from data. We demonstrate successful performance on multimedia clips captured in real world scenarios using off-the-shelf equipment. <s> BIB003 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> One of the major problems in sensor fusion is that sensors frequently provide spurious observations which are difficult to predict and model. The spurious measurements from sensors must be identified and eliminated since their incorporation in the fusion pool might lead to inaccurate estimation. This paper presents a unified sensor fusion strategy based on a modified Bayesian approach that can automatically identify the inconsistency in sensor measurements so that the spurious measurements can be eliminated from the data fusion process. The proposed method adds a term to the commonly used Bayesian formulation. This term is an estimate of the probability that the data is not spurious, based upon the measured data and the unknown value of the true state. In fusing two measurements, it has the effect of increasing the variance of the posterior distribution when measurement from one of the sensors is inconsistent with respect to the other. The increase or decrease in variance can be estimated using the information theoretic measure "entropy." The proposed strategy was verified with the help of extensive computations performed on simulated data from three sensors. A comparison was made between two different fusion schemes: centralized fusion in which data obtained from all sensors were fused simultaneously, and a decentralized or sequential Bayesian scheme that proved useful for identifying and eliminating spurious data from the fusion process. The simulations verified that the proposed strategy was able to identify spurious sensor measurements and eliminate them from the fusion process, thus leading to a better overall estimate of the true state. The proposed strategy was also validated with the help of experiments performed using stereo vision cameras, one infrared proximity sensor, and one laser proximity sensor. The information from these three sensing sources was fused to obtain an occupancy profile of the robotic workspace <s> BIB004 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> We present a unified view of matrix factorization that frames the differences among popular methods, such as NMF, Weighted SVD, E-PCA, MMMF, pLSI, pLSI-pHITS, Bregman co-clustering, and many others, in terms of a small number of modeling choices. Many of these approaches can be viewed as minimizing a generalized Bregman divergence, and we show that (i) a straightforward alternating projection algorithm can be applied to almost any model in our unified view; (ii) the Hessian for each projection has special structure that makes a Newton projection feasible, even when there are equality constraints on the factors, which allows for matrix co-clustering; and (iii) alternating projections can be generalized to simultaneously factor a set of matrices that share dimensions. These observations immediately yield new optimization algorithms for the above factorization methods, and suggest novel generalizations of these methods such as incorporating row and column biases, and adding or relaxing clustering constraints. <s> BIB005 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> The acquisition of multiple brain imaging types for a given study is a very common practice. There have been a number of approaches proposed for combining or fusing multitask or multimodal information. These can be roughly divided into those that attempt to study convergence of multimodal imaging, for example, how function and structure are related in the same region of the brain, and those that attempt to study the complementary nature of modalities, for example, utilizing temporal EEG information and spatial functional magnetic resonance imaging information. Within each of these categories, one can attempt data integration (the use of one imaging modality to improve the results of another) or true data fusion (in which multiple modalities are utilized to inform one another). We review both approaches and present a recent computational approach that first preprocesses the data to compute features of interest. The features are then analyzed in a multivariate manner using independent component analysis. We describe the approach in detail and provide examples of how it has been used for different fusion tasks. We also propose a method for selecting which combination of modalities provides the greatest value in discriminating groups. Finally, we summarize and describe future research topics. <s> BIB006 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> As a consequence of our information society, not only more and larger data sets become available, but also data sets that include multiple sorts of information regarding the same system. Such data sets can be denoted by the terms coupled, linked, or multiset data, and the associated data analysis can be denoted by the term data fusion. In this paper, we first give a formal description of coupled data, which allows the data-analyst to typify the structure of a coupled data set at hand. Second, we list two meta-questions and a series of complicating factors that may be useful to focus the initial content-driven research questions that go with coupled data, and to choose a suitable data-analytic method. Third, we propose a generic framework for a family of decomposition-based models pertaining to an important subset of data fusion problems. This framework is intended to constitute both a means to arrive at a better understanding of the features and the interrelations of the specific models subsumed by it, and as a powerful device for the development of novel, custom-made data fusion models. We conclude the paper by showing how the proposed formal data description, meta-questions, and generic model may assist the data-analyst in choosing and developing suitable strategies for the treatment of coupled data in practice. Throughout the paper we illustrate with examples from the domain of systems biology. <s> BIB007 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> Abstract The problem of incomplete data – i.e., data with missing or unknown values – in multi-way arrays is ubiquitous in biomedical signal processing, network traffic analysis, bibliometrics, social network analysis, chemometrics, computer vision, communication networks, etc. We consider the problem of how to factorize data sets with missing values with the goal of capturing the underlying latent structure of the data and possibly reconstructing missing values (i.e., tensor completion). We focus on one of the most well-known tensor factorizations that captures multi-linear structure, CANDECOMP/PARAFAC (CP). In the presence of missing data, CP can be formulated as a weighted least squares problem that models only the known entries. We develop an algorithm called CP-WOPT (CP Weighted OPTimization) that uses a first-order optimization approach to solve the weighted least squares problem. Based on extensive numerical experiments, our algorithm is shown to successfully factorize tensors with noise and up to 99% missing data. A unique aspect of our approach is that it scales to sparse large-scale data, e.g., 1000 × 1000 × 1000 with five million known entries (0.5% dense). We further demonstrate the usefulness of CP-WOPT on two real-world applications: a novel EEG (electroencephalogram) application where missing data is frequently encountered due to disconnections of electrodes and the problem of modeling computer network traffic where data may be absent due to the expense of the data collection process. <s> BIB008 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> In recent years, neuroimaging studies have increasingly been acquiring multiple modalities of data and searching for task- or disease-related changes in each modality separately. A major challenge in analysis is to find systematic approaches for fusing these differing data types together to automatically find patterns of related changes across multiple modalities, when they exist. Independent Component Analysis (ICA) is a popular unsupervised learning method that can be used to find the modes of variation in neuroimaging data across a group of subjects. When multimodal data is acquired for the subjects, ICA is typically performed separately on each modality, leading to incompatible decompositions across modalities. Using a modular Bayesian framework, we develop a novel "Linked ICA" model for simultaneously modelling and discovering common features across multiple modalities, which can potentially have completely different units, signal- and contrast-to-noise ratios, voxel counts, spatial smoothnesses and intensity distributions. Furthermore, this general model can be configured to allow tensor ICA or spatially-concatenated ICA decompositions, or a combination of both at the same time. Linked ICA automatically determines the optimal weighting of each modality, and also can detect single-modality structured components when present. This is a fully probabilistic approach, implemented using Variational Bayes. We evaluate the method on simulated multimodal data sets, as well as on a real data set of Alzheimer's patients and age-matched controls that combines two very different types of structural MRI data: morphological data (grey matter density) and diffusion data (fractional anisotropy, mean diffusivity, and tensor mode). <s> BIB009 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> In many areas of science, research questions imply the analysis of a set of coupled data blocks, with, for instance, each block being an experimental unit by variable matrix, and the variables being the same in all matrices. To obtain an overall picture of the mechanisms that play a role in the different data matrices, the information in these matrices needs to be integrated. This may be achieved by applying a data-analytic strategy in which a global model is fitted to all data matrices simultaneously, as in some forms of simultaneous component analysis (SCA). Since such a strategy implies that all data entries, regardless the matrix they belong to, contribute equally to the analysis, it may obfuscate the overall picture of the mechanisms underlying the data when the different data matrices are subject to different amounts of noise. One way out is to downweight entries from noisy data matrices in favour of entries from less noisy matrices. Information regarding the amount of noise that is present in each matrix, however, is, in most cases, not available. To deal with these problems, in this paper a novel maximum-likelihood-based simultaneous component analysis method, referred to as MxLSCA, is proposed. Being a stochastic extension of SCA, in MxLSCA the amount of noise in each data matrix is estimated and entries from noisy data matrices are downweighted. Both in an extensive simulation study and in an application to data stemming from cross-cultural emotion psychology, it is shown that the novel MxLSCA strategy outperforms the SCA strategy with respect to disclosing the mechanisms underlying the coupled data. <s> BIB010 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> In many research domains different pieces of information are collected regarding the same set of objects. Each piece of information constitutes a data block, and all these (coupled) blocks have the object mode in common. When analyzing such data, an important aim is to obtain an overall picture of the structure underlying the whole set of coupled data blocks. A further challenge consists of accounting for the differences in information value that exist between and within (i.e., between the objects of a single block) data blocks. To tackle these issues, analysis techniques may be useful in which all available pieces of information are integrated and in which at the same time noise heterogeneity is taken into account. For the case of binary coupled data, however, only methods exist that go for a simultaneous analysis of all data blocks but that do not account for noise heterogeneity. Therefore, in this paper, the SIMCLAS model, being a Hierarchical Classes model for the simultaneous analysis of coupled binary two-way matrices, is presented. In this model, noise heterogeneity between and within the data blocks is accounted for by downweighting entries from noisy blocks/objects within a block. In a simulation study it is shown that (1) the SIMCLAS technique recovers the underlying structure of coupled data to a very large extent, and (2) the SIMCLAS technique outperforms a Hierarchical Classes technique in which all entries contribute equally to the analysis (i.e., noise homogeneity within and between blocks). The latter is also demonstrated in an application of both techniques to empirical data on categorization of semantic concepts. <s> BIB011 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> There has been an ever-increasing interest in multi-disciplinary research on multisensor data fusion technology, driven by its versatility and diverse areas of application. Therefore, there seems to be a real need for an analytical review of recent developments in the data fusion domain. This paper proposes a comprehensive review of the data fusion state of the art, exploring its conceptualizations, benefits, and challenging aspects, as well as existing methodologies. In addition, several future directions of research in the data fusion community are highlighted and described. <s> BIB012 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> Incorporating domain-specific side information via coupled factorization models is useful in source separation applications. Coupled models can easily incorporate information from source modalities with different statistical properties by estimating shared factors via divergence minimization. Here, it is useful to use mixed divergences, a specific divergence for each modality. However, this extra freedom requires choosing the correct divergence as well as an optimal weighting mechanism to select the relative ‘importance’. In this paper, we present an approach for determining the relative weights, framed as dispersion parameter estimation, based on an inference framework introduced previously as Generalized Coupled Tensor Factorization (GCTF). The dispersion parameters play a key role on inference as they form a balance between the information obtained from multimodal observations. We tackle the problem of optimal weighting by maximum likelihood exploiting the relation between β-divergences and the family of Tweedie distributions. We demonstrate the usefulness of our approach on a drum source separation application. <s> BIB013 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> Autonomous vehicle board an increasing quantity of sensors such as GNSS receivers, road maps, LIDARs, to ensure a reliable localization function. This expanding number leads to abandon a classical centralised data fusion method and to adopt more complex architectures such as Distributed Data Fusion. In this paper, we propose an approach to detect faults of the different sensors in such configuration and to dynamically reconfigure the fusion method by using simple concepts of Information Theory. In order to detect faults, consistency is examined through a log likelihood ratio between the information innovation of each sensor. The current study illustrates the performance of the proposed fault detection algorithm and the pertinence of the dynamical reconfiguration of the multi-sensors data fusion. Experimental results, using data from antilock braking system sensors, a differential global positioning system receiver, and an accurate digital roadmap illustrate the performance of the proposed approach. <s> BIB014 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> Abstract Recent technological advances enable us to collect huge amounts of data from multiple sources. Jointly analyzing such multi-relational data from different sources, i.e., data fusion (also called multi-block, multi-view or multi-set data analysis), often enhances knowledge discovery. For instance, in metabolomics, biological fluids are measured using a variety of analytical techniques such as Liquid Chromatography–Mass Spectrometry and Nuclear Magnetic Resonance Spectroscopy. Data measured using different analytical methods may be complementary and their fusion may help in the identification of chemicals related to certain diseases. Data fusion has proved useful in many fields including social network analysis, collaborative filtering, neuroscience and bioinformatics. In this paper, unlike many studies demonstrating the success of data fusion, we explore the limitations as well as the advantages of data fusion. We formulate data fusion as a coupled matrix and tensor factorization (CMTF) problem, which jointly factorizes multiple data sets in the form of higher-order tensors and matrices by extracting a common latent structure from the shared mode. Using numerical experiments on simulated and real data sets, we assess the performance of coupled analysis compared to the analysis of a single data set in terms of missing data estimation and demonstrate cases where coupled analysis outperforms analysis of a single data set and vice versa. <s> BIB015 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> This study deals with missing link prediction, the problem of predicting the existence of missing connections between entities of interest. We approach the problem as filling in missing entries in a relational dataset represented by several matrices and multiway arrays, that will be simply called tensors. Consequently, we address the link prediction problem by data fusion formulated as simultaneous factorization of several observation tensors where latent factors are shared among each observation. Previous studies on joint factorization of such heterogeneous datasets have focused on a single loss function (mainly squared Euclidean distance or Kullback---Leibler-divergence) and specific tensor factorization models (CANDECOMP/PARAFAC and/or Tucker). However, in this paper, we study various alternative tensor models as well as loss functions including the ones already studied in the literature using the generalized coupled tensor factorization framework. Through extensive experiments on two real-world datasets, we demonstrate that (i) joint analysis of data from multiple sources via coupled factorization significantly improves the link prediction performance, (ii) selection of a suitable loss function and a tensor factorization model is crucial for accurate missing link prediction and loss functions that have not been studied for link prediction before may outperform the commonly-used loss functions, (iii) joint factorization of datasets can handle difficult cases, such as the cold start problem that arises when a new entity enters the dataset, and (iv) our approach is scalable to large-scale data. <s> BIB016 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> Abstract. One of the main challenges for meteorological and hydrological modelling is accurate rainfall measurement and mapping across time and space. To date, the most effective methods for large-scale rainfall estimates are radar, satellites, and, more recently, received signal level (RSL) measurements derived from commercial microwave networks (CMNs). While these methods provide improved spatial resolution over traditional rain gauges, they have their limitations as well. For example, wireless CMNs, which are comprised of microwave links (ML), are dependant upon existing infrastructure and the ML' arbitrary distribution in space. Radar, on the other hand, is known in its limitation for accurately estimating rainfall in urban regions, clutter areas and distant locations. In this paper the pros and cons of the radar and ML methods are considered in order to develop a new algorithm for improving rainfall measurement and mapping, which is based on data fusion of the different sources. The integration is based on an optimal weighted average of the two data sets, taking into account location, number of links, rainfall intensity and time step. Our results indicate that, by using the proposed new method, we not only generate more accurate 2-D rainfall reconstructions, compared with actual rain intensities in space, but also the reconstructed maps are extended to the maximum coverage area. By inspecting three significant rain events, we show that our method outperforms CMNs or the radar alone in rain rate estimation, almost uniformly, both for instantaneous spatial measurements, as well as in calculating total accumulated rainfall. These new improved 2-D rainfall maps, as well as the accurate rainfall measurements over large areas at sub-hourly timescales, will allow for improved understanding, initialization, and calibration of hydrological and meteorological models mainly necessary for water resource management and planning. <s> BIB017 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> Higher-order tensors and their decompositions are abundantly present in domains such as signal processing (e.g., higher-order statistics [1] and sensor array processing [2]), scientific computing (e.g., discretized multivariate functions [3]?[6]), and quantum information theory (e.g., representation of quantum many-body states [7]). In many applications, the possibly huge tensors can be approximated well by compact multilinear models or decompositions. Tensor decompositions are more versatile tools than the linear models resulting from traditional matrix approaches. Compared to matrices, tensors have at least one extra dimension. The number of elements in a tensor increases exponentially with the number of dimensions, and so do the computational and memory requirements. The exponential dependency (and the problems that are caused by it) is called the curse of dimensionality. The curse limits the order of the tensors that can be handled. Even for a modest order, tensor problems are often large scale. Large tensors can be handled, and the curse can be alleviated or even removed by using a decomposition that represents the tensor instead of using the tensor itself. However, most decomposition algorithms require full tensors, which renders these algorithms infeasible for many data sets. If a tensor can be represented by a decomposition, this hypothesized structure can be exploited by using compressed sensing (CS) methods working on incomplete tensors, i.e., tensors with only a few known elements. <s> BIB018 </s> Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects <s> B. Challenges Due to Various Types of Uncertainty <s> We present structured data fusion (SDF) as a framework for the rapid prototyping of knowledge discovery in one or more possibly incomplete data sets. In SDF, each data set—stored as a dense, sparse, or incomplete tensor—is factorized with a matrix or tensor decomposition. Factorizations can be coupled, or fused, with each other by indicating which factors should be shared between data sets. At the same time, factors may be imposed to have any type of structure that can be constructed as an explicit function of some underlying variables. With the right choice of decomposition type and factor structure, even well-known matrix factorizations such as the eigenvalue decomposition, singular value decomposition and QR factorization can be computed with SDF. A domain specific language (DSL) for SDF is implemented as part of the software package Tensorlab, with which we offer a library of tensor decompositions and factor structures to choose from. The versatility of the SDF framework is demonstrated by means of four diverse applications, which are all solved entirely within Tensorlab’s DSL. <s> BIB019
We now turn to discussing uncertainty in the data. Any real-world set of observations is prone to various types of uncertainty. The presence of heterogeneous multiple data sets creates new types of uncertainty that may also be heterogeneous. We argue that in such cases, it is the complementarity and diversity (Definition I.1) of the data sets that should be exploited to resolve these challenges BIB012 . Challenge IV-B.1: Noise: Thermal noise, calibration errors, finite precision, quantization or any other quality degradation in the measurements is unavoidable. For simplicity, we denote all these unavoidable phenomena as ''noise.'' Naturally, each acquisition method produces not only heterogeneous types of desired data, but also heterogeneous types of errors BIB007 . The question of how to jointly weigh or balance different sources of error is brought up in a number of data fusion scenarios, although most data fusion work currently ignores noise. Naturally, in the presence of noise, an appropriate model yields a more precise inference. Several authors BIB001 - BIB009 use an additive noise model with a distribution whose parameters may vary within and across data sets. A Bayesian or maximumlikelihood (ML) framework is then applied to estimate the noise parameters. In some cases, the noise estimates are interpreted as weights that balance the contribution of each element BIB010 (note the link with Challenge IV-B.2). Beal et al. BIB003 propose a graphical model for audiovisual object tracking in which they attribute different parameters to audio and video noise, and estimate both in a Bayesian inference framework. All these methods assume independence among sources of noise across modalities. However, ignoring possible links (correlations) between noise across data sets may lead to bias BIB012 . Challenge IV-B.2: Balancing Information From Different Origins: In practice, for various reasons, not all observations or data entries have the same level of confidence, reliability or information quality  , BIB006 , BIB011 . Below, we list scenarios in which this occurs, as well as some approaches to resolve these problems. In real-life scenarios, certain sensors may be provide information that has more value than others, or certain measurements might be taken at better controlled scenarios. For example, in a medical questionnaire about patients Â symptoms, certain symptoms may be more obvious, whereas others may be harder to define BIB011 . In the same vein, heterogeneity of acquisition methods implies heterogeneity in their level of importance or usefulness. For example, we may use questionnaires filled by specialists (experts) and others by patients (nonexperts). Alternatively, if we consider two medical questionnaires, patients Â symptoms and patients Â diagnosis, the first one may be more reliable since symptoms are observed directly whereas diagnosis relies on interpretation BIB011 . In order to address this issue, Wilderjans et al. BIB010 , BIB011 propose to associate the level of reliability with noise, and use appropriate weights, obtained via an ML variant of simultaneous component analysis (SCA). yimzekli et al. BIB013 propose individual weights for data sets with different divergence measures based on their relative ''importance.'' Similar to BIB010 , these weights are interpreted as noise variances. Finally, in BIB017 , Liberman et al. process several meteorological monitoring modalities separately and then make a soft decision, using an optimal weighted average based on location, number of links, rainfall intensity and other parameters. Another source of potential unbalance is data sets of different size (recall Challenge IV-A.3). In the absence of additional assumptions, a simulation study favors equal weight to each data entry regardless of its data set of origin, over the alternative approach of weighting data sets by the number of their respective entries  . This approach is generalized to the case of missing values, where the weights should be proportional to the number of nonmissing entries in each data set BIB019 . Challenge IV-B.3: Conflicting, Contradicting, or Inconsistent Data: Whenever more than one origin of information is available, be it a single sensor or an ensemble of observations, conflicts, contradictions, and inconsistencies may occur. If data are fused at the decision level, then a decision or voting BIB007 rule may be applied, as in the fusion of different classification maps in remote sensing (see Example II-C.1). Other approaches, related to multisensor data fusion, are discussed in BIB012 and BIB002 . When only two data sets are confronted, more elaborate solutions may be required. An obvious challenge is to devise a suitable compromise. A more fundamental challenge, however, is identifying these inconsistencies. In BIB014 , Tmazirte et al. consider the problem of detecting faults in multimodal sensors in a distributed data fusion framework, and dynamically reconfigure the system using information theoretical concepts. Their approach is based on detecting inconsistency in the mutual information contribution of each sensor w.r.t. its history. Kumar et al. BIB004 deal with the problem of multimodal sensors that occasionally produce spurious data, possibly due to sensor failure or environmental issues, and thus may bias estimation. The challenge arises from the fact that spurious events are difficult to predict and to model. Kumar et al. BIB004 propose a Bayesian approach that can identify and eliminate spurious data from a sensor. The procedure attributes less weight to the measurement from a suspected sensor when fused with measurements from other sensors. In the inference of cosmological parameters (Example II-C.3), detecting and explaining (in)consistencies of observations from different experiments is of utmost importance. A methodology for validation is comparing various error measures on several types of analytical products (Example V-A.1): cosmological parameters, CMB power spectra, and full sky maps, with and without the inclusion of data sets from both space-borne satellite missions and Earth-bound telescopes  . These experiments vary in the spectral bands at which they observe the sky, angular (spatial) resolution, sensitivity to different types of polarization, sky coverage, skyscanning strategies  Sec. 4 .1], types of noise, and other parameters. Therefore, they carry complementary information. Challenge IV-B.4: Missing Values: The challenge of missing values is not new and not unique to data fusion. The problem of matrix and tensor completion is longstanding in linear and multilinear algebra. However, its prevalence in data fusion draws special attention to it. In Section III, we have seen that low-rank tensor decompositions provide redundancy that results in strong uniqueness that is further improved in the presence of coupling or additional constraints. It turns out that the same applies also in the case of missing values; see BIB019 , BIB015 , BIB008 - BIB018 , and references therein. Approaches to missing values that are motivated by various aspects of data fusion can be found, e.g., in BIB019 , BIB008 , and BIB016 . Missing values may occur in various scenarios. While the first case that we mention below is not unique to data fusion, the remaining ones are. More specifically, the first case deals with samples that are locally missing within an individual data set, whereas the other cases arise due to interaction among data sets. First, certain data entries may be unreliable, discarded, or unavailable due to faulty detectors, occlusions, partial coverage, or any other unavoidable effects. Second, sometimes a modality can report only on part of the system w.r.t. the other modalities, as is the case with EEG versus MEG  , nuclear magnetic resonance (NMR) versus liquid chromatography/mass spectrometry (LC-MS) BIB015 , occlusions or partial spatial coverage in remote sensing (Example II-C.1), audiovideo (Example II-A.1), meteorological monitoring (Example II-C.2), and HMI (Example II-A.2). A third scenario is illustrated in Fig. 2(b) . In this case, there exist several data sets, depicted as complete third-order tensors. However, when linked together, they can be regarded as elements in a larger thirdorder tensor in which they are all contained, and whose volume is only partially filled. Fourth, data may be regarded as structurally missing if samples at different modalities are not taken at comparable sampling points BIB007 (recall Challenge IV-A.2), and we would like to construct a more complete picture from the entire sample set. In this case, each modality is properly sampled on its own, but there exist points on the common sampling grid that do not contain data from all modalities; these points can be regarded as missing values. A fifth scenario is link prediction. This is a common issue in recommender systems and social network analysis. In social network analysis, the challenge is predicting social links or activities based on an existing database of connections or activities, where only a few entries are known. As an example for a recommender system we mention the ''Netflix Prize,'' where the challenge is defined as improving the accuracy of predictions about how much a person is going to enjoy a future movie based on past preferences. The data can be regarded as an incomplete user Â movie matrix, whose entries are user ratings in an ordinal 1-5 scale, and the challenge is to fill in the missing entries (initially set to zero). Among the many and diverse methods that have been proposed we mention that some are based on (coupled) matrix or tensor factorizations, possibly by augmenting these data with further types of diversity; see, e.g., BIB019 , BIB016 , BIB005 , and references therein.