We consider the problem of recovering a common latent source with independent components from multiple views. This applies to settings in which a variable is measured with multiple experimental modalities, and where the goal is to synthesize the disparate measurements into a single unified representation. We consider the case that the observed views are a nonlinear mixing of component-wise corruptions of the sources. When the views are considered separately, this reduces to nonlinear Independent Component Analysis (ICA) for which it is provably impossible to undo the mixing. We present novel identifiability proofs that this is possible when the multiple views are considered jointly, showing that the mixing can theoretically be undone using function approximators such as deep neural networks. In contrast to known identifiability results for nonlinear ICA, we prove that independent latent sources with arbitrary mixing can be recovered as long as multiple, sufficiently different noisy views are available.
Given two (or more) random variables, the goal of Canonical Correlation Analysis (CCA) @cite_28 is to find a corresponding pair of linear subspaces that have high cross-correlation, so that each component within one of the subspaces is correlated with a single component from the other subspace @cite_14 . In dealing with correlation instead of independence, CCA is more closely related to Principal Component Analysis (PCA) than to ICA.
@cite_28 @cite_14
2025341678 1663973292
Concepts of correlation and regression may be applied not only to ordinary one-dimensional variates but also to variates of two or more dimensions. Marksmen side by side firing simultaneous shots at targets, so that the deviations are in part due to independent individual errors and in part to common causes such as wind, provide a familiar introduction to the theory of correlation; but only the correlation of the horizontal components is ordinarily discussed, whereas the complex consisting of horizontal and vertical deviations may be even more interesting. The wind at two places may be compared, using both components of the velocity in each place. A fluctuating vector is thus matched at each moment with another fluctuating vector. The study of individual differences in mental and physical traits calls for a detailed study of the relations between sets of correlated variates. For example the scores on a number of mental tests may be compared with physical measurements on the same persons. The questions then arise of determining the number and nature of the independent relations of mind and body shown by these data to exist, and of extracting from the multiplicity of correlations in the system suitable characterizations of these independent relations. As another example, the inheritance of intelligence in rats might be studied by applying not one but s different mental tests to N mothers and to a daughter of each Cristopher M. BishopInformation Science and StatisticsSpringer 2006, 738 pagesAs the author writes in the preface of the book, pattern recognition has its origin inengineering, whereas machine learning grew out of computer science. However, theseactivities can be viewed as two facets of the same ﬁeld, and they have undergonesubstantial development over the past years.Bayesian methods are widely used, while graphical models have emerged as a generalframework for describing and applying probabilistic models. Similarly new modelsbased on kernels have had signiﬁcant impact on both algorithms and applications.This textbook reﬂects these recent developments while providing a comprehensiveintroduction to the ﬁelds of pattern recognition and machine learning. It is aimedat advanced undergraduate or ﬁrst year PhD students, as well as researchers andpractitioners. It can be consider as an introductory course to the subject.The ﬁrst four chapters are devoted to the concepts of Probability and Statistics that areneededforreadingtherestofthebook,sowecanimaginethatthespeedishighinorderto get from zero to inﬁnity. I believe that it is better to study the book after a previouscourse on Probability and Statistics. On the other hand, a basic knowledge of linearalgebra and multivariate calculus is assumed.The other chapters give to a classic probabilist or statistician a point of view on someapplications that are very interesting but far from his usual world. In all the text themathematical aspects are at the second level in relation withthe ideas and intuitionsthatthe author wants to communicate.The book is supported by a great deal of additional material, including lecture slides aswell as the complete set of ﬁgures used in it, and the reader is encouraged to visit thebook web site for the latest information. So it can be very useful for a course or a talkabout the subject.