Factor analysis provides linear factors that describe relationships between individual variables of a data set. We extend this classical formulation into linear factors that describe relationships between groups of variables, where each group represents either a set of related variables or a data set. The model also naturally extends canonical correlation analysis to more than two sets, in a way that is more flexible than previous extensions. Our solution is formulated as variational inference of a latent variable model with structural sparsity, and it consists of two hierarchical levels: The higher level models the relationships between the groups, whereas the lower models the observed variables given the higher level. We show that the resulting solution solves the group factor analysis problem accurately, outperforming alternative factor analysis based solutions as well as more straightforward implementations of group factor analysis. The method is demonstrated on two life science data sets, one on brain activation and the other on systems biology, illustrating its applicability to the analysis of different types of high-dimensional data sources.
For two groups, @math , the model is equivalent to Bayesian CCA and inter-battery factor analysis @cite_4 ; some factors model the correlations whereas some describe the residual noise within either group.
@cite_4
2052637985 2087962626 2144903813 2950903552
McDonald's (1970) generalization of Tucker's (1958) inter-battery factor analysis model to multiple batteries is considered. The identification of parameters is examined and an iterative algorithm for obtaining maximum-likelihood estimates is provided. Consideration of the relationship between inter-battery factor analysis and canonical correlation analysis in the case of two batteries suggests a generalization of canonical coefficients to the situation where there are several batteries. Examples of the application of the procedure are given. The inter-battery method of factor analysis was devised to provide information relevant to the stability of factors over different selections of tests. Two batteries of tests, postulated to depend on the same common factors, but not parallel tests, are given to one sample of individuals. Factors are determined from the correlation of the tests in one battery with the tests in the other battery. These factors are only those that are common to the two batteries. No communality estimates are required. A statistical test is provided for judging the minimum number of factors involved. Rotation of axes is carried out independently for the two batteries. A final step provides the correlation between factors determined by scores on the tests in the two batteries. The correlations between corresponding factors are taken as factor reliability coefficients. Canonical correlation analysis (CCA) is a classical method for seeking correlations between two multivariate data sets. During the last ten years, it has received more and more attention in the machine learning community in the form of novel computational formulations and a plethora of applications. We review recent developments in Bayesian models and inference methods for CCA which are attractive for their potential in hierarchical extensions and for coping with the combination of large dimensionalities and small sample sizes. The existing methods have not been particularly successful in fulfilling the promise yet; we introduce a novel efficient solution that imposes group-wise sparsity to estimate the posterior of an extended model which not only extracts the statistical dependencies (correlations) between data sets but also decomposes the data into shared and data set-specific components. In statistics literature the model is known as inter-battery factor analysis (IBFA), for which we now provide a Bayesian treatment. We introduce a factor analysis model that summarizes the dependencies between observed variable groups, instead of dependencies between individual variables as standard factor analysis does. A group may correspond to one view of the same set of objects, one of many data sets tied by co-occurrence, or a set of alternative variables collected from statistics tables to measure one property of interest. We show that by assuming group-wise sparse factors, active in a subset of the sets, the variation can be decomposed into factors explaining relationships between the sets and factors explaining away set-specific variation. We formulate the assumptions in a Bayesian model which provides the factors, and apply the model to two data analysis tasks, in neuroimaging and chemical systems biology.