Factor analysis provides linear factors that describe relationships between individual variables of a data set. We extend this classical formulation into linear factors that describe relationships between groups of variables, where each group represents either a set of related variables or a data set. The model also naturally extends canonical correlation analysis to more than two sets, in a way that is more flexible than previous extensions. Our solution is formulated as variational inference of a latent variable model with structural sparsity, and it consists of two hierarchical levels: The higher level models the relationships between the groups, whereas the lower models the observed variables given the higher level. We show that the resulting solution solves the group factor analysis problem accurately, outperforming alternative factor analysis based solutions as well as more straightforward implementations of group factor analysis. The method is demonstrated on two life science data sets, one on brain activation and the other on systems biology, illustrating its applicability to the analysis of different types of high-dimensional data sources.
Most multi-set extensions of CCA, however, are not equivalent to our model. For example, @cite_12 and @cite_18 extend CCA for @math , but instead of GFA they solve the more limited problem of multiple-battery factor analysis @cite_29 @cite_2 . The MBFA models provide one set of factors that describe the relationships between groups, and then model the variation specific to each group either with a free covariance matrix or a separate set of factors for that group. Besides the multi-set extensions of CCA, also the probabilistic interpretation of sparse matrix factorization @cite_11 , and the JIVE model for integrated analysis of multiple data types @cite_10 @cite_20 belong to the family of MBFA models. These models differ in their priors, parameterization and inference, but are all conceptually equivalent.
@cite_18 @cite_29 @cite_2 @cite_20 @cite_10 @cite_12 @cite_11
2144903813 2082246284 5384379 1968244248
Canonical correlation analysis (CCA) is a classical method for seeking correlations between two multivariate data sets. During the last ten years, it has received more and more attention in the machine learning community in the form of novel computational formulations and a plethora of applications. We review recent developments in Bayesian models and inference methods for CCA which are attractive for their potential in hierarchical extensions and for coping with the combination of large dimensionalities and small sample sizes. The existing methods have not been particularly successful in fulfilling the promise yet; we introduce a novel efficient solution that imposes group-wise sparsity to estimate the posterior of an extended model which not only extracts the statistical dependencies (correlations) between data sets but also decomposes the data into shared and data set-specific components. In statistics literature the model is known as inter-battery factor analysis (IBFA), for which we now provide a Bayesian treatment. SUMMARY Models for the analysis of longitudinal data must recogrlize the relationship between serial observations on the same unit. Multivariate models with general covariance structure are often difficult to apply to highly unbalanced data, whereas two-stage random-effects models can be used easily. In two-stage models, the probability distributions for the response vectors of different individuals belong to a single family, but some random-effects parameters vary across individuals, with a distribution specified at the second stage. A general family of models is discussed, which includes both growth models and repeated-measures models as special cases. A unified approach to fitting these models, based on a combination of empirical Bayes and maximum likelihood estimation of model parameters and using the EM algorithm, is discussed. Two examples are taken from a current epidemiological study of the health effects of air pollution. Many longitudinal studies are designed to investigate changes over time in a characteristic which is measured repeatedly for each study participant. In medical studies, the measurement might be blood pressure, cholesterol level, lung volume, or serum glucose. Multiple measurements are obtained from each individual, at different times and possibly under changing experimental conditions. Often, we cannot fully control the circumstances under which the measurements are taken, and there may be considerable variation among individuals in the number and timing of observations. The resulting unbalanced data sets are typically not amenable to analysis using a general multivariate model with unrestricted covariance structure. Statisticians have often analyzed data of this form using some variant of a two-stage model. In this formulation, the probability distribution for the multiple measurements has the same form for each individual, but the parameters of that distribution vary over individuals. The distribution of these parameters, or 'random effects', in the population constitutes the second stage of the model. In a study of changes in lung volume during childhood, for instance, it may be reasonable to assume that the relationship between lung volume and the cube of height is linear for each child, but with linear regression parameters that vary among children. If we assume that the usual linear regression model applies for each child, conditional on the child's individual parameters, and that the regression parameters have a bivariate normal distribution in the population, the marginal distribution of the serial measurements is multivariate normal with a special covariance structure. Recently, some recommendation methods try to improve the prediction results by integrating information from user's multiple types of behaviors. How to model the dependence and independence between different behaviors is critical for them. In this paper, we propose a novel recommendation model, the Group-Sparse Matrix Factorization (GSMF), which factorizes the rating matrices for multiple behaviors into the user and item latent factor space with group sparsity regularization. It can (1) select out the different subsets of latent factors for different behaviors, addressing that users' decisions on different behaviors are determined by different sets of factors; (2) model the dependence and independence between behaviors by learning the shared and private factors for multiple behaviors automatically; (3) allow the shared factors between different behaviors to be different, instead of all the behaviors sharing the same set of factors. Experiments on the real-world dataset demonstrate that our model can integrate users' multiple types of behaviors into recommendation better, compared with other state-of-the-arts. Building a common representation for several related data sets is an important problem in multi-view learning. CCA and its extensions have shown that they are effective in finding the shared variation among all data sets. However, these models generally fail to exploit the common structure of the data when the views are with private information. Recently, methods explicitly modeling the information into shared part and private parts have been proposed, but they presume to know the prior knowledge about the latent space, which is usually impossible to obtain. In this paper, we propose a probabilistic model, which could simultaneously learn the structure of the latent space whilst factorize the information correctly, therefore the prior knowledge of the latent space is unnecessary. Furthermore, as a probabilistic model, our method is able to deal with missing data problem in a natural way. We show that our approach attains the performance of state-of-art methods on the task of human pose estimation when the motion capture view is completely missing, and significantly improves the inference accuracy with only a few observed data.