Survey on Probabilistic Models of Low-Rank Matrix Factorizations <s> Probabilistic Matrix Factorization <s> Many existing approaches to collaborative filtering can neither handle very large datasets nor easily deal with users who have very few ratings. In this paper we present the Probabilistic Matrix Factorization (PMF) model which scales linearly with the number of observations and, more importantly, performs well on the large, sparse, and very imbalanced Netflix dataset. We further extend the PMF model to include an adaptive prior on the model parameters and show how the model capacity can be controlled automatically. Finally, we introduce a constrained version of the PMF model that is based on the assumption that users who have rated similar sets of movies are likely to have similar preferences. The resulting model is able to generalize considerably better for users with very few ratings. When the predictions of multiple PMF models are linearly combined with the predictions of Restricted Boltzmann Machines models, we achieve an error rate of 0.8861, that is nearly 7% better than the score of Netflix's own system. <s> BIB001 </s> Survey on Probabilistic Models of Low-Rank Matrix Factorizations <s> Probabilistic Matrix Factorization <s> The variational Bayesian (VB) approximation is known to be a promising approach to Bayesian estimation, when the rigorous calculation of the Bayes posterior is intractable. The VB approximation has been successfully applied to matrix factorization (MF), offering automatic dimensionality selection for principal component analysis. Generally, finding the VB solution is a nonconvex problem, and most methods rely on a local search algorithm derived through a standard procedure for the VB approximation. In this paper, we show that a better option is available for fully-observed VBMF--the global solution can be analytically computed. More specifically, the global solution is a reweighted SVD of the observed matrix, and each weight can be obtained by solving a quartic equation with its coefficients being functions of the observed singular value. We further show that the global optimal solution of empirical VBMF (where hyperparameters are also learned from data) can also be analytically computed. We illustrate the usefulness of our results through experiments in multi-variate analysis. <s> BIB002
We still consider Gaussian noise corruptions, that is, E ij ∼ N (0, σ 2 ). Furthermore, the zero-mean spherical Gaussian priors are respectively imposed on each row of W and each column of Z: Probabilistic matrix factorization (PMF) BIB001 regards both σ 2 w and σ 2 z as two deterministic parameters. The point estimations of W, Z can be obtained by maximizing the posterior distribution with the following form: If the priors are respectively placed on σ 2 w and σ 2 z , we can obtain the generalized model of probabilistic matrix factorization BIB001 . In this case, the likelihood function is derived as By maximizing the above posterior distribution, we can obtain the point estimations of parameters {W, Z} and hyperparameters σ 2 w , σ 2 z . Furthermore, two derivatives of PMF are also presented, i.e., PMF with a adaptive prior and PMF with constraining user-specific feature vectors. In BIB002 , a fully-observed variational Bayesian matrix factorization, an extension of PMF, was discussed. Meanwhile, it is shown that this new probabilistic matrix factorization can weaken the decomposability assumption and obtain the exact global analytic solution for rectangular cases.