Survey on Probabilistic Models of Low-Rank Matrix Factorizations <s> Bayesian Model for L1-Norm Low-Rank Matrix Factorizations <s> Low-rank matrix approximation methods provide one of the simplest and most effective approaches to collaborative filtering. Such models are usually fitted to data by finding a MAP estimate of the model parameters, a procedure that can be performed efficiently even on very large datasets. However, unless the regularization parameters are tuned carefully, this approach is prone to overfitting because it finds a single point estimate of the parameters. In this paper we present a fully Bayesian treatment of the Probabilistic Matrix Factorization (PMF) model in which model capacity is controlled automatically by integrating over all model parameters and hyperparameters. We show that Bayesian PMF models can be efficiently trained using Markov chain Monte Carlo methods by applying them to the Netflix dataset, which consists of over 100 million movie ratings. The resulting models achieve significantly higher prediction accuracy than PMF models trained using MAP estimation. <s> BIB001 </s> Survey on Probabilistic Models of Low-Rank Matrix Factorizations <s> Bayesian Model for L1-Norm Low-Rank Matrix Factorizations <s> The L1 -norm low-rank matrix factorization (LRMF) has been attracting much attention due to its wide applications to computer vision and pattern recognition. In this paper, we construct a new hierarchical Bayesian generative model for the L1 -norm LRMF problem and design a mean-field variational method to automatically infer all the parameters involved in the model by closed-form equations. The variational Bayesian inference in the proposed method can be understood as solving a weighted LRMF problem with different weights on matrix elements based on their significance and with L2 -regularization penalties on parameters. Throughout the inference process of our method, the weights imposed on the matrix elements can be adaptively fitted so that the adverse influence of noises and outliers embedded in data can be largely suppressed, and the parameters can be appropriately regularized so that the generalization capability of the problem can be statistically guaranteed. The robustness and the efficiency of the proposed method are substantiated by a series of synthetic and real data experiments, as compared with the state-of-the-art L1 -norm LRMF methods. Especially, attributed to the intrinsic generalization capability of the Bayesian methodology, our method can always predict better on the unobserved ground truth data than existing methods. <s> BIB002
For low-rank matrix factorizations, L1-norm minimization is more robust than L2-norm minimization in the presence of outliers or non-Gaussian noises. Based on this, we assume that the noise E ij follows the Laplace distribution: E ij ∼ Lap(0, √ λ/2). Since the Laplace noise is inconvenient for Bayesian inference, a hierarchical Bayesian model was formulated in BIB002 . Concretely speaking, a two-level hierarchical prior is imposed on the Laplace prior: The generative models of W and Z are constructed as follows: In addition, Gamma priors are placed on the precision parameters of the above Gaussian distributions: The trial posterior distribution for {W, Z, τ W , τ Z , T} is specified as: And the joint probability distribution is expressed as VB inference was adopted to approximate the full posterior distribution BIB002 . Furthermore, varying precision parameters are also considered for different rows of W or different columns of Z. All parameters are automatically tuned to adapt to the data, and the proposed method is applied in computer vision problems to validate its efficiency and robustness. In Table 3 , we list all probabilistic models of matrix factorizations discussed in this section, and compare their probability distributions, priors and solving strategy.  -VB Bayesian PMF BIB001 E ij ∼ N (0, β −1 ),