Survey on Probabilistic Models of Low-Rank Matrix Factorizations <s> Matrix Factorizations <s> We study the common problem of approximating a target matrix with a matrix of lower rank. We provide a simple and efficient (EM) algorithm for solving weighted low-rank approximation problems, which, unlike their unweighted version, do not admit a closed-form solution in general. We analyze, in addition, the nature of locally optimal solutions that arise in this context, demonstrate the utility of accommodating the weights in reconstructing the underlying low-rank representation, and extend the formulation to non-Gaussian noise models such as logistic models. Finally, we apply the methods developed to a collaborative filtering task. <s> BIB001 </s> Survey on Probabilistic Models of Low-Rank Matrix Factorizations <s> Matrix Factorizations <s> A method of principal component analysis (PCA) based on a new L1-norm optimization technique is proposed. Unlike conventional PCA which is based on L2-norm, the proposed method is robust to outliers because it utilizes L1-norm which is less sensitive to outliers. It is invariant to rotations as well. The proposed L1-norm optimization technique is intuitive, simple, and easy to implement. It is also proven to find a locally maximal solution. The proposed method is applied to several datasets and the performances are compared with those of other conventional methods. <s> BIB002
For given data matrix X, its low-rank factorization model is written as where W ∈ R d×r , Z ∈ R r×N , E is the noise matrix and r < d. We assume that E ij ∼ N (0, σ 2 ) are independent and identically distributed. We can get the optimal W and Z according to the maximum likelihood estimation. More specifically, we need to solve the following minimization problem: The closed-form solution to problem (60) can be obtained by the Singular Value Decomposition (SVD). To enhance the robustness to outliers or large sparse noise, we now assume that E ij ∼ Lap(0, σ). For the moment, we solve the following optimization problem: where · 1 is the L1-norm of a matrix (i.e., the sum of the absolute value of all elements). This problem is also called L1-norm PCA and the corresponding optimization methods were proposed in  BIB002 . Srebro and Jaakkola considered the weighted low-rank approximations problems and provided an EM algorithm BIB001 .