Towards Bayesian Deep Learning: A Survey <s> Symmetric Collaborative Deep Learning <s> Collaborative filtering (CF) is a successful approach commonly used by many recommender systems. Conventional CF-based methods use the ratings given to items by users as the sole source of information for learning to make recommendation. However, the ratings are often very sparse in many applications, causing CF-based methods to degrade significantly in their recommendation performance. To address this sparsity problem, auxiliary information such as item content information may be utilized. Collaborative topic regression (CTR) is an appealing recent method taking this approach which tightly couples the two components that learn from two different sources of information. Nevertheless, the latent representation learned by CTR may not be very effective when the auxiliary information is very sparse. To address this problem, we generalize recently advances in deep learning from i.i.d. input to non-i.i.d. (CF-based) input and propose in this paper a hierarchical Bayesian model called collaborative deep learning (CDL), which jointly performs deep representation learning for the content information and collaborative filtering for the ratings (feedback) matrix. Extensive experiments on three real-world datasets from different domains show that CDL can significantly advance the state of the art. <s> BIB001 </s> Towards Bayesian Deep Learning: A Survey <s> Symmetric Collaborative Deep Learning <s> Collaborative filtering (CF) has been widely employed within recommender systems to solve many real-world problems. Learning effective latent factors plays the most important role in collaborative filtering. Traditional CF methods based upon matrix factorization techniques learn the latent factors from the user-item ratings and suffer from the cold start problem as well as the sparsity problem. Some improved CF methods enrich the priors on the latent factors by incorporating side information as regularization. However, the learned latent factors may not be very effective due to the sparse nature of the ratings and the side information. To tackle this problem, we learn effective latent representations via deep learning. Deep learning models have emerged as very appealing in learning effective representations in many applications. In particular, we propose a general deep architecture for CF by integrating matrix factorization with deep feature learning. We provide a natural instantiations of our architecture by combining probabilistic matrix factorization with marginalized denoising stacked auto-encoders. The combined framework leads to a parsimonious fit over the latent features as indicated by its improved performance in comparison to prior state-of-art models over four large datasets for the tasks of movie/book recommendation and response prediction. <s> BIB002 </s> Towards Bayesian Deep Learning: A Survey <s> Symmetric Collaborative Deep Learning <s> Collaborative Filtering with Implicit Feedbacks e.g., browsing or clicking records, named as CF-IF, is demonstrated to be an effective way in recommender systems. Existing works of CF-IF can be mainly classified into two categories, i.e., point-wise regression based and pair-wise ranking based, where the latter one relaxes assumption and usually obtains better performance in empirical studies. In real applications, implicit feedback is often very sparse, causing CF-IF based methods to degrade significantly in recommendation performance. In this case, side information e.g., item content is usually introduced and utilized to address the data sparsity problem. Nevertheless, the latent feature representation learned from side information by topic model may not be very effective when the data is too sparse. To address this problem, we propose collaborative deep ranking CDR, a hybrid pair-wise approach with implicit feedback, which leverages deep feature representation of item content into Bayesian framework of pair-wise ranking model in this paper. The experimental analysis on a real-world dataset shows CDR outperforms three state-of-art methods in terms of recall metric under different sparsity level. <s> BIB003
Models like BIB001 , BIB003 focus the deep learning component on modeling the item content. Besides the content information from the items, attributes of users sometimes contain much more important information. It is therefore desirable to extend CDL to model user attributes as well BIB002 . We call this variant symmetric CDL. For example, using an extra mSDAE on the user attributes gives the following joint log-likelihood: where Y 0,j * (a k-by-D matrix for user attributes) is the collection of k different corrupted versions of Y 0,j * and Y c,i * (also a k-by-D matrix) is the k-time repeated version of Y c,i * (the clean user attributes). P 2 is the transformation matrix for user latent factors and D is the number of user attributes. Similar to the marginalized CDL, the solution for W 2 given other parameters is: where , the hinge variables Ω h = {V, U}, and the task variables Ω t = {P 1 , P 2 , R}.