Collaborative filtering is widely used in modern recommender systems. Recent research shows that variational autoencoders (VAEs) yield state-of-the-art performance by integrating flexible representations from deep neural networks into latent variable models, mitigating limitations of traditional linear factor models. VAEs are typically trained by maximizing the likelihood (MLE) of users interacting with ground-truth items. While simple and often effective, MLE-based training does not directly maximize the recommendation-quality metrics one typically cares about, such as top-N ranking. In this paper we investigate new methods for training collaborative filtering models based on actor-critic reinforcement learning, to directly optimize the non-differentiable quality metrics of interest. Specifically, we train a critic network to approximate ranking-based metrics, and then update the actor network (represented here by a VAE) to directly optimize against the learned metrics. In contrast to traditional learning-to-rank methods that require to re-run the optimization procedure for new lists, our critic-based method amortizes the scoring process with a neural network, and can directly provide the (approximate) ranking scores for new lists. Empirically, we show that the proposed methods outperform several state-of-the-art baselines, including recently-proposed deep learning approaches, on three large-scale real-world datasets. The code to reproduce the experimental results and figure plots is on Github: https: github.com samlobel RaCT_CF
Learning to Rank (L2R). The idea of L2R has existed for two decades in the information-retrieval community. The goal is to directly optimize against ranking-based evaluation metrics @cite_32 @cite_43 . Previous work on L2R employs objective relaxations @cite_33 . Some techniques can be extended to recommendation settings @cite_46 @cite_6 @cite_44 @cite_9 @cite_13 . Many L2R methods in recommendation are essentially trained by optimizing a classification function, such as the popular pairwise L2R method BPR @cite_46 and WARP @cite_12 described Section 2.1. One limitation is that they are computationally expensive when the number of items is large. To accelerate these approaches, cheap approximations are made in each training step, which results in degraded performance. In contrast, the proposed RaCT is efficient and scalable. In fact, the traditional L2R methods can be integrated into our actor-critic framework, yielding improved performance as shown in our experiments.
@cite_33 @cite_9 @cite_32 @cite_6 @cite_44 @cite_43 @cite_46 @cite_13 @cite_12
2079648696 2554853703 2785013514 2159545104
Learning to rank has received great attentions in the field of text retrieval for several years. However, a few researchers introduce the topic into visual reranking due to the special nature of image presentation. In this paper, a novel unsupervised visual reranking is proposed, termed rank via the convolutional neural networks (RankCNN). This approach integrates deep learning with pseudo preference feedback. The optimal set of pseudo preference pairs is first detected from initial list by a modified graph-based method. Ranking is then reduced to pairwise classification in the architecture of CNN. In addition, Accelerated Mini-Batch Stochastic Dual Coordinate Ascent (ASDCA) is introduced to the framework to accelerate the training. The experiments indicate the competitive performance on the LETOR 4.0, the Paris and the Francelandmark dataset. Low-rank learning has attracted much attention recently due to its efficacy in a rich variety of real-world tasks, e.g., subspace segmentation and image categorization. Most low-rank methods are incapable of capturing low-dimensional subspace for supervised learning tasks, e.g., classification and regression. This paper aims to learn both the discriminant low-rank representation (LRR) and the robust projecting subspace in a supervised manner. To achieve this goal, we cast the problem into a constrained rank minimization framework by adopting the least squares regularization. Naturally, the data label structure tends to resemble that of the corresponding low-dimensional representation, which is derived from the robust subspace projection of clean data by low-rank learning. Moreover, the low-dimensional representation of original data can be paired with some informative structure by imposing an appropriate constraint, e.g., Laplacian regularizer. Therefore, we propose a novel constrained LRR method. The objective function is formulated as a constrained nuclear norm minimization problem, which can be solved by the inexact augmented Lagrange multiplier algorithm. Extensive experiments on image classification, human pose estimation, and robust face recovery have confirmed the superiority of our method. The problem of ranking the documents according to their relevance to a given query is a hot topic in information retrieval. Most learning-to-rank methods are supervised and use human editor judgements for learning. In this paper, we introduce novel pairwise method called YetiRank that modifies Friedman's gradient boosting method in part of gradient computation for optimization and takes uncertainty in human judgements into account. Proposed enhancements allowed YetiRank to outperform many state-of-the-art learning to rank methods in offline experiments as well as take the first place in the second track of the Yahoo! learning-to-rank contest. Even more remarkably, the first result in the learning to rank competition that consisted of a transfer learning task was achieved without ever relying on the bigger data from the "transfer-from" domain. Ranking a set of retrieved documents according to their relevance to a query is a popular problem in information retrieval. Methods that learn ranking functions are difficult to optimize, as ranking performance is typically judged by metrics that are not smooth. In this paper we propose a new listwise approach to learning to rank. Our method creates a conditional probability distribution over rankings assigned to documents for a given query, which permits gradient ascent optimization of the expected value of some performance measure. The rank probabilities take the form of a Boltzmann distribution, based on an energy function that depends on a scoring function composed of individual and pairwise potentials. Including pairwise potentials is a novel contribution, allowing the model to encode regularities in the relative scores of documents; existing models assign scores at test time based only on individual documents, with no pairwise constraints between documents. Experimental results on the LETOR3.0 data set show that our method out-performs existing learning approaches to ranking.