Learning to rank is a machine learning technique broadly used in many areas such as document retrieval, collaborative filtering or question answering. We present experimental results which suggest that the performance of the current state-of-the-art learning to rank algorithm LambdaMART, when used for document retrieval for search engines, can be improved if standard regression trees are replaced by oblivious trees. This paper provides a comparison of both variants and our results demonstrate that the use of oblivious trees can improve the performance by more than @math . Additional experimental analysis of the influence of a number of features and of a size of the training set is also provided and confirms the desirability of properties of oblivious decision trees.
For instance, LTR problem can be reformulated as a regression task of the relevance label prediction which approaches the problem in a pointwise manner. Each query-document pair is then considered a single data sample and the relations between documents belonging to a particular query are not taken into account. (MSE) is then usually used as the objective function @cite_0 . Random Forest @cite_0 or Multiple Additive Regression Trees (MART) @cite_20 can be utilised to solve the task in the aforementioned manner. Similarly, PRank algorithm proposed in @cite_30 uses a neural network to predict the relevance label. However, the authors of @cite_30 extend the task to ordinal regression, where the relevance score is converted to the relevance class (resp. label) in the end. Besides, there are also pointwise algorithms that treat the problem as a classification problem. For instance, McRank algorithm @cite_7 uses gradient boosting tree algorithm and reformulates the task as a multiple ordinal classification.
@cite_0 @cite_7 @cite_30 @cite_20
2120391124 2029731618 2909867291 1834987204
We cast the ranking problem as (1) multiple classification ("Mc") (2) multiple ordinal classification, which lead to computationally tractable learning algorithms for relevance ranking in Web search. We consider the DCG criterion (discounted cumulative gain), a standard quality measure in information retrieval. Our approach is motivated by the fact that perfect classifications result in perfect DCG scores and the DCG errors are bounded by classification errors. We propose using the Expected Relevance to convert class probabilities into ranking scores. The class probabilities are learned using a gradient boosting tree algorithm. Evaluations on large-scale datasets show that our approach can improve LambdaRank [5] and the regressions-based ranker [6], in terms of the (normalized) DCG scores. An efficient implementation of the boosting tree algorithm is also presented. We address the task of learning a semantic segmentation from weakly supervised data. Our aim is to devise a system that predicts an object label for each pixel by making use of only image level labels during training – the information whether a certain object is present or not in the image. Such coarse tagging of images is faster and easier to obtain as opposed to the tedious task of pixelwise labeling required in state of the art systems. We cast this task naturally as a multiple instance learning (MIL) problem. We use Semantic Texton Forest (STF) as the basic framework and extend it for the MIL setting. We make use of multitask learning (MTL) to regularize our solution. Here, an external task of geometric context estimation is used to improve on the task of semantic segmentation. We report experimental results on the MSRC21 and the very challenging VOC2007 datasets. On MSRC21 dataset we are able, by using 276 weakly labeled images, to achieve the performance of a supervised STF trained on pixelwise labeled training set of 56 images, which is a significant reduction in supervision needed. Most Learning To Rank (LTR) algorithms like Ranking SVM, RankNet, LambdaRank and LambdaMART use only relevance label judgments as ground truth for training. But in common scenarios like ranking of information cards (google now, other personal assistants), mobile notifications, netflix recommendations, etc. there is additional information which can be captured from user behavior and how user interacts with the retrieved items. Within the relevance labels, there might be different sets whose information (i.e. cluster information) can be derived implicitly from user interaction (positive, negative, neutral, etc.) or from explicit-user feedback ('Do not show again', 'I like this suggestion', etc). This additional information provides significant knowledge for training any ranking algorithm using two-dimensional output variable. This paper proposes a novel method to use the relevance label along with cluster information to better train the ranking models. Results for user-trial Notification Ranking dataset and standard datasets like LETOR 4.0, MSLR-WEB10K and YahooLTR further support this claim. Recommending phrases from web pages for advertisers to bid on against search engine queries is an important research problem with direct commercial impact. Most approaches have found it infeasible to determine the relevance of all possible queries to a given ad landing page and have focussed on making recommendations from a small set of phrases extracted (and expanded) from the page using NLP and ranking based techniques. In this paper, we eschew this paradigm, and demonstrate that it is possible to efficiently predict the relevant subset of queries from a large set of monetizable ones by posing the problem as a multi-label learning task with each query being represented by a separate label. We develop Multi-label Random Forests to tackle problems with millions of labels. Our proposed classifier has prediction costs that are logarithmic in the number of labels and can make predictions in a few milliseconds using 10 Gb of RAM. We demonstrate that it is possible to generate training data for our classifier automatically from click logs without any human annotation or intervention. We train our classifier on tens of millions of labels, features and training points in less than two days on a thousand node cluster. We develop a sparse semi-supervised multi-label learning formulation to deal with training set biases and noisy labels harvested automatically from the click logs. This formulation is used to infer a belief in the state of each label for each training ad and the random forest classifier is extended to train on these beliefs rather than the given labels. Experiments reveal significant gains over ranking and NLP based techniques on a large test set of 5 million ads using multiple metrics.