Production Ranking Systems: A Review <s> Discussion <s> Recent work in unsupervised feature learning and deep learning has shown that being able to train large models can dramatically improve performance. In this paper, we consider the problem of training a deep network with billions of parameters using tens of thousands of CPU cores. We have developed a software framework called DistBelief that can utilize computing clusters with thousands of machines to train large models. Within this framework, we have developed two algorithms for large-scale distributed training: (i) Downpour SGD, an asynchronous stochastic gradient descent procedure supporting a large number of model replicas, and (ii) Sandblaster, a framework that supports a variety of distributed batch optimization procedures, including a distributed implementation of L-BFGS. Downpour SGD and Sandblaster L-BFGS both increase the scale and speed of deep network training. We have successfully used our system to train a deep network 30x larger than previously reported in the literature, and achieves state-of-the-art performance on ImageNet, a visual object recognition task with 16 million images and 21k categories. We show that these same techniques dramatically accelerate the training of a more modestly- sized deep network for a commercial speech recognition service. Although we focus on and report performance of these methods as applied to training large neural networks, the underlying algorithms are applicable to any gradient-based machine learning algorithm. <s> BIB001 </s> Production Ranking Systems: A Review <s> Discussion <s> A recommender system (RS) aims to provide personalized recommendations to users for specific items (e.g., music, books). Popular techniques involve content-based (CB) models and collaborative filtering (CF) approaches. In this paper, we deal with a very important problem in RSs: The cold start problem. This problem is related to recommendations for novel users or new items. In case of new users, the system does not have information about their preferences in order to make recommendations. We propose a model where widely known classification algorithms in combination with similarity techniques and prediction mechanisms provide the necessary means for retrieving recommendations. The proposed approach incorporates classification methods in a pure CF system while the use of demographic data help for the identification of other users with similar behavior. Our experiments show the performance of the proposed system through a large number of experiments. We adopt the widely known dataset provided by the GroupLens research group. We reveal the advantages of the proposed solution by providing satisfactory numerical results in different experimental scenarios. <s> BIB002 </s> Production Ranking Systems: A Review <s> Discussion <s> This issue's "Cloud Tidbit" focuses on container technology and how it's emerging as an important part of the cloud computing infrastructure. It looks at Docker, an open source project that automates the faster deployment of Linux applications, and Kubernetes, an open source cluster manager for Docker containers. <s> BIB003 </s> Production Ranking Systems: A Review <s> Discussion <s> Machine learning offers a fantastically powerful toolkit for building useful complex prediction systems quickly. This paper argues it is dangerous to think of these quick wins as coming for free. Using the software engineering framework of technical debt, we find it is common to incur massive ongoing maintenance costs in real-world ML systems. We explore several ML-specific risk factors to account for in system design. These include boundary erosion, entanglement, hidden feedback loops, undeclared consumers, data dependencies, configuration issues, changes in the external world, and a variety of system-level anti-patterns. <s> BIB004 </s> Production Ranking Systems: A Review <s> Discussion <s> In most sponsored search platforms, advertisers bid on some keywords for their advertisements (ads). Given a search request, ad retrieval module rewrites the query into bidding keywords, and uses these keywords as keys to select Top N ads through inverted indexes. In this way, an ad will not be retrieved even if queries are related when the advertiser does not bid on corresponding keywords. Moreover, most ad retrieval approaches regard rewriting and ad-selecting as two separated tasks, and focus on boosting relevance between search queries and ads. Recently, in e-commerce sponsored search more and more personalized information has been introduced, such as user profiles, long-time and real-time clicks. Personalized information makes ad retrieval able to employ more elements (e.g. real-time clicks) as search signals and retrieval keys, however it makes ad retrieval more difficult to measure ads retrieved through different signals. ::: ::: To address these problems, we propose a novel ad retrieval framework beyond keywords and relevance in e-commerce sponsored search. Firstly, we employ historical ad click data to initialize a hierarchical network representing signals, keys and ads, in which personalized information is introduced. Then we train a model on top of the hierarchical network by learning the weights of edges. Finally we select the best edges according to the model, boosting RPM/CTR. Experimental results on our e-commerce platform demonstrate that our ad retrieval framework achieves good performance. <s> BIB005 </s> Production Ranking Systems: A Review <s> Discussion <s> Tasks such as search and recommendation have become increasingly important for E-commerce to deal with the information overload problem. To meet the diverse needs of different users, personalization plays an important role. In many large portals such as Taobao and Amazon, there are a bunch of different types of search and recommendation tasks operating simultaneously for personalization. However, most of current techniques address each task separately. This is suboptimal as no information about users shared across different tasks. In this work, we propose to learn universal user representations across multiple tasks for more effective personalization. In particular, user behavior sequences (e.g., click, bookmark or purchase of products) are modeled by LSTM and attention mechanism by integrating all the corresponding content, behavior and temporal information. User representations are shared and learned in an end-to-end setting across multiple tasks. Benefiting from better information utilization of multiple tasks, the user representations are more effective to reflect their interests and are more general to be transferred to new tasks. We refer this work as Deep User Perception Network (DUPN) and conduct an extensive set of offline and online experiments. Across all tested five different tasks, our DUPN consistently achieves better results by giving more effective user representations. Moreover, we deploy DUPN in large scale operational tasks in Taobao. Detailed implementations, e.g., incremental model updating, are also provided to address the practical issues for the real world applications. <s> BIB006 </s> Production Ranking Systems: A Review <s> Discussion <s> The application of deep learning techniques resulted in remarkable improvement of machine learning models. In this paper provides detailed characterizations of deep learning models used in many Facebook social network services. We present computational characteristics of our models, describe high performance optimizations targeting existing systems, point out their limitations and make suggestions for the future general-purpose/accelerated inference hardware. Also, we highlight the need for better co-design of algorithms, numerics and computing platforms to address the challenges of workloads often run in data centers. <s> BIB007 </s> Production Ranking Systems: A Review <s> Discussion <s> E-commerce sponsored search contributes an important part of revenue for the e-commerce company. In consideration of effectiveness and efficiency, a large-scale sponsored search system commonly adopts a multi-stage architecture. We name these stages as ad retrieval, ad pre-ranking and ad ranking. Ad retrieval and ad pre-ranking are collectively referred to as ad matching in this paper. We propose an end-to-end neural matching framework (EENMF) to model two tasks---vector-based ad retrieval and neural networks based ad pre-ranking. Under the deep matching framework, vector-based ad retrieval harnesses user recent behavior sequence to retrieve relevant ad candidates without the constraint of keyword bidding. Simultaneously, the deep model is employed to perform the global pre-ranking of ad candidates from multiple retrieval paths effectively and efficiently. Besides, the proposed model tries to optimize the pointwise cross-entropy loss which is consistent with the objective of predict models in the ranking stage. We conduct extensive evaluation to validate the performance of the proposed framework. In the real traffic of a large-scale e-commerce sponsored search, the proposed approach significantly outperforms the baseline. <s> BIB008 </s> Production Ranking Systems: A Review <s> Discussion <s> In this paper we present Horizon, Facebook's open source applied reinforcement learning (RL) platform. Horizon is an end-to-end platform designed to solve industry applied RL problems where datasets are large (millions to billions of observations), the feedback loop is slow (vs. a simulator), and experiments must be done with care because they don't run in a simulator. Unlike other RL platforms, which are often designed for fast prototyping and experimentation, Horizon is designed with production use cases as top of mind. The platform contains workflows to train popular deep RL algorithms and includes data preprocessing, feature transformation, distributed training, counterfactual policy evaluation, optimized serving, and a model-based data understanding tool. We also showcase and describe real examples where reinforcement learning models trained with Horizon significantly outperformed and replaced supervised learning systems at Facebook. <s> BIB009
Although user interaction data is rich, incorporation of side information is necessary in any ranking system. is data is employed to combat popular item bias, and to address the cold-start problem BIB002 . is side information allows correlations learned from user interactions to be extended to items with few impressions based on relationships which exist in item descriptions, item category information, etc. Although proper processing of user interaction data is required for training high performance ranking models, we nd e ective ways to leverage side information a ords the highest coverage and most diversity. Models which rely too heavily on user interaction data over t to head queries and popular items. Processing this data, which is o en multiple gigabytes, must be done over a distributed computing platform, as it would be infeasible to t the dataset in memory on a single machine. ese processes are o en done in batch, with user interaction data being processed into new training data at regular intervals. As the need for real time personalization and sponsored advertising increases, stream processing methodologies are emerging which allow data to be processed and featurized in near real time. is trend allows ranking systems to become increasingly responsive to user interactions and increase user engagement by modifying representations of users and query intent as users progress through a platform. As state of the art work in deep learning continues to produce more e ective representation learning techniques, we nd the best approach is to employ embeddings which can surface relationships within the underlying data. Validation of this layer is di cult, and not o en discussed in the literature. Instead this layer is o en validated in conjunction with the subsequent layers. is can cause improvements within this layer to be hidden by poor tuning in these subsequent layers. As such e ective use of representation learning requires development of clear validation of this layer in isolation to the ranking system. Concerns with this layer of ranking and form of separation is propagation of error from new embedding experiments leaking into later Figure 2 : is gure highlights how ranking systems enable real time interaction with users. Users interact with the platform and the system processes their interactions to update their query results. e rst row illustrates a global result. e second row shows results for the same query when the user signs in, and provides context based on their past history. e third row presents results for the user for the same query once a product has been added to the cart. is updates the context to the ranking model by providing an additional item, thus narrowing down to the user's information need. At each stage the system is dynamically ranking candidates a er interpreting new information about the user to better infer their intent and each items relavance to the intent. layers due to proper lack of tuning of this layer. We nd that hashing based methodologies are likely the most relevant to ranking, as much work is spent on representation learning. Graph based approaches are weak in that they require massive datasets, and only the largest organizations can a ord to employ them. Graph based approaches have the further complication of lacking an obvious approach to addressing the cold start problem, as graph based representations cannot directly represent items which do not already exist within the graph. e most common approaches for real time models are shallow binary classi cations, especially those relying on pairwise approaches, which can rank an entire set of candidates against one another while still posing the problem as a binary classi cation BIB006 . ese approaches are best suited for machine learning, as binary classi ers are a well studied set of models. We view both pointwise and pairwise approaches as being the most optimal solutions. e challenge with these models is addressed by careful feature engineering in data processing and representation learning layers. Listwise approaches seem ill-posed in comparison, as true ranking data is  impossible to obtain with implicit feedback and the models su er from complexities of trying to learn relevancy of multiple classes at once. Current design paradigms rely heavily on the coordination of tasks across both distributed and one-box architectures. Data processing and candidate selection are performed distributed, while representation learning and online models are handled via one-box architectures. Tasks are coordinated via schedulers  and one box architectures are containerized and deployed in parallel to support requests from many users BIB003 . We nd that recent trends aim to allow service of complex models directly by making complex models more e cient BIB001 BIB007 BIB008 BIB005 . is methodology has the potential to reduce the layers and separation of concerns within ranking systems. is would be a powerful improvement as it would reduce the complexity of the overall model, allowing for a uni ed approach. Improvements to streaming data processing technologies could further support these developments as complex computations can be run over distributed data caches in near real time to transform data into input features for deep learning models which are served live. Although promising, this allows for deep learning models to be deployed in real time, but the work lacks a generalized approach, lacks open source support and thus lacks wide spread adoption. Focus on ranking metrics for the overall system is necessary, but we propose that each layer requires its own independent metrics as well to avoid obfuscating errors and biases. Data processing layers should document assumptions with metrics dashboards, and gauge distributions of data as well as any underlying shi s within these distributions over time. Validation on learned representations is not documented in most production ranking system architectures, instead they are only measured in their improvement of applications for modeling and candidate selection. Requiring each component to have independent functionality tests as well as tests of the entire system can more clearly surface errors BIB009 BIB004 .