The recently launched LinkedIn Salary product has been designed with the goal of providing compensation insights to the world's professionals and thereby helping them optimize their earning potential. We describe the overall design and architecture of the statistical modeling system underlying this product. We focus on the unique data mining challenges while designing and implementing the system, and describe the modeling components such as Bayesian hierarchical smoothing that help to compute and present robust compensation insights to users. We report on extensive evaluation with nearly one year of de-identified compensation data collected from over one million LinkedIn users, thereby demonstrating the efficacy of the statistical models. We also highlight the lessons learned through the deployment of our system at LinkedIn.
Statistical Smoothing : The idea of Bayesian hierarchical statistical smoothing originates from the smoothing of sparse events (e.g., CTR) in the context of computational advertising @cite_8 @cite_4 , where a natural hierarchy for the combination of ads category and publisher category is used for an (ad, publisher) pair with very little data to borrow strength from its ancestor nodes. However, that is an entirely different context and the models used are hence different.
@cite_4 @cite_8
2140160814 1969675113
Online advertisement is a significant element of the Web browsing experience. A good advertising can not only bring benefits to publisher but also improve user satisfaction and extends advertiser's product marketing. To satisfy the desire of all three parties, the click through rate (CTR) prediction of a user to a specified ad in a specific context is of great importance. This challenging problem plays a key role in online advertising system and has to deal with several hard issues. Firstly, the model must process very high dimensional features from frequently changing ad, user and context, most of which are category features having large cardinality and sparse nature extending the dimensionality by two orders of magnitude. Secondly, nonlinear features such as conjunction information must be integrated into the model for a better prediction accuracy. Finally, the model must be able to parallelized efficiently to train from very large scale data sets. To address these problems, we proposed a novel model called Coupled Logistic Regression (CLR), for accurate and efficient CTR prediction. CLR can exploit all features from ad, user, context and nonlinear features among them by seamlessly integrate the conjunction information by employing factorization machine to achieve precise prediction result. And the high-dimensional problem is avoided by decomposing the decision function into two sub ones. Scalability of CLR is ensured through a newly invited MapReduce parallelization strategy, which can reduce communication and waiting time between nodes. Experimental results on real-world data set show that our CLR model can guarantee both accuracy and efficiency on large scale CTR prediction proble In online advertising campaigns, to measure purchase propensity, click-through rate (CTR), defined as a ratio of number of clicks to number of impressions, is one of the most informative metrics used in business activities such as performance evaluation and budget planning. No matter what channel an ad goes through (display ads, sponsored search or contextual advertising), CTR estimation for rare events is essential but challenging, often incurring with huge variance, due to the sparsity in data. In this chapter, to alleviate this sparsity, we develop models and methods to smoothen CTR estimation by taking advantage of the data hierarchy in nature or by clustering and data continuity in time to leverage information from data close to the events of interest. In a contextual advertising system running at Yahoo!, we demonstrate that our methods lead to significantly more accurate estimation of CTRs.