As the largest e-commerce platform, Taobao helps advertisers reach billions of search requests each day with its sponsored search service, which has also contributed considerable revenue to the platform. How to design a suit of marketing optimization tool to cater various advertiser demands while balancing platform revenue and consumer experience is significant to a healthy and sustainable marketing ecosystem, among which bidding strategy plays a critical role. Traditional keyword-level bidding optimization only provides a coarse-grained match between advertisement and impression. Meanwhile impression-level expected value bidder is not applicable to various demand optimization of massive advertisers, not to mention its lack of mechanism to balance benefits of three parties. In this paper we propose , a bidding optimization framework which designs an impression-level bidding strategy to reflect advertiser's conversion willingness and budget control. In this way, with a simplified control ability for advertisers, CIA is capable of fulfilling various e-commerce advertiser demands in different levels, such as GMV optimization, style comparison etc. Additionally, a replay based simulation system is designed to predict the performance of different take-rate. CIA unifies the benefits of three parties in the marketing ecosystem without changing the classic expected Cost Per Mille mechanism. Our extensive offline simulations and large-scale online experiments on platform verify the high effectiveness of the CIA framework. Moreover, CIA has been deployed online as a major bidding tool for advertisers in TSA.
Other novel work also exists in display advertising. Considering the sequential decision processes in impression bidding, utilized reinforcement learning to bid @cite_10 . With the assumption of i.i.d impression features, they derived an optimal real-time bidder under budget and auction volume constraints. A similar idea was also used in JD DSP business, with deep Q-learner in its bidder @cite_14 . Reinforcement learning is a straightforward solution to the scenario of delayed rewards in online advertising. improved single-impression bidder by attributing post-impression actions into previous bids to decouple the sequential bidding process @cite_3 . Another related line of bidding optimization research lies in smart pacing, which helps budget-limited advertisers decide whether to bid or an impression or not @cite_11 @cite_7 .
@cite_14 @cite_7 @cite_3 @cite_10 @cite_11
2562337727 2513944453 2770888277 2033798573
The majority of online display ads are served through real-time bidding (RTB) --- each ad display impression is auctioned off in real-time when it is just being generated from a user visit. To place an ad automatically and optimally, it is critical for advertisers to devise a learning algorithm to cleverly bid an ad impression in real-time. Most previous works consider the bid decision as a static optimization problem of either treating the value of each impression independently or setting a bid price to each segment of ad volume. However, the bidding for a given ad campaign would repeatedly happen during its life span before the budget runs out. As such, each bid is strategically correlated by the constrained budget and the overall effectiveness of the campaign (e.g., the rewards from generated clicks), which is only observed after the campaign has completed. Thus, it is of great interest to devise an optimal bidding strategy sequentially so that the campaign budget can be dynamically allocated across all the available impressions on the basis of both the immediate and future rewards. In this paper, we formulate the bid decision process as a reinforcement learning problem, where the state space is represented by the auction information and the campaign's real-time parameters, while an action is the bid price to set. By modeling the state transition via auction competition, we build a Markov Decision Process framework for learning the optimal bidding policy to optimize the advertising performance in the dynamic real-time bidding environment. Furthermore, the scalability problem from the large real-world auction volume and campaign budget is well handled by state value approximation using neural networks. The empirical study on two large-scale real-world datasets and the live A B testing on a commercial platform have demonstrated the superior performance and high efficiency compared to state-of-the-art methods. In real-time display advertising, ad slots are sold per impression via an auction mechanism. For an advertiser, the campaign information is incomplete --- the user responses (e.g, clicks or conversions) and the market price of each ad impression are observed only if the advertiser's bid had won the corresponding ad auction. The predictions, such as bid landscape forecasting, click-through rate (CTR) estimation, and bid optimisation, are all operated in the pre-bid stage with full-volume bid request data. However, the training data is gathered in the post-bid stage with a strong bias towards the winning impressions. A common solution for learning over such censored data is to reweight data instances to correct the discrepancy between training and prediction. However, little study has been done on how to obtain the weights independent of previous bidding strategies and consequently integrate them into the final CTR prediction and bid generation steps. In this paper, we formulate CTR estimation and bid optimisation under such censored auction data. Derived from a survival model, we show that historic bid information is naturally incorporated to produce Bid-aware Gradient Descents (BGD) which controls both the importance and the direction of the gradient to achieve unbiased learning. The empirical study based on two large-scale real-world datasets demonstrates remarkable performance gains from our solution. The learning framework has been deployed on Yahoo!'s real-time bidding platform and provided 2.97 AUC lift for CTR estimation and 9.30 eCPC drop for bid optimisation in an online A B test. Real-time bidding (RTB) based display advertising has become one of the key technological advances in computational advertising. RTB enables advertisers to buy individual ad impressions via an auction in real-time and facilitates the evaluation and the bidding of individual impressions across multiple advertisers. In RTB, the advertisers face three main challenges when optimizing their bidding strategies, namely (i) estimating the utility (e.g., conversions, clicks) of the ad impression, (ii) forecasting the market value (thus the cost) of the given ad impression, and (iii) deciding the optimal bid for the given auction based on the first two. Previous solutions assume the first two are solved before addressing the bid optimization problem. However, these challenges are strongly correlated and dealing with any individual problem independently may not be globally optimal. In this paper, we propose Bidding Machine , a comprehensive learning to bid framework, which consists of three optimizers dealing with each challenge above, and as a whole, jointly optimizes these three parts. We show that such a joint optimization would largely increase the campaign effectiveness and the profit. From the learning perspective, we show that the bidding machine can be updated smoothly with both offline periodical batch or online sequential training schemes. Our extensive offline empirical study and online A B testing verify the high effectiveness of the proposed bidding machine. We describe a real-time bidding algorithm for performance-based display ad allocation. A central issue in performance display advertising is matching campaigns to ad impressions, which can be formulated as a constrained optimization problem that maximizes revenue subject to constraints such as budget limits and inventory availability. The current practice is to solve the optimization problem offline at a tractable level of impression granularity (e.g., the page level), and to serve ads online based on the precomputed static delivery scheme. Although this offline approach takes a global view to achieve optimality, it fails to scale to ad allocation at the individual impression level. Therefore, we propose a real-time bidding algorithm that enables fine-grained impression valuation (e.g., targeting users with real-time conversion data), and adjusts value-based bids according to real-time constraint snapshots (e.g., budget consumption levels). Theoretically, we show that under a linear programming (LP) primal-dual formulation, the simple real-time bidding algorithm is indeed an online solver to the original primal problem by taking the optimal solution to the dual problem as input. In other words, the online algorithm guarantees the offline optimality given the same level of knowledge an offline optimization would have. Empirically, we develop and experiment with two real-time bid adjustment approaches to adapting to the non-stationary nature of the marketplace: one adjusts bids against real-time constraint satisfaction levels using control-theoretic methods, and the other adjusts bids also based on the statistically modeled historical bidding landscape. Finally, we show experimental results with real-world ad delivery data that support our theoretical conclusions.