Functional Optimization Reinforcement Learning for Real-Time Bidding
Real-time bidding is the new paradigm of programmatic advertising. An
advertiser wants to make the intelligent choice of utilizing a
\textbf{Demand-Side Platform} to improve the performance of their ad campaigns.
Existing approaches are struggling to provide a satisfactory solution for
bidding optimization due to stochastic bidding behavior. In this paper, we
proposed a multi-agent reinforcement learning architecture for RTB with
functional optimization. We designed four agents bidding environment: three
Lagrange-multiplier based functional optimization agents and one baseline agent
(without any attribute of functional optimization) First, numerous attributes
have been assigned to each agent, including biased or unbiased win probability,
Lagrange multiplier, and click-through rate. In order to evaluate the proposed
RTB strategy's performance, we demonstrate the results on ten sequential
simulated auction campaigns. The results show that agents with functional
actions and rewards had the most significant average winning rate and winning
surplus, given biased and unbiased winning information respectively. The
experimental evaluations show that our approach significantly improve the
campaign's efficacy and profitability.