Domain name registrars and URL shortener service providers place advertisements on the parked domains (Internet domain names which are not in service) in order to generate profits. As the web contents have been removed, it is critical to make sure the displayed ads are directly related to the intents of the visitors who have been directed to the parked domains. Because of the missing contents in these domains, it is non-trivial to generate the keywords to describe the previous contents and therefore the users intents. In this paper we discuss the adaptive keywords extraction problem and introduce an algorithm based on the BM25F term weighting and linear multi-armed bandits. We built a prototype over a production domain registration system and evaluated it using crowdsourcing in multiple iterations. The prototype is compared with other popular methods and is shown to be more effective.
In @cite_25 the authors propose a combination algorithm of using upper confidence bound (UCB) and @math -greedy to solve the exploration-exploitation dilemma. Their target is to select high profit ads which would be fed in contextual advertising platforms. The feature vectors of ads are not used in their system as side information, instead they use standard bandits considering the reward following an unknown stochastic process. Our research is partially inspired by their work.
@cite_25
1822035899 2949366694 2952951433 2950238385
This paper considers stochastic bandits with side observations, a model that accounts for both the exploration exploitation dilemma and relationships between arms. In this setting, after pulling an arm i, the decision maker also observes the rewards for some other actions related to i. We will see that this model is suited to content recommendation in social networks, where users' reactions may be endorsed or not by their friends. We provide efficient algorithms based on upper confidence bounds (UCBs) to leverage this additional information and derive new bounds improving on standard regret guarantees. We also evaluate these policies in the context of movie recommendation in social networks: experiments on real datasets show substantial learning rate speedups ranging from 2.2x to 14x on dense networks. This paper considers the use of a simple posterior sampling algorithm to balance between exploration and exploitation when learning to optimize actions such as in multi-armed bandit problems. The algorithm, also known as Thompson Sampling, offers significant advantages over the popular upper confidence bound (UCB) approach, and can be applied to problems with finite or infinite action spaces and complicated relationships among action rewards. We make two theoretical contributions. The first establishes a connection between posterior sampling and UCB algorithms. This result lets us convert regret bounds developed for UCB algorithms into Bayesian regret bounds for posterior sampling. Our second theoretical contribution is a Bayesian regret bound for posterior sampling that applies broadly and can be specialized to many model classes. This bound depends on a new notion we refer to as the eluder dimension, which measures the degree of dependence among action rewards. Compared to UCB algorithm Bayesian regret bounds for specific model classes, our general bound matches the best available for linear models and is stronger than the best available for generalized linear models. Further, our analysis provides insight into performance advantages of posterior sampling, which are highlighted through simulation results that demonstrate performance surpassing recently proposed UCB algorithms. In this paper, we study the stochastic combinatorial multi-armed bandit (CMAB) framework that allows a general nonlinear reward function, whose expected value may not depend only on the means of the input random variables but possibly on the entire distributions of these variables. Our framework enables a much larger class of reward functions such as the @math function and nonlinear utility functions. Existing techniques relying on accurate estimations of the means of random variables, such as the upper confidence bound (UCB) technique, do not work directly on these functions. We propose a new algorithm called stochastically dominant confidence bound (SDCB), which estimates the distributions of underlying random variables and their stochastically dominant confidence bounds. We prove that SDCB can achieve @math distribution-dependent regret and @math distribution-independent regret, where @math is the time horizon. We apply our results to the @math -MAX problem and expected utility maximization problems. In particular, for @math -MAX, we provide the first polynomial-time approximation scheme (PTAS) for its offline problem, and give the first @math bound on the @math -approximation regret of its online problem, for any @math . We tackle the problem of online reward maximisation over a large finite set of actions described by their contexts. We focus on the case when the number of actions is too big to sample all of them even once. However we assume that we have access to the similarities between actions' contexts and that the expected reward is an arbitrary linear function of the contexts' images in the related reproducing kernel Hilbert space (RKHS). We propose KernelUCB, a kernelised UCB algorithm, and give a cumulative regret bound through a frequentist analysis. For contextual bandits, the related algorithm GP-UCB turns out to be a special case of our algorithm, and our finite-time analysis improves the regret bound of GP-UCB for the agnostic case, both in the terms of the kernel-dependent quantity and the RKHS norm of the reward function. Moreover, for the linear kernel, our regret bound matches the lower bound for contextual linear bandits.