The Survival Bandit Problem
We study the survival bandit problem, a variant of the multi-armed bandit
problem introduced in an open problem by Perotto et al. (2019), with a
constraint on the cumulative reward; at each time step, the agent receives a
(possibly negative) reward and if the cumulative reward becomes lower than a
prespecified threshold, the procedure stops, and this phenomenon is called
ruin. This is the first paper studying a framework where the ruin might occur
but not always. We first discuss that a sublinear regret is unachievable under
a naive definition of the regret. Next, we provide tight lower bounds on the
probability of ruin (as well as matching policies). Based on this lower bound,
we define the survival regret as an objective to minimize and provide a policy
achieving a sublinear survival regret (at least in the case of integral
rewards) when the time horizon $T$ is known.