We study a multiplayer stochastic multi-armed bandit problem in which players cannot communicate, and if two or more players pull the same arm, a collision occurs and the involved players receive zero reward. We consider the challenging heterogeneous setting, in which different arms may have different means for different players, and propose a new, efficient algorithm that combines the idea of leveraging forced collisions for implicit communication and that of performing matching eliminations. We give a finite-time analysis of our algorithm, bounding its regret by O((log T)^ 1+ ) for any fixed >0. If the optimal assignment of players to arms is unique, we further show that it attains the optimal O(log(T)) regret, solving an open question raised at NeurIPS 2018.
The only previous paper on the setting we study here is by @cite_16 , who use a very different approach from ours. The Game-of-Thrones algorithm proceeds in epochs, and each epoch is divided into three phases. The first is an exploration phase in which each player select arms at random in order to estimate the mean of each arm. In the second phase (called the GOT phase) the players jointly run a Markov chain (which is called the games of thrones dynamics) whose unique stochastically stable state corresponds to a maximum matching of the empirical means, on which the Markov chain will spend most of its time. The players then enter an exploitation phase in which they keep playing the arm they visited most often during the GOT phase.
@cite_16
2912712349
We consider the restless multiarmed bandit problem with unknown dynamics in which a player chooses one out of N arms to play at each time. The reward state of each arm transits according to an unknown Markovian rule when it is played and evolves according to an arbitrary unknown random process when it is passive. The performance of an arm selection policy is measured by regret, defined as the reward loss with respect to the case where the player knows which arm is the most rewarding and always plays the best arm. We construct a policy with an interleaving exploration and exploitation epoch structure that achieves a regret with logarithmic order. We further extend the problem to a decentralized setting where multiple distributed players share the arms without information exchange. Under both an exogenous restless model and an endogenous restless model, we show that a decentralized extension of the proposed policy preserves the logarithmic regret order as in the centralized setting. The results apply to adaptive learning in various dynamic systems and communication networks, as well as financial investment.