[1]} initiates the study of competitive reinforcement learning under the framework of Markov games and proposes an extension of the single-player Q-learning algorithm, called minimax-Q, which is later proven to converge under some conditions [2]}. While minimax-Q can run in a decentralized manner, it is conservative and only converges to the minimax policy but not the best response to the opponent.
