Logit-response dynamics (Alos-Ferrer and Netzer, Games and Economic Behavior 2010) are a rich and natural class of noisy best-response dynamics. In this work we revise the price of anarchy and the price of stability by considering the quality of long-run equilibria in these dynamics. Our results show that prior studies on simpler dynamics of this type can strongly depend on a synchronous schedule of the players' moves. In particular, a small noise by itself is not enough to improve the quality of equilibria as soon as other very natural schedules are used.
Stochastic stability is the tool to show that in the coordination game players select the risk dominant strategy (see @cite_11 for discussion and references). Recent works have pointed out that in general revision processes (including independent-learning) logit-response dynamics do converge to Nash equilibria in potential games ( discuss this issue explicitly).
@cite_11
2172106845 2254533881 2548493877 2031564575
We develop a characterization of stochastically stable states for the logit-response learning dynamics in games, with arbitrary specification of revision opportunities. The result allows us to show convergence to the set of Nash equilibria in the class of best-response potential games and the failure of the dynamics to select potential maximizers beyond the class of exact potential games. We also study to which extent equilibrium selection is robust to the specification of revision opportunities. Our techniques can be extended and applied to a wide class of learning dynamics in games. We investigate a class of reinforcement learning dynamics where players adjust their strategies based on their actions' cumulative payoffs over time-specifically, by playing mixed strategies that maximize their expected cumulative payoff minus a regularization term. A widely studied example is exponential reinforcement learning, a process induced by an entropic regularization term which leads mixed strategies to evolve according to the replicator dynamics. However, in contrast to the class of regularization functions used to define smooth best responses in models of stochastic fictitious play, the functions used in this paper need not be infinitely steep at the boundary of the simplex; in fact, dropping this requirement gives rise to an important dichotomy between steep and nonsteep cases. In this general framework, we extend several properties of exponential learning, including the elimination of dominated strategies, the asymptotic stability of strict Nash equilibria, and the convergence of time-averaged trajectories in zero-sum games with an interior Nash equilibrium. We consider in this chapter a class of two-player nonzero-sum stochastic games with incomplete information, which is inspired by recent applications of game theory in network security. We develop fully distributed reinforcement learning algorithms, which require for each player a minimal amount of information regarding the other player. At each time, each player can be in an active mode or in a sleep mode. If †This material is based upon work supported in part by the U.S. Air Force Office of Scientific Research (AFOSR) under grant number AFOSR MURI FA9550-09-1-0249. D R A F T October 2, 2011, 1:59am D R A F T 2 HYBRID LEARNING IN STOCHASTIC GAMES AND ITS APPLICATION IN NETWORK SECURITY a player is in an active mode, she updates her strategy and estimates of unknown quantities using a specific pure or hybrid learning pattern. The players’ intelligence and rationality are captured by the weighted linear combination of different learning patterns. We use stochastic approximation techniques to show that, under appropriate conditions, the pure or hybrid learning schemes with random updates can be studied using their deterministic ordinary differential equation (ODE) counterparts. Convergence to state-independent equilibria is analyzed for special classes of games, namely, games with two actions, and potential games. Results are applied to network security games between an intruder and an administrator, where the noncooperative behaviors are well characterized by the features of distributed hybrid learning. In stochastic dynamical systems, different concepts of stability can be obtained in different limits. A particularly interesting example is evolutionary game theory, which is traditionally based on infinite populations, where strict Nash equilibria correspond to stable fixed points that are always evolutionarily stable. However, in finite populations stochastic effects can drive the system away from strict Nash equilibria, which gives rise to a new concept for evolutionary stability. The conventional and the new stability concepts may apparently contradict each other leading to conflicting predictions in large yet finite populations. We show that the two concepts can be derived from the frequency dependent Moran process in different limits. Our results help to determine the appropriate stability concept in large finite populations. The general validity of our findings is demonstrated showing that the same results are valid employing vastly different co-evolutionary processes.