We introduce -Rank, a principled evolutionary dynamics methodology for the evaluation and ranking of agents in large-scale multi-agent interactions, grounded in a novel dynamical game-theoretic solution concept called Markov-Conley chains (MCCs). The approach leverages continuous- and discrete-time evolutionary dynamical systems applied to empirical games, and scales tractably in the number of agents, the type of interactions, and the type of empirical games (symmetric and asymmetric). Current models are fundamentally limited in one or more of these dimensions and are not guaranteed to converge to the desired game-theoretic solution concept (typically the Nash equilibrium). -Rank provides a ranking over the set of agents under evaluation and provides insights into their strengths, weaknesses, and long-term dynamics. This is a consequence of the links we establish to the MCC solution concept when the underlying evolutionary model's ranking-intensity parameter, , is chosen to be large, which exactly forms the basis of -Rank. In contrast to the Nash equilibrium, which is a static concept based on fixed points, MCCs are a dynamical solution concept based on the Markov chain formalism, Conley's Fundamental Theorem of Dynamical Systems, and the core ingredients of dynamical systems: fixed points, recurrent sets, periodic orbits, and limit cycles. -Rank runs in polynomial time with respect to the total number of pure strategy profiles, whereas computing a Nash equilibrium for a general-sum game is known to be intractable. We introduce proofs that not only provide a unifying perspective of existing continuous- and discrete-time evolutionary evaluation models, but also reveal the formal underpinnings of the -Rank methodology. We empirically validate the method in several domains including AlphaGo, AlphaZero, MuJoCo Soccer, and Poker.
From the theoretical biology perspective, researchers have additionally deployed discrete-time evolutionary dynamics models @cite_49 . These models typically provide insights in the macro-dynamics of the overall behavior of agents in strategy space, corresponding to flow rates at the edges of a manifold @cite_65 @cite_52 @cite_33 @cite_73 @cite_63 . These studies usually focus on biological games, the evolution of cooperation and fairness in social dilemma's like the iterated prisoner's dilemma or signalling games, deploying, amongst others, imitation dynamics with low mutation rates @cite_56 @cite_1 . Similar efforts investigating evolutionary dynamics inspired by statistical physics models have been taken as well @cite_81 @cite_82 .
@cite_33 @cite_65 @cite_52 @cite_1 @cite_56 @cite_81 @cite_49 @cite_63 @cite_73 @cite_82
2031564575 2097141283 2156173537 220156613 2119086496 2897212219 604170243 2042040675 2051166571 2771187719
In stochastic dynamical systems, different concepts of stability can be obtained in different limits. A particularly interesting example is evolutionary game theory, which is traditionally based on infinite populations, where strict Nash equilibria correspond to stable fixed points that are always evolutionarily stable. However, in finite populations stochastic effects can drive the system away from strict Nash equilibria, which gives rise to a new concept for evolutionary stability. The conventional and the new stability concepts may apparently contradict each other leading to conflicting predictions in large yet finite populations. We show that the two concepts can be derived from the frequency dependent Moran process in different limits. Our results help to determine the appropriate stability concept in large finite populations. The general validity of our findings is demonstrated showing that the same results are valid employing vastly different co-evolutionary processes. Darwinian dynamics based on mutation and selection form the core of mathematical models for adaptation and coevolution of biological populations. The evolutionary outcome is often not a fitness-maximizing equilibrium but can include oscillations and chaos. For studying frequency-dependent selection, game-theoretic arguments are more appropriate than optimization algorithms. Replicator and adaptive dynamics describe short- and long-term evolution in phenotype space and have found applications ranging from animal behavior and ecology to speciation, macroevolution, and human language. Evolutionary game theory is an essential component of a mathematical and computational approach to biology. We study evolutionary game dynamics in finite populations. We analyze an evolutionary process, which we call pairwise comparison, for which we adopt the ubiquitous Fermi distribution function from statistical mechanics. The inverse temperature in this process controls the intensity of selection, leading to a unified framework for evolutionary dynamics at all intensities of selection, from random drift to imitation dynamics. We derive a simple closed formula that determines the feasibility of cooperation in finite populations, whenever cooperation is modeled in terms of any symmetric two-person game. In contrast with previous results, the present formula is valid at all intensities of selection and for any initial condition. We investigate the evolutionary dynamics of cooperators in finite populations, and study the interplay between intensity of selection and the remnants of interior fixed points in infinite populations, as a function of a given initial number of cooperators, showing how this interplay strongly affects the approach to fixation of a given trait in finite populations, leading to counterintuitive results at different intensities of selection. We model evolution according to an asymmetric game as occurring in multiple finite populations, one for each role in the game, and study the effect of subjecting individuals to stochastic strategy mutations. We show that, when these mutations occur sufficiently infrequently, the dynamics over all population states simplify to an ergodic Markov chain over just the pure population states (where each population is monomorphic). This makes calculation of the stationary distribution computationally feasible. The transition probabilities of this embedded Markov chain involve fixation probabilities of mutants in single populations. The asymmetry of the underlying game leads to fixation probabilities that are derived from frequency-independent selection, in contrast to the analogous single-population symmetric-game case (Fudenberg and Imhof, 2006). This frequency independence is useful in that it allows us to employ results from the population genetics literature to calculate the stationary distribution of the evolutionary process, giving sharper, and sometimes even analytic, results. We demonstrate the utility of this approach by applying it to a battle-of-the-sexes game, a Crawford–Sobel signalling game, and the beer-quiche game of Cho and Kreps (1987). This note characterizes the impact of adding rare stochastic mutations to an â€œimitation dynamic,â€ meaning a process with the properties that absent strategies remain absent, and non-homogeneous states are transient. The resulting system will spend almost all of its time at the absorbing states of the no-mutation process. The work of Freidlin and Wentzell [Random Perturbations of Dynamical Systems, Springer, New York, 1984] and its extensions provide a general algorithm for calculating the limit distribution, but this algorithm can be complicated to apply. This note provides a simpler and more intuitive algorithm. Loosely speaking, in a process with K strategies, it is sufficient to find the invariant distribution of a KA—K Markov matrix on the K homogeneous states, where the probability of a transit from â€œall play iâ€ to â€œall play jâ€ is the probability of a transition from the state â€œall agents but 1 play i, 1 plays jâ€ to the state â€œall play jâ€ . Pro-social punishment and exclusion are common means to elevate the level of cooperation among unrelated individuals. Indeed, it is worth pointing out that the combined use of these two strategies is quite common across human societies. However, it is still not known how a combined strategy where punishment and exclusion are switched can promote cooperation from the theoretical perspective. In this paper, we thus propose two different switching strategies, namely, peer switching that is based on peer punishment and peer exclusion, and pool switching that is based on pool punishment and pool exclusion. Individuals adopting the switching strategy will punish defectors when their numbers are below a threshold and exclude them otherwise. We study how the two switching strategies influence the evolutionary dynamics in the public goods game. We show that an intermediate value of the threshold leads to a stable coexistence of cooperators, defectors, and players adopting the switching strategy in a well-mixed population, and this regardless of whether the pool-based or the peer-based switching strategy is introduced. Moreover, we show that the pure exclusion strategy alone is able to evoke a limit cycle attractor in the evolutionary dynamics, such that cooperation can coexist with other strategies.Pro-social punishment and exclusion are common means to elevate the level of cooperation among unrelated individuals. Indeed, it is worth pointing out that the combined use of these two strategies is quite common across human societies. However, it is still not known how a combined strategy where punishment and exclusion are switched can promote cooperation from the theoretical perspective. In this paper, we thus propose two different switching strategies, namely, peer switching that is based on peer punishment and peer exclusion, and pool switching that is based on pool punishment and pool exclusion. Individuals adopting the switching strategy will punish defectors when their numbers are below a threshold and exclude them otherwise. We study how the two switching strategies influence the evolutionary dynamics in the public goods game. We show that an intermediate value of the threshold leads to a stable coexistence of cooperators, defectors, and players adopting the switching strategy in a well-mixed pop... Preface 1. Introduction 2. What Evolution Is 3. Fitness Landscapes and Sequence Spaces 4. Evolutionary Games 5. Prisoners of the Dilemma 6. Finite Populations 7. Games in Finite Populations 8. Evolutionary Graph Theory 9. Spatial Games 10. HIV Infection 11. The Evolution of Virulence 12. The Evolutionary Dynamics of Cancer 13. Language Evolution 14. Conclusion Further Reading References Index Often groups need to meet repeatedly before a decision is reached. Hence, most individual decisions will be contingent on decisions taken previously by others. In particular, the decision to cooperate or not will depend on one’s own assessment of what constitutes a fair group outcome. Making use of a repeated N-person prisoner’s dilemma, we show that reciprocation towards groups opens a window of opportunity for cooperation to thrive, leading populations to engage in dynamics involving both coordination and coexistence, and characterized by cycles of cooperation and defection. Furthermore, we show that this process leads to the emergence of fairness, whose level will depend on the dilemma at stake. A finite-population dynamic evolutionary model is presented, which shows that increasing the individual capacity of sending pre-play signals (without any pre-defined meaning), opens a route for cooperation. The population dynamics leads individuals to discriminate between different signals and react accordingly to the signals received. The proportion of time that the population spends in different states can be calculated analytically. We show that increasing the number of different signals benefits cooperative strategies, illustrating how cooperators may take profit from a diverse signaling portfolio to forecast future behaviors and avoid being cheated by defectors. Cooperation is a difficult proposition in the face of Darwinian selection. Those that defect have an evolutionary advantage over cooperators who should therefore die out. However, spatial structure enables cooperators to survive through the formation of homogeneous clusters, which is the hallmark of network reciprocity. Here we go beyond this traditional setup and study the spatiotemporal dynamics of cooperation in a population of populations. We use the prisoner's dilemma game as the mathematical model and show that considering several populations simultaneously gives rise to fascinating spatiotemporal dynamics and pattern formation. Even the simplest assumption that strategies between different populations are payoff-neutral with one another results in the spontaneous emergence of cyclic dominance, where defectors of one population become prey of cooperators in the other population, and vice versa. Moreover, if social interactions within different populations are characterized by significantly different temptations to defect, we observe that defectors in the population with the largest temptation counterintuitively vanish the fastest, while cooperators that hang on eventually take over the whole available space. Our results reveal that considering the simultaneous presence of different populations significantly expands the complexity of evolutionary dynamics in structured populations, and it allows us to understand the stability of cooperation under adverse conditions that could never be bridged by network reciprocity alone.