The problem of finding an efficient learning rule is perhaps best illustrated with a key model of evolutionary game theory: the repeated prisoner's dilemma [1]}. In each round of the game, two players independently choose whether to cooperate or defect. Payoffs are such that players have an incentive to defect even though mutual cooperation is in everybody's interest. In the limit of infinitely many rounds, there is a wide range of possible equilibria [2]}. In particular, players may always cooperate, always defect, or alternate between cooperation and defection [3]}.
After decades of research, much is known about particular strategies that sustain cooperation, such as “tit-for-tat” [4]} or “win-stay, lose-shift” [5]}. Much less is known how such desirable strategies can be learned in the first place, especially when players are learning concurrently. If strategy updating is described by imitation, for example, players may end up defecting for a substantial amount of time [6]}. Such detrimental outcomes become increasingly likely when the game involves only a few rounds, offers a small benefit of cooperation, or when players commit errors [7]}.
