In natural evolution, the simple yet powerful process of natural selection has led to the vast complexity of the living world. It is believed that one of the main drivers of biological complexity are evolutionary arms races between competing organisms [1]}. A similar dynamic of competing agents creating new tasks for each other has been exploited in competitive multi-agent reinforcement learning (RL) to reach super-human performance in games such as Backgammon [2]}, Go [3]}, Dota [4]} and Starcraft [5]} and to showcase impressive emergent complexity from simple game rules in physically grounded environments [6]}, [7]}, [8]}, [9]}, [10]}. Yet just as evolution happened in bursts at specific times and places [11]}, finding the precise environmental conditions that lead to interesting emergent skills in complex environments is often a time-consuming and laborious process. One strategy to mitigate this problem is to train agents on a wide distribution of randomized environments, in the hope that some small fraction of them will lead to the emergence of a new skill [10]}, [4]}, [8]}.
