To improve the data-efficiency further in the sim-to-real policy adaptation approach, repertoire-based learning approaches optimize the policy on a discretized outcome-space, which is often of a lower dimensionality than the policy parameter space [1]}, [2]}. The outcome-space can be defined as a user-defined space where the outcome or the behaviour of the policies can be specified. For a robot hitting a hockey puck, for instance, the outcome-space can be defined as the 2D space of \(\langle x,y \rangle \)  coordinate position of the puck after executing the policy on the robot. Similarly, for a walking robot, the outcome-space can correspond to the different types of gaits produced by the policies on the robot. The core idea behind this approach is to evolve a large repertoire (i.e., a collection) of high rewarding policies in simulation and associate each of them with a unique outcome in the discretized outcome-space. Then, on the physical robot, the optimal policy is chosen typically through Bayesian optimization [3]} in the outcome-space. The main hypothesis of this approach is that due to the diversity of the policies in the repertoire, some policies in the repertoire will still produce high rewards on the physical robot even in the presence of a large reality-gap. For instance, a robot with a damaged leg can still walk if the repertoire has a policy to produce a walking gait that does not use the broken leg for walking.
