Surprisingly, counter to what has been hypothesized in prior work [1]}, we found that it is possible to train a standard ES [2]}, [3]}, [4]}, [5]}, [6]} policy to convergence fully in simulation, then apply it directly to the real robot as a means of obtaining an initial dataset. We found empirically that this approach was sufficient to get a real robot to make safe contact with the ball, bootstrapping further autonomous learning. However, we note that the initial policy obtained by ES was narrow, causing most of the examples to land close to the net in the right half of the opponent's side of the table (see Figure REF ). To overcome this limitation and more effectively cover the test goal distribution, we applied steps (2) relabeled imitation learning and (3) self-practice, setting the goal during policy rollouts higher up the table on the opponent side, with the intention of shifting the distribution up. Then we sought to expand the goal coverage by changing the goal distribution to the entirety of the opponent's side of the table. Additionally we perturbed 4 of the 8 robot joint angles during the data gathering stage in order to increase the variety of the ball landing points. The final demonstration dataset ball landing distribution is shown in Figure REF .
