We find that the essential ingredients of success are: 1) A minimal, but non-goal-directed “bootstrap" dataset to overcome an initial difficult exploration problem [1]}. 2) Relabeled goal conditioned imitation: GoalsEye uses simple and sample efficient relabeled behavior cloning [1]}, [3]}, [4]}, to train a goal-directed policy to reach any goal state in the dataset without reward information. 3) Iterative self-supervised goal reaching: The agent improves continuously by giving itself random goals, then attempting to reach them using the current policy [5]}. All attempts, including failures, are relabeled into a continuously expanding training set.
