Goal conditioned imitation learning.
While many of the above methods have been shown to scale to undirected table tennis, few have tackled the problem of goal-directed table tennis. Goal directed control is an active area of robot learning, with many recent examples in both IL and RL [1]}, [2]}, [3]}, [4]}. Given the complexities of even single task real world robot learning [5]} finding simple methods that scale to goal-directed real world behavior remains an open question. While goal-conditioned imitation learning [2]}, [3]} offers a simple approach to multitask control, no instances yet have been shown to scale to hard physical problems like the one studied in this work, being largely validated in simulation instead. We find surprisingly that the simple combination of two existing IL methods [2]}, [9]} indeed scales to this setting, while being able to 1) learn from less burdensome suboptimal (in the sense of being non goal-directed) demonstrations, 2) use relabeled learning to learn goal-reaching without rewards, and 3) continuously self-improve beyond the initial data by using self-supervised goal reaching.
