Generalizing to New Tasks via One-Shot Compositional Subgoals
The ability to generalize to previously unseen tasks with little to no
supervision is a key challenge in modern machine learning research. It is also
a cornerstone of a future "General AI". Any artificially intelligent agent
deployed in a real world application, must adapt on the fly to unknown
environments. Researchers often rely on reinforcement and imitation learning to
provide online adaptation to new tasks, through trial and error learning.
However, this can be challenging for complex tasks which require many timesteps
or large numbers of subtasks to complete. These "long horizon" tasks suffer
from sample inefficiency and can require extremely long training times before
the agent can learn to perform the necessary longterm planning. In this work,
we introduce CASE which attempts to address these issues by training an
Imitation Learning agent using adaptive "near future" subgoals. These subgoals
are recalculated at each step using compositional arithmetic in a learned
latent representation space. In addition to improving learning efficiency for
standard long-term tasks, this approach also makes it possible to perform
one-shot generalization to previously unseen tasks, given only a single
reference trajectory for the task in a different environment. Our experiments
show that the proposed approach consistently outperforms the previous
state-of-the-art compositional Imitation Learning approach by 30%.