AnyMorph: Learning Transferable Polices By Inferring Agent Morphology
The prototypical approach to reinforcement learning involves training
policies tailored to a particular agent from scratch for every new morphology.
Recent work aims to eliminate the re-training of policies by investigating
whether a morphology-agnostic policy, trained on a diverse set of agents with
similar task objectives, can be transferred to new agents with unseen
morphologies without re-training. This is a challenging problem that required
previous approaches to use hand-designed descriptions of the new agent's
morphology. Instead of hand-designing this description, we propose a
data-driven method that learns a representation of morphology directly from the
reinforcement learning objective. Ours is the first reinforcement learning
algorithm that can train a policy to generalize to new agent morphologies
without requiring a description of the agent's morphology in advance. We
evaluate our approach on the standard benchmark for agent-agnostic control, and
improve over the current state of the art in zero-shot generalization to new
agents. Importantly, our method attains good performance without an explicit
description of morphology.