Leveraging Language for Accelerated Learning of Tool Manipulation
Robust and generalized tool manipulation requires an understanding of the
properties and affordances of different tools. We investigate whether
linguistic information about a tool (e.g., its geometry, common uses) can help
control policies adapt faster to new tools for a given task. We obtain diverse
descriptions of various tools in natural language and use pre-trained language
models to generate their feature representations. We then perform
language-conditioned meta-learning to learn policies that can efficiently adapt
to new tools given their corresponding text descriptions. Our results
demonstrate that combining linguistic information and meta-learning
significantly accelerates tool learning in several manipulation tasks including
pushing, lifting, sweeping, and hammering.