@inproceedings{
liu2023tail,
title={{TAIL}: Task-specific Adapters for Imitation Learning with Large Pretrained Models},
author={Zuxin Liu and Jesse Zhang and Kavosh Asadi and Yao Liu and Ding Zhao and Shoham Sabach and Rasool Fakoor},
booktitle={NeurIPS 2023 Foundation Models for Decision Making Workshop},
year={2023},
url={https://openreview.net/forum?id=jnzjnYTSlO}
}
The full potential of large pretrained models remains largely untapped in control domains like robotics. This is mainly because of the scarcity of data and the computational challenges associated with training or fine-tuning these large models for such applications. Prior work mainly emphasizes effective \emph{pretraining} of large models for decision-making, with little exploration into how to perform data-efficient continual \emph{adaptation} of these models for new tasks. Recognizing these constraints, we introduce TAIL (Task-specific Adapters for Imitation Learning), a framework for efficient adaptation to new control tasks. Inspired by recent advancements in parameter-efficient fine-tuning in language domains, we explore efficient fine-tuning techniques---e.g., Bottleneck Adapters, P-Tuning, and Low-Rank Adaptation (LoRA)---in TAIL to adapt large pretrained models for new tasks with limited demonstration data. Our extensive experiments comparing prevalent parameter-efficient fine-tuning techniques and adaptation baselines suggest that TAIL with LoRA can achieve the best post-adaptation performance with only 1% of the trainable parameters of full fine-tuning, while avoiding catastrophic forgetting and preserving adaptation plasticity in continual learning settings.
TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models
Zuxin Liu Jesse Zhang Kavosh Asadi Yao Liu Ding Zhao Shoham Sabach Rasool Fakoor