@inproceedings{
kirsch2023towards,
title={Towards General-Purpose In-Context Learning Agents},
author={Louis Kirsch and James Harrison and C. Freeman and Jascha Sohl-Dickstein and J{\"u}rgen Schmidhuber},
booktitle={NeurIPS 2023 Workshop on Generalization in Planning},
year={2023},
url={https://openreview.net/forum?id=eDZJTdUsfe}
}
Reinforcement Learning (RL) algorithms are usually hand-crafted, driven by the research and engineering of humans. An alternative approach is to automate this research process via meta-learning. A particularly ambitious objective is to automatically discover new RL algorithms from scratch that use in-context learning to learn-how-to-learn entirely from data while also generalizing to a wide range of environments. Those RL algorithms are implemented entirely in neural networks, by conditioning on previous experience from the environment, without any explicit optimization-based routine at meta-test time. To achieve generalization, this requires a broad task distribution of diverse and challenging environments. Our Transformer-based Generally Learning Agents (GLAs) are an important first step in this direction. Our GLAs are meta-trained using supervised learning techniques on an offline dataset with experiences from RL environments that is augmented with random projections to generate task diversity. During meta-testing our agents perform in-context meta-RL on entirely different robotic control problems such as Reacher, Cartpole, or HalfCheetah that were not in the meta-training distribution.
Towards General-Purpose In-Context Learning Agents
Louis Kirsch James Harrison C. Freeman Jascha Sohl-Dickstein Jürgen Schmidhuber