@inproceedings{
ramesh2023how,
title={How Capable Can a Transformer Become? A Study on Synthetic, Interpretable Tasks},
author={Rahul Ramesh and Mikail Khona and Robert P. Dick and Hidenori Tanaka and Ekdeep Singh Lubana},
booktitle={NeurIPS 2023 Workshop on Symmetry and Geometry in Neural Representations},
year={2023},
url={https://openreview.net/forum?id=fv0W1Yyg2v}
}
Transformers trained on huge text corpora exhibit a remarkable set of capabilities. Given the inherent compositional nature of language, one can expect the model to learn to compose these capabilities, potentially yielding a combinatorial explosion of what operations it can perform on an input. Motivated by the above, we aim to assess in this paper “how capable can a transformer become?”. In this work, we train Transformer models on a data-generating process that involves compositions of a set of well-defined monolithic capabilities and show that: (1) Transformers generalize to exponentially or even combinatorially many functions not seen in the training data; (2) composing functions by generating intermediate outputs is more effective at generalizing to unseen compositions; (3) the training data has a significant impact on the model’s ability to compose functions (4) Attention layers in the latter half of the model seem critical to compositionality.
How Capable Can a Transformer Become? A Study on Synthetic, Interpretable Tasks
Rahul Ramesh Mikail Khona Robert P. Dick Hidenori Tanaka Ekdeep Singh Lubana