@inproceedings{
rannen-triki2024revisiting,
title={Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models},
author={Amal Rannen-Triki and Jorg Bornschein and Razvan Pascanu and Alexandre Galashov and Michalis Titsias and Marcus Hutter and Andr{\'a}s Gy{\"o}rgy and Yee Whye Teh},
booktitle={NeurIPS 2023 Workshop on Distribution Shifts: New Frontiers with Foundation Models},
year={2024},
url={https://openreview.net/forum?id=iRz8qi7QB8}
}
We consider the problem of online finetuning the parameters of a language model at test time, also known as dynamic evaluation. While it is generally known that this approach improves the overall predictive performance, especially when considering distributional shift between training and evaluation data, we here emphasize the perspective that online-adaptation turns parameters into temporally changing states and provides a form of context-length extension with _memory in weights_, more in line with the concept of _memory_ in neuroscience. 
We pay particular attention to the speed of adaptation (in terms of sample efficiency), sensitivity to overall distributional drift, 
and computational overhead for performing gradient computation and parameter updates. Our empirical study provides insights on when online adaptation is particularly interesting. We highlight that with online adaptation the conceptual distinction between in-context learning and finetuning blurs: Both are methods to condition the model on previously observed tokens.
Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models
Amal Rannen-Triki Jorg Bornschein Razvan Pascanu Alexandre Galashov Michalis Titsias Marcus Hutter András György Yee Whye Teh