@inproceedings{
galashov2024nonstationary,
title={Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset},
author={Alexandre Galashov and Michalis Titsias and Andr{\'a}s Gy{\"o}rgy and Clare Lyle and Razvan Pascanu and Yee Whye Teh and Maneesh Sahani},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=fDiZJ7mmOV}
}
Neural networks are traditionally trained under the assumption that data come from a stationary distribution. However, settings which violate this assumption are becoming more popular;  examples include supervised learning under distributional shifts, reinforcement learning, continual learning and non-stationary contextual bandits. In this work we introduce a novel learning approach that automatically models and adapts to non-stationarity, via an Ornstein-Uhlenbeck process with an adaptive drift parameter. The adaptive drift tends to draw the parameters towards the initialisation distribution, so the approach can be understood as a form of soft parameter reset. We show empirically that our approach performs well in non-stationary supervised and off-policy reinforcement learning settings.
Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset
Alexandre Galashov Michalis Titsias András György Clare Lyle Razvan Pascanu Yee Whye Teh Maneesh Sahani