@inproceedings{
adkins2024a,
title={A Method for Evaluating Hyperparameter Sensitivity in Reinforcement Learning},
author={Jacob Adkins and Michael Bowling and Adam White},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=4OJdZhcwBb}
}
The performance of modern reinforcement learning algorithms critically relies
on tuning ever increasing numbers of hyperparameters. Often, small changes in
a hyperparameter can lead to drastic changes in performance, and different environments require very different hyperparameter settings to achieve state-of-the-art
performance reported in the literature. We currently lack a scalable and widely
accepted approach to characterizing these complex interactions. This work proposes a new empirical methodology for studying, comparing, and quantifying the
sensitivity of an algorithm’s performance to hyperparameter tuning for a given set
of environments. We then demonstrate the utility of this methodology by assessing
the hyperparameter sensitivity of several commonly used normalization variants of
PPO. The results suggest that several algorithmic performance improvements may,
in fact, be a result of an increased reliance on hyperparameter tuning.
A Method for Evaluating Hyperparameter Sensitivity in Reinforcement Learning
Jacob Adkins Michael Bowling Adam White