User-Interactive Offline Reinforcement Learning
Offline reinforcement learning algorithms still lack trust in practice due to
the risk that the learned policy performs worse than the original policy that
generated the dataset or behaves in an unexpected way that is unfamiliar to the
user. At the same time, offline RL algorithms are not able to tune their most
important hyperparameter - the proximity of the learned policy to the original
policy. We propose an algorithm that allows the user to tune this
hyperparameter at runtime, thereby overcoming both of the above mentioned
issues simultaneously. This allows users to start with the original behavior
and grant successively greater deviation, as well as stopping at any time when
the policy deteriorates or the behavior is too far from the familiar one.