An exploration challenge. One of the constraints set for controlling the system is to avoid switching the pump operation too frequently to protect the assets. Thus, exhaustive exploration strategies such as epsilon-greedy [1]} are contradictory to the idea of performing the same action selection over many timesteps and may prematurely converge to suboptimal policies. Furthermore, approaches that attempt to cover unknown areas of state space can lead to meaningless states that do not meet the safety constraints. Thus, the pump scheduling problem can be a setting for evaluating exploration strategies that aim to enhance sample efficiency or limit state visitation to those that meet system safety constraints [2]}.
