The most commonly used RL algorithm in mobile health interventions are bandit algorithms [1]}, [2]}, [3]}, one of the simplest types of RL algorithms. Bandit algorithms are commonly used because of their ability to run reliably and stably in an online environment, which is critical for high-stakes clinical trial settings. Mobile health clinical trials can require years of work by an interdisciplinary team to develop and the RL algorithm cannot be trivially changed once the study begins. So it is critical that the RL algorithm runs stably (e.g., the algorithm cannot take too long to update and fail to be ready to select actions each day) [4]}.
