Concerns around privacy in statistical analysis have existed for decades [1]}.
The specific problem of satisfying privacy concerns while maximizing reward in a reinforcement learning context was introduced in [2]} and [3]}. Since then, several works have tackled the privacy-preserving RL problem in various special cases, such as linear contextual bandits [4]}, [5]}, multi-armed bandits [6]}, [7]}, [8]}, and online learning with bandit feedback [9]}, [10]}. In non-bandit settings where the agent's actions result in changes to the state (i.e. the `general RL' setting) there is much less prior work. In the tabular setting [11]} formulate a method with guarantees on regret and privacy which match the known lower bounds in terms of dependence on certain environment-specific parameters such as action space size. The only prior work we are aware of in the continuous state setting is [12]}, developing a variant of Q-learning which can find policies which obey differential privacy constraints on the reward function.
