Previous work has examined the problem of learning differentially-private (DP) policies in the bandit [1]}, [2]}, [3]} and Markov Decision Process (MDP) setting [4]}, [5]}. However, our mutual-information privacy constraint is qualitatively different from a differential privacy constraint, as we describe in the related work section. We experimentally demonstrate the difference between a DP RL algorithm and our MI-constrained approach, comparing the policies learned on a small environment.
Another strand of information-aware RL research has investigated inferring agents' goals from observation of states and actions [6]}, [7]}.
This strand of research is closest to this work, but has a different problem setting: in our problem an adversary intends to infer the sensitive state only given the actions, and not any of the state variables.
We also allow our sensitive state to influence the environmental dynamics, while agents' goals typically do not.
