The previous work that is most closely related to our setting is in the field of mutual information constraints for reinforcement learning. The original motivation for information-regularization was in the context of goal-directed RL.
Here, the agent has a goal \(g_t\)  which effects the agent's choice of action, but not the dynamics.
First, [1]} explored this in the options framework, regularizing \(I(a_t;g_t|\tau _{a_{1:t-1}}, \tau _{x_{1:t-1}})\) .
More recently, both [2]} and [3]} studied the behaviour of policies regularized with the term \(I(a_t;g_t|s_t)\) .
Both explore a rearrangement of this regularizer as the KL-divergence between the learned policy and a `default' policy.
In the case of [2]} this is used to learn policies which have diverse goals, in a similar fashion to the classic information bottleneck learning framework.
In the case of [3]} the motivation is explicitly to learn agents that either share or hide action-goal information (depending on the sign of the regularizer's coefficient).
Although a similar motivation, the threat model considered is different to ours–in theirs, the adversary knows the state at time \(t\)  and wants to infer the goal from the actions, while in ours the whole state is unobserved and the adversary wants to infer some subset of the state from the actions. Perhaps closest to our work is their section assuming that actions are
unobserved–controlling \(I(s_t;g_t)\) . We can see our approach as the corresponding problem for unobserved states–controlling \(I(a_t;g_t)\) , with the additional aspect that our `goal' may in general influence the dynamics of the environment.
