The results are shown in figure 1. Trajectories from the \(u\) -constrained policy have a much higher variance in the \(u\) -direction, with correspondingly lower mutual information. Trajectories from the policy without the \(u\) -constraint have approximately equal variance in both directions. This illustrates that the use of a learned \(p_\psi (u_t|a_t)\)  discriminator is practical on this domain. We also observe that the privacy-constrained trajectories are on average further away from the center, and so receive less reward. In other words, the policy trades off reward in order to satisfy the privacy constraint. We also plot \(I(a_t;u_t)\) , first assuming a Gaussian distribution (as is used for the discriminator in training), and then using a nonparametric kernel density estimator, which doesn't assume Gaussianity. We see that our policy learns to reduce the mutual information under a Gaussian assumption to zero, but the true MI as measured by the KDE starts to increase at later timesteps, as the distribution of \(u\)  becomes more non-Gaussian. A more flexible density model for \(p(u_t|a_t)\) , such as a normalizing flow [1]}, could be used to avoid a Gaussian assumption in the density model.
