Despite the common use of mutual information in machine learning and statistics, it is in general
intractable to compute the MI between two random variables [1]}.
The reinforcement learning setting provides us with an additional challenge, as to perform efficient gradient-based
optimization we must form explicit Monte-Carlo estimators of the gradient in terms of distributions
that we are able to sample from. Two common tricks in working with mutual information are
to approximate posterior distributions with adversarial training [2]}, and to form upper bounds by introducing auxiliary
variables. We use both tricks to obtain two different gradient estimators for our objective.
