
where \(\theta \in \Theta \)  parameterizes a family of functions \(T_{\theta } : \mathcal {X}\times \mathcal {Y} \rightarrow \mathbb {R}\)  through the use of a deep neural network. However, Monte Carlo sampling renders MINE a biased estimator. To avoid biased gradients, the authors in [1]} suggested to replace the partition function \(\mathbb {E}_{p_X p_Y}[e^{T_{\theta }}]\)  with an exponential moving average over mini-data-batches.
