
where \(q_{X,\phi }(\mathbf {x}|\mathbf {y})\)  is a valid conditional distribution, parameterized by \(\phi \) . Instead, the discriminative approach attempts to directly estimate the density ratio (see (REF ) in the following). It usually exploits an energy-based variational family of functions to provide a lower bound on the Kullback-Leibler (KL) divergence. As an example, the Donsker-Varadhan dual representation of the KL divergence [1]} can be derived from (REF ) [2]} to produce the bound optimized in MINE [3]}
\(I(X;Y) \ge & I_{MINE}(X;Y) \nonumber \\= & \sup _{\theta \in \Theta } \mathbb {E}_{(\mathbf {x},\mathbf {y})\sim p_{XY}(\mathbf {x},\mathbf {y})}[T_{\theta }(\mathbf {x},\mathbf {y})] \nonumber \\& - \log (\mathbb {E}_{(\mathbf {x},\mathbf {y})\sim p_X(\mathbf {x}) p_Y(\mathbf {y})}[e^{T_{\theta }(\mathbf {x},\mathbf {y})}]),\) 
