
where we define \(E_{\mathbf {\theta }}(\mathbf {x},y)=\frac{1}{2\gamma ^2}||\mathbf {\phi }(\mathbf {x})-\mathbf {\mu }_y||_2^2\) , and \(Z(\mathbf {\theta })=\int \exp (-E_{\mathbf {\theta }}(\mathbf {x},y))d\mathbf {x}dy\) , which is an intractable partition function. To avoid the expense of evaluating the partition function, we follow Mnih and Teh [1]} and approximate \(Z(\mathbf {\theta })\)  as a constant (e.g., \(Z(\mathbf {\theta })=1\) ). This turns out to be an effective approximation for neural networks with lots of parameters as it encourages the model to have “self-normalized" outputs. With this approximation, the log of the joint distribution can be simplified as
\(\log p_{\mathbf {\theta }}(\mathbf {x},y)&= -\frac{1}{2\gamma ^2}||\mathbf {\phi }(\mathbf {x})-\mathbf {\mu }_y||_2^2 - \log Z(\mathbf {\mathbf {\theta }})\nonumber \\&\approx -E_{\mathbf {\theta }}(\mathbf {x},y) + \text{constant}.\) 
