where \(\theta _{}\)  is generative model parameters, and \(p_{\theta _{}}(\mathbf {z})\)  is the prior distribution over latent variable \(\mathbf {z}\) , and \(p_{\theta _{}}(\mathbf {x}|\mathbf {z})\)  is the conditional distribution that generates data \(\mathbf {x}\) . Variational Auto-Encoder (VAE) [1]} models the conditional probability \(p_{\theta _{}}(\mathbf {x}|\mathbf {z})\)  with a neural network as a flexible approximator. However, introducing a complex non-linear mapping for the generation process from \(\mathbf {z}\)  to \(\mathbf {x}\)  results in intractable inference of posterior \(p_{\theta _{}}(\mathbf {z}|\mathbf {x})\) . Hence VAE uses a variational approximation \(q_{\phi }(\mathbf {z}|\mathbf {x})\)  of the posterior, i.e. the evidence lower bound (ELBO) of the marginal likelihood as written by:

\(\mathcal {L}(\theta _{},\phi ;\mathbf {x})=-D_{KL}\left(q_{\phi }(\mathbf {z}|\mathbf {x})\Vert p_{\theta _{}}(\mathbf {z})\right)+\mathbb {E}_{q_{\phi }(\mathbf {z}|\mathbf {x})}[\log p_{\theta _{}}(\mathbf {x}|\mathbf {z})]\) 
