Partial Variational Autoencoder.

We adapt a variational autoencoder (VAE) to synthesize the feature map of a complete image from the hidden state \(h_t\) . A VAE learns a joint distribution between the feature map \(f\)  and the latent variable \(z\)  given \(h_t\) , \(p(f,z|h_t)=p(f|z)p(z|h_t)\) . An encoder approximates the posterior \(q(z|f,h_t)\) , and a decoder infers the likelihood \(p(f|z)\) . The optimization of VAE requires calculation of \(D_{\mathrm {KL}}[q(z|f,h_t)||p(z|h_t)]\)  [1]}. As the hard attention model does not observe the complete image, it cannot estimate \(q(z|f,h_t)\) . Hence, we cannot incorporate the standard VAE directly into a hard attention framework and use the following approach.
