But is the vaes quantification of reliability reliable? To investigate, we fit a vae with a two-dimensional latent representation to the mnist dataset [1]}, and illustrate the predicted uncertainty of \(p(\mathbf {x}| \mathbf {z})\)  in fig:teasera. The model learns to assign high uncertainty to low-level image features such as edges, but predicts its smallest values far away from the data distribution. Not only is such behavior counter-intuitive, it is also suboptimal in terms of data likelihood (Sec. REF ). Retrospectively, this should not be surprising as the uncertainty levels away from the data are governed by the extrapolatory behavior of the neural network determining \(p(\mathbf {x}| \mathbf {z})\) . This suggests that perhaps uncertainty should be a derived quantity rather than a predicted one.
