Probabilistic modeling enables combining domain knowledge with learning from data, thereby supporting learning from fewer training instances than purely data-driven methods. However, learning probabilistic models is difficult and has not achieved the level of performance of methods such as deep neural networks on many tasks. In this paper, we attempt to address this issue by presenting a method for learning the parameters of a probabilistic program using backpropagation. Our approach opens the possibility to building deep probabilistic programming models that are trained in a similar way to neural networks.
Edward @cite_3 is a probabilistic programming language that enables explicit representation of inference models. These inference models are implemented in Tensor Flow, enabling many of the scalability benefits of neural networks. The main difference between Edward and our approach is that Edward requires the inference model to be written explicitly whereas our approach is black box, with inference being worked out automatically. Having to specify an inference model can be an advantage, because it enables you to encode algorithms (e.g. variational methods) that couldn't be easily derived. However, it might be hard to scale to more complex models and would be particularly difficult for non-experts to use.
@cite_3
2539792571
Probabilistic modeling is a powerful approach for analyzing empirical information. We describe Edward, a library for probabilistic modeling. Edward's design reflects an iterative process pioneered by George Box: build a model of a phenomenon, make inferences about the model given data, and criticize the model's fit to the data. Edward supports a broad class of probabilistic models, efficient algorithms for inference, and many techniques for model criticism. The library builds on top of TensorFlow to support distributed training and hardware such as GPUs. Edward enables the development of complex probabilistic models and their algorithms at a massive scale.