Sum-product networks: A survey <s> VIII. Software for SPNs <s> Sum-product networks (SPNs) are a deep probabilistic representation that allows for efficient, exact inference. SPNs generalize many other tractable models, including thin junction trees, latent tree models, and many types of mixtures. Previous work on learning SPN structure has mainly focused on using top-down or bottom-up clustering to find mixtures, which capture variable interactions indirectly through implicit latent variables. In contrast, most work on learning graphical models, thin junction trees, and arithmetic circuits has focused on finding direct interactions among variables. In this paper, we present ID-SPN, a new algorithm for learning SPN structure that unifies the two approaches. In experiments on 20 benchmark datasets, we find that the combination of direct and indirect interactions leads to significantly better accuracy than several state-of-the-art algorithms for learning SPNs and other tractable models. <s> BIB001 </s> Sum-product networks: A survey <s> VIII. Software for SPNs <s> Copyright 2015 by the authors. Sum-product networks (SPNs) are a promising avenue for probabilistic modeling and have been successfully applied to various tasks. However, some theoretic properties about SPNs are not yet well understood. In this paper we fill some gaps in the theoretic foundation of SPNs. First, we show that the weights of any complete and consistent SPN can be transformed into locally normalized weights without changing the SPN distribu-tion. Second, we show that consistent SPNs cannot model distributions significantly (ex-ponentially) more compactly than decompos-able SPNs. As a third contribution, we ex-tend the inference mechanisms known for SPNs with finite states to generalized SPNs with arbitrary input distributions. <s> BIB002 </s> Sum-product networks: A survey <s> VIII. Software for SPNs <s> The Libra Toolkit is a collection of algorithms for learning and inference with discrete probabilistic models, including Bayesian networks, Markov networks, dependency networks, and sum-product networks. Compared to other toolkits, Libra places a greater emphasis on learning the structure of tractable models in which exact inference is efficient. It also includes a variety of algorithms for learning graphical models in which inference is potentially intractable, and for performing exact and approximate inference. Libra is released under a 2-clause BSD license to encourage broad use in academia and industry. <s> BIB003 </s> Sum-product networks: A survey <s> VIII. Software for SPNs <s> We introduce Graph-Structured Sum-Product Networks (GraphSPNs), a probabilistic approach to structured prediction for problems where dependencies between latent variables are expressed in terms of arbitrary, dynamic graphs. While many approaches to structured prediction place strict constraints on the interactions between inferred variables, many real-world problems can be only characterized using complex graph structures of varying size, often contaminated with noise when obtained from real data. Here, we focus on one such problem in the domain of robotics. We demonstrate how GraphSPNs can be used to bolster inference about semantic, conceptual place descriptions using noisy topological relations discovered by a robot exploring large-scale office spaces. Through experiments, we show that GraphSPNs consistently outperform the traditional approach based on undirected graphical models, successfully disambiguating information in global semantic maps built from uncertain, noisy local evidence. We further exploit the probabilistic nature of the model to infer marginal distributions over semantic descriptions of as yet unexplored places and detect spatial environment configurations that are novel and incongruent with the known evidence. <s> BIB004 </s> Sum-product networks: A survey <s> VIII. Software for SPNs <s> Sum-Product Networks (SPNs) are hierarchical, probabilistic graphical models capable of fast and exact inference that can be trained directly from high-dimensional, noisy data. Traditionally, SPNs struggle with capturing relationships in complex spatial data such as images. To this end, we introduce Deep Generalized Convolutional Sum-Product Networks (DGC-SPNs), which encode spatial features through products and sums with scopes corresponding to local receptive fields. As opposed to existing convolutional SPNs, DGC-SPNs allow for overlapping convolution patches through a novel parameterization of dilation and strides, resulting in significantly improved feature coverage and feature resolution. DGC-SPNs substantially outperform other convolutional and non-convolutional SPN approaches across several visual datasets and for both generative and discriminative tasks, including image completion and image classification. In addition, we demonstrate a modificiation to hard EM learning that further improves the generative performance of DGC-SPNs. While fully probabilistic and versatile, our model is scalable and straightforward to apply in practical applications in place of traditional deep models. Our implementation is tensorized, employs efficient GPU-accelerated optimization techniques, and is available as part of an open-source library based on TensorFlow. <s> BIB005 </s> Sum-product networks: A survey <s> VIII. Software for SPNs <s> We introduce SPFlow, an open-source Python library providing a simple interface to inference, learning and manipulation routines for deep and tractable probabilistic models called Sum-Product Networks (SPNs). The library allows one to quickly create SPNs both from data and through a domain specific language (DSL). It efficiently implements several probabilistic inference routines like computing marginals, conditionals and (approximate) most probable explanations (MPEs) along with sampling as well as utilities for serializing, plotting and structure statistics on an SPN. Moreover, many of the algorithms proposed in the literature to learn the structure and parameters of SPNs are readily available in SPFlow. Furthermore, SPFlow is extremely extensible and customizable, allowing users to promptly distill new inference and learning routines by injecting custom code into a lightweight functional-oriented API framework. This is achieved in SPFlow by keeping an internal Python representation of the graph structure that also enables practical compilation of an SPN into a TensorFlow graph, C, CUDA or FPGA custom code, significantly speeding-up computations. <s> BIB006
Every publication about SPNs presents some experiments, and in many cases the source code is publicly available. The web page https://github.com/arranger1044/ awesome-spn contains many references about SPNs, classified by year and by topic; the section "Resources" includes links to talks and tutorials, the source code for some of those publications, and several datasets commonly used for the experiments. Most of the software is written in Python or C++. In particular, there are two projects that aim to develop comprehensive, simple, and extensible libraries for SPNs. Both of them are written in Python and use TensorFlow as a backend for speeding up some operations. LibSPN, 6 initiated by Andrzej Pronobis at the University of Washington, Seattle, WA  , implements several methods for inference (marginal and conditional probabilities, and approximate MPE), parameter learning (batch and online, with GD and hard EM), and visualization of SPNs. It lacks algorithms for structural learning, but it allows building convolutional SPNs with a layer-oriented interface BIB005 . The SPNs, stored as Python structures, are compiled into Tensor-Flow graphs for parameter learning and inference; for this purpose LibSPN has implemented in C++ and CUDA some operations that cannot be performed efficiently with native TensorFlow operations. Several tutorials in Jupyter Notebook are available at its website. It has been used mainly for computer vision and robotics  , BIB004 , BIB005 . The other library, SPFlow 7 , is developed by Alejandro Molina at the University of Darmstadt, Germany, with contributors from several countries BIB006 . It implements methods for inference (marginal and conditional probabilities, and approximate MPE), parameter learning (with GD) and several structural learning algorithms, and can be extended and customized to implement new algorithms. SPNs are usually compiled into TensorFlow for fast computation, but they can also be compiled into C, CUDA, or FPGA code. There are also some smaller libraries of interest, such as SumProductNetworks.jl for Julia, BIB002 which implements inference and parameter learning, and the Libra Toolkit BIB003 , 9 a collection of algorithms written in OCaml for learning several types of probabilistic models, such as BNs, SPNs, and others, including the ID-SPN algorithm BIB001 .