Sum-product networks: A survey <s> D. Other applications <s> The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. Although specific domain knowledge can be used to help design representations, learning with generic priors can also be used, and the quest for AI is motivating the design of more powerful representation-learning algorithms implementing such priors. This paper reviews recent work in the area of unsupervised feature learning and deep learning, covering advances in probabilistic models, autoencoders, manifold learning, and deep networks. This motivates longer term unanswered questions about the appropriate objectives for learning good representations, for computing representations (i.e., inference), and the geometrical connections between representation learning, density estimation, and manifold learning. <s> BIB001 </s> Sum-product networks: A survey <s> D. Other applications <s> In recent years, several probabilistic techniques have been applied to various debugging problems. However, most existing probabilistic debugging systems use relatively simple statistical models, and fail to generalize across multiple programs. In this work, we propose Tractable Fault Localization Models (TFLMs) that can be learned from data, and probabilistically infer the location of the bug. While most previous statistical debugging methods generalize over many executions of a single program, TFLMs are trained on a corpus of previously seen buggy programs, and learn to identify recurring patterns of bugs. Widely-used fault localization techniques such as TARANTULA evaluate the suspiciousness of each line in isolation; in contrast, a TFLM defines a joint probability distribution over buggy indicator variables for each line. Joint distributions with rich dependency structure are often computationally intractable; TFLMs avoid this by exploiting recent developments in tractable probabilistic models (specifically, Relational SPNs). Further, TFLMs can incorporate additional sources of information, including coverage-based features such as TARANTULA. We evaluate the fault localization performance of TFLMs that include TARANTULA scores as features in the probabilistic model. Our study shows that the learned TFLMs isolate bugs more effectively than previous statistical methods or using TARANTULA directly. <s> BIB002 </s> Sum-product networks: A survey <s> D. Other applications <s> This paper describes a novel approach to study bacterial relationships in soil datasets using probabilistic graphical models. We demonstrate how to access and reformat publicly available datasets in order to apply machine learning techniques. We first learn a Bayesian network in order to read independencies in linear time between bacterial community characteristics. These independencies are useful in understanding the semantic relationships between bacteria within communities. Next, we learn a Sum-Product network in order to perform inference in linear time. Here, inference can be conducted to answer traditional queries, involving posterior probabilities, or MPE queries, requesting the most likely values of the non-evidence variables given evidence. Our results extend the literature by showing that known relationships between soil bacteria holding in one or a few datasets in fact hold across at least 3500 diverse datasets. This study paves the way for future large-scale studies of agricultural, health, and environmental applications, for which data are publicly available. <s> BIB003 </s> Sum-product networks: A survey <s> D. Other applications <s> Sum-Product Networks (SPNs) are deep tractable probabilistic models by which several kinds of inference queries can be answered exactly and in a tractable time. They have been largely used as black box density estimators, assessed by comparing their likelihood scores on different tasks. In this paper we explore and exploit the inner representations learned by SPNs. By taking a closer look at the inner workings of SPNs, we aim to better understand what and how meaningful the representations they learn are, as in a classic Representation Learning framework. We firstly propose an interpretation of SPNs as Multi-Layer Perceptrons, we then devise several criteria to extract representations from SPNs and finally we empirically evaluate them in several (semi-)supervised tasks showing they are competitive against classical feature extractors like RBMs, DBNs and deep probabilistic autoencoders, like MADEs and VAEs. <s> BIB004 </s> Sum-product networks: A survey <s> D. Other applications <s> Making sense of a dataset in an automatic and unsupervised fashion is a challenging problem in statistics and AI. Classical approaches for exploratory data analysis are usually not flexible enough to deal with the uncertainty inherent to real-world data: they are often restricted to fixed latent interaction models and homogeneous likelihoods; they are sensitive to missing, corrupt and anomalous data; moreover, their expressiveness generally comes at the price of intractable inference. As a result, supervision from statisticians is usually needed to find the right model for the data. However, since domain experts are not necessarily also experts in statistics, we propose Automatic Bayesian Density Analysis (ABDA) to make exploratory data analysis accessible at large. Specifically, ABDA allows for automatic and efficient missing value estimation, statistical data type and likelihood discovery, anomaly detection and dependency structure mining, on top of providing accurate density estimation. Extensive empirical evidence shows that ABDA is a suitable tool for automatic exploratory analysis of mixed continuous and discrete tabular data. <s> BIB005
Vergari et al.  used SPNs as autoencoders (SPAEs) for feature extraction. They trained the SPNs with Learn-SPN and used the values of the internal nodes or the states of the latent variables associated to sum nodes as the codification variables. Although this model was not trained to reconstruct its inputs, experiments showed that SPAEs are competitive with state-of-the-art autoencoder architectures for several multilabel classification problems. Butz et al. BIB003 used Bayesian networks to recognize independencies in 3,500 datasets of soil bacteria and combined them into an SPN in order to efficiently compute conditional probabilities and the MPE. Nath and Domingos BIB002 used relational SPNs (cf. Sec. VI-H) for fault localization, i.e. finding the most probable location of bugs in computer source code. The networks, trained on a corpus of previously diagnosed buggy programs, learned to identify recurring patterns of bugs. They could also receive clues about bug suspicion from other bug detectors, such as Tarantula. Hilprecht et al.  proposed learning database management systems from data instead of queries, using ensembles of relational SPNs. This approach provides better accuracy and better generalization to unseen queries than different state of the art methods. Vergari et al. BIB004 also evaluated SPNs for representation learning BIB001 . SPNs encode a hierarchy of part-based representations which can be ordered by scope length. When compared with other representation learners, such as VAEs or random Boltzmann machines (RBMs), they provided competitive results both in supervised and semisupervised settings. Moreover, the model trained for extracting representations can be used as-is for inference. Vergari et al. BIB005 designed a tool for automatic exploratory data analysis without the need for expert statistical knowledge. It leverages Gibbs sampling and a modification of mixed SPNs  to model the data, and provides functionalities such as data type recognition, missing values imputation, anomaly detection, and others. Roy et al.  addressed the explanation of activity recognition and localization in videos. A deep convolutional neural network is used for localization and its output is introduced to an SPN. Both models are learned jointly. The explainability of the system was evaluated by the subjective user trust in the explanations that the SPN provides about the criteria of the neural net.