Sum-product networks: A survey <s> D. Other algorithms <s> We consider the selectivity constraint on the structure of sum-product networks (SPNs), which allows each sum node to have at most one child with non-zero output for each possible input. This allows us to find globally optimal maximum likelihood parameters in closed form. Although being a constrained class of SPNs, these models still strictly generalize classical graphical models such as Bayesian networks. Closed form parameter estimation opens the door for structure learning using a principled scoring function, trading off training likelihood and model complexity. In experiments we show that these models are easy to learn and compete well with state of the art. <s> BIB001 </s> Sum-product networks: A survey <s> D. Other algorithms <s> Sum-product networks (SPNs) are a recently developed class of deep probabilistic models where inference is tractable. We present two new structure learning algorithms for sum-product networks, in the generative and discriminative settings, that are based on recursively extracting rank-one submatrices from data. The proposed algorithms find the subSPNs that are the most coherent jointly in the instances and variables - that is, whose instances are most strongly correlated over the given variables. ::: ::: Experimental results show that SPNs learned using the proposed generative algorithm have better likelihood and inference results - and also much faster - than previous approaches. Finally, we apply the discriminative SPN structure learning algorithm to handwritten digit recognition tasks, where it achieves state-of-the-art performance for an SPN. <s> BIB002
Peharz et al. BIB001 proposed a structure learner that searches within the space of selective SPNs and showed that it is competitive with LearnSPN. Adel et al. BIB002 pointed out that previous work had only compared algorithms on binary datasets. They designed SVD-SPN, which proceeds by finding rank-1 matrices. This allows the algorithm to cluster and split at the same time, producing optimal data matrix pieces. It operates recursively, like LearnSPN, but constructs the SPN from the rank-1 submatrices extracted. It also considers a multivariate base case when the variables in the pieces of the data matrix are highly correlated. In this case a sum node is created with as many children as instances in the piece of the matrix; each child is a product node of all the variables in the matrix. In their experiments, SVD-SPN obtained results similar to those of LearnSPN and ID-SPN for binary datasets, but outperformed them in multiple-category datasets, such as Caltech-101, and is 5 times faster.