Sum-product networks: A survey <s> 5) Comparison of MLE algorithms: <s> We present a unified approach for learning the parameters of Sum-Product networks (SPNs). We prove that any complete and decomposable SPN is equivalent to a mixture of trees where each tree corresponds to a product of univariate distributions. Based on the mixture model perspective, we characterize the objective function when learning SPNs based on the maximum likelihood estimation (MLE) principle and show that the optimization problem can be formulated as a signomial program. We construct two parameter learning algorithms for SPNs by using sequential monomial approximations (SMA) and the concave-convex procedure (CCCP), respectively. The two proposed methods naturally admit multiplicative updates, hence effectively avoiding the projection operation. With the help of the unified framework, we also show that, in the case of SPNs, CCCP leads to the same algorithm as Expectation Maximization (EM) despite the fact that they are different in general. <s> BIB001 </s> Sum-product networks: A survey <s> 5) Comparison of MLE algorithms: <s> It has been shown in recent work that Expectation-Maximization (EM) is an attractive way to learn the parameters of Sum-Product Networks (SPNs). However, previous derivations of EM do not easily extend to SPNs with general leaf distributions and shared parameters. In this paper we obtain a new derivation of EM for SPNs allowing to derive EM updates for \emph{arbitrary leaf distributions} and for \emph{shared parameters} between SPN nodes. Here, EM translates in a set of independent weighted maximum likelihood problems for each leaf distribution, and therefore \emph{any} distribution that can be trained with maximum likelihood (even approximately) fits in this framework. The ability of learning complex leaf distributions in a principled way makes SPN parameter learning more flexible than learning SPN weights only. We demonstrate the potential of this approach with extensive experiments on twenty widely used benchmark datasets, using tree graphical models as leaf distributions. <s> BIB002
The application of the EM to SPNs has been justified with different mathematical arguments. Peharz  exploited the interpretation of the sum nodes in the augmented network as the sum of conditional probability functions (cf. Eqs. 39 and 40). In turn, Zhao et al. BIB001 , using a unified framework based on signomial programming, designed two algorithms for learning the parameters of SPNs: sequential monomial approximations (SMA) and the concave-convex procedure (CCCP). GD is a special case of SMA, while CCCP coincides with EM in the case of SPNs, despite being different algorithms in general. Their experiments proved that EM/CCCP converges much faster than the other algorithms, including GD. In turn, Desana and Schnörr BIB002 derived the EM algorithm for SPNs whose leaf nodes may represent complex probability distributions. In discriminative learning, neither EM nor CCCP have a closed-form expression for updating the weights  . Rashwan et al.  addressed this problem with the extended Baum-Welch (EBW) algorithm, which updates the parameters of the network using a transformation that increases the value of the likelihood function monotonically. In the generative case, this transformation coincides with the update formula of EM/CCCP (the M-step), while in the discriminative case it provides a method to maximize the (conditional) likelihood function with a closed-form formula. They also adapted this method to SPNs with Gaussian leaves. Both the algorithm of Desana and Schnörr and EBW outperformed GD and EM in a wide variety of datasets.