Sum-product networks: A survey <s> G. Learning with dynamic data <s> In this paper, we describe a model of causal reasoning that accounts for knowledge concerning cause-and-effect relationships and knowledge concerning the tendency for propositions to persist or not as a function of time passing. Our model has a natural encoding in the form of a network representation for probabilistic models. We explore the computational properties of our model by considering recent advances in computing the consequences of models encoded in this network representation. <s> BIB001 </s> Sum-product networks: A survey <s> G. Learning with dynamic data <s> Sum-product networks have recently emerged as an attractive representation due to their dual view as a special type of deep neural network with clear semantics and a special type of probabilistic graphical model for which inference is always tractable. Those properties follow from some conditions (i.e., completeness and decomposability) that must be respected by the structure of the network. As a result, it is not easy to specify a valid sum-product network by hand and therefore structure learning techniques are typically used in practice. This paper describes a new online structure learning technique for feed-forward and recurrent SPNs. The algorithm is demonstrated on real-world datasets with continuous features for which it is not clear what network architecture might be best, including sequence datasets of varying length. <s> BIB002
Data are said to be dynamic when all the variables (or at least some of them) have different values in different time points-for example, Income-at-year-1, Income-atyear-2, etc. The set of variables for a specific time point is usually called a slice. The slice structure, called template, is replicated and chained to accommodate as many time points as necessary. The length of the chain is called the horizon. For this problem Melibari et al.  proposed dynamic SPNs (DSPNs), which extend SPNs in the same way that dynamic Bayesian networks (DBNs) BIB001 extend BNs. A local-search structure learner generates an initial template SPN and searches for neighboring structures, trying to maximize the likelihood. Every neighbor, which comes from replacing a product node, represents a specific choice of factorization of the variables in its scope. The algorithm searches over other choices of factorizations and updates the structure if a better one is found. This method outperforms non-dynamic algorithms, such as LearnSPN, and other models, such as dynamic Bayesian networks and recurrent neural networks. Later, Kalra et al. BIB002 extended oSLRAU to the dynamic setting by unrolling the SPN to match the length of the chain to the horizon, with shared weights and a shared covariance matrix, to decide when a new correlation requires a change in the template. This algorithm surpassed that of Melibari et al.  and hidden Markov models in 5 sequential datasets, and recurrent neural networks in 4 of those datasets.