Learning probabilistic logic programming languages is receiving an increasing attention, and systems are available for learning the parameters (PRISM, LeProbLog, LFI-ProbLog and EMBLEM) or both structure and parameters (SEM-CP-logic and SLIPCASE) of these languages. In this paper we present the algorithm SLIPCOVER for “Structure LearnIng of Probabilistic logic programs by searChing OVER the clause space.” It performs a beam search in the space of probabilistic clauses and a greedy search in the space of theories using the log likelihood of the data as the guiding heuristics. To estimate the log likelihood, SLIPCOVER performs Expectation Maximization with EMBLEM. The algorithm has been tested on five real world datasets and compared with SLIPCASE, SEM-CP-logic, Aleph and two algorithms for learning Markov Logic Networks (Learning using Structural Motifs (LSM) and ALEPH++ExactL1). SLIPCOVER achieves higher areas under the precision-recall and receiver operating characteristic curves in most cases.
SEM-CP-logic @cite_46 learns parameters and structure of ground CP-logic programs. It performs learning by considering the Bayesian networks equivalent to CP-logic programs and by applying techniques for learning Bayesian networks. In particular, it applies the Structural Expectation Maximization (SEM) algorithm @cite_57 : it iteratively generates refinements of the equivalent Bayesian network and it greedily chooses the one that maximizes the BIC score @cite_5 . In SLIPCOVER, we used the LL as a score because experiments with BIC were giving inferior results. Moreover, SLIPCOVER differs from SEM-CP-logic also because it searches the clause space and it refines clauses with standard ILP refinement operators, which allow to learn non ground theories.
@cite_57 @cite_5 @cite_46
1566045017 2168175751 2129578857
In recent years there has been a flurry of works on learning Bayesian networks from data. One of the hard problems in this area is how to effectively learn the structure of a belief network from incomplete data--that is, in the presence of missing values or hidden variables. In a recent paper, I introduced an algorithm called Structural EM that combines the standard Expectation Maximization (EM) algorithm, which optimizes parameters, with structure search for model selection. That algorithm learns networks based on penalized likelihood scores, which include the BIC MDL score and various approximations to the Bayesian score. In this paper, I extend Structural EM to deal directly with Bayesian model selection. I prove the convergence of the resulting algorithm and show how to apply it for learning a large class of probabilistic models, including Bayesian networks and some variants thereof. Causal relations are present in many application domains. Causal Probabilistic Logic (CP-logic) is a probabilistic modeling language that is especially designed to express such relations. This paper investigates the learning of CP-logic theories (CP-theories) from training data. Its first contribution is SEM-CP-logic, an algorithm that learns CP-theories by leveraging Bayesian network (BN) learning techniques. SEM-CP-logic is based on a transformation between CP-theories and BNs. That is, the method applies BN learning techniques to learn a CP-theory in the form of an equivalent BN. To this end, certain modifications are required to the BN parameter learning and structure search, the most important one being that the refinement operator used by the search must guarantee that the constructed BNs represent valid CP-theories. The paper's second contribution is a theoretical and experimental comparison between CP-theory and BN learning. We show that the most simple CP-theories can be represented with BNs consisting of noisy-OR nodes, while more complex theories require close to fully connected networks (unless additional unobserved nodes are introduced in the network). Experiments in a controlled artificial domain show that in the latter cases CP-theory learning with SEM-CP-logic requires fewer training data than BN learning. We also apply SEM-CP-logic in a medical application in the context of HIV research, and show that it can compete with state-of-the-art methods in this domain.