We propose a novel score-based approach to learning a directed acyclic graph (DAG) from observational data. We adapt a recently proposed continuous constrained optimization formulation to allow for nonlinear relationships between variables using neural networks. This extension allows to model complex interactions while being more global in its search compared to other greedy approaches. In addition to comparing our method to existing continuous optimization methods, we provide missing empirical comparisons to nonlinear greedy search methods. On both synthetic and real-world data sets, this new method outperforms current continuous methods on most tasks while being competitive with existing greedy search methods on important metrics for causal inference. Structure learning and causal inference have many important applications in different areas of science such as genetics [5, 12] , biology [13] and economics [7] . Bayesian networks (BN), which encode conditional independencies using directed acyclic graphs (DAG), are powerful models which are both interpretable and computationally tractable. Causal graphical models (CGM) [12] are BNs which support interventional queries like: What will happen if someone external to the system intervene on variable X? Recent work suggests that causality could partially solve challenges faced by current machine learning systems such as robustness to out-of-distribution samples, adaptability and explainability [8, 6] . However, structure and causal learning are daunting tasks due to both the combinatorial nature of the space of structures and the question of structure identifiability [12] . Nevertheless, these graphical models known qualities and promises of improvement for machine intelligence renders the quest for structure/causal learning appealing. The problem of structure learning can be seen as an inverse problem in which the learner tries to infer the causal structure which has generated the observation. In this work, we propose a novel score-based method [5, 12] for structure learning named GraN-DAG which makes use of a recent reformulation of the original combinatorial problem of finding an optimal DAG into a continuous constrained optimization problem. In the original method named NOTEARS [18] , the directed graph is encoded as a weighted adjacency matrix W which represents coefficients in a linear structural equation model (SEM) [7] . To enforce acyclicity, the authors propose a constraint which is both efficiently computable and easily differentiable. Most popular score-based methods for DAG learning usually tackle the combinatorial nature of the problem via greedy search procedures relying on multiple heuristics [3, 2, 11] . Moving toward the continuous paradigm allows one to use gradient-based optimization algorithms instead of handdesigned greedy search algorithms. Our first contribution is to extend the work of [18] to deal with nonlinear relationships between variables using neural networks (NN) [4] . GraN-DAG is general enough to deal with a large variety of parametric families of conditional probability distributions. To adapt the acyclicity constraint to our nonlinear model, we use an argument similar to what is used in [18] and apply it first at the level of neural network paths and then at the level of graph paths. Our adapted constraint allows us to exploit the full flexibility of NNs. On both synthetic and real-world tasks, we show GraN-DAG outperforms other approaches which leverage the continuous paradigm, including DAG-GNN [16] , a recent nonlinear extension of [18] independently developed which uses an evidence lower bound as score. Our second contribution is to provide a missing empirical comparison to existing methods that support nonlinear relationships but tackle the optimization problem in its discrete form using greedy search procedures such as CAM [2] . We show that GraN-DAG is competitive on the wide range of tasks we considered. We suppose the natural phenomenon of interest can be described by a random vector X ∈ R d entailed by an underlying CGM (P X , G) where P X is a probability distribution over X and G = (V, E) is a DAG [12] . Each node i ∈ V corresponds to exactly one variable in the system. Let π G i denote the set of parents of node i in G and let X π G i denote the random vector containing the variables corresponding to the parents of i in G. We assume there are no hidden variables. In a CGM, the distribution P X is said to be Markov to G which means we can write the probability density function (pdf) as p( . A CGM can be thought of as a BN in which directed edges are given a causal meaning, allowing it to answer queries regarding interventional distributions [5] .
We are proposing a new score-based approach to structure/causal learning leveraging neural networks and a recent continuous constrained formulation to this problem
{"src": "scitldr-AIC_train", "paper_id": "ryl6nX398r"}