Tearing Apart NOTEARS: Controlling the Graph Prediction via Variance Manipulation
Simulations are ubiquitous in machine learning. Especially in graph learning,
simulations of Directed Acyclic Graphs (DAG) are being deployed for evaluating
new algorithms. In the literature, it was recently argued that
continuous-optimization approaches to structure discovery such as NOTEARS might
be exploiting the sortability of the variable's variances in the available data
due to their use of least square losses. Specifically, since structure
discovery is a key problem in science and beyond, we want to be invariant to
the scale being used for measuring our data (e.g. meter versus centimeter
should not affect the causal direction inferred by the algorithm). In this
work, we further strengthen this initial, negative empirical suggestion by both
proving key results in the multivariate case and corroborating with further
empirical evidence. In particular, we show that we can control the resulting
graph with our targeted variance attacks, even in the case where we can only
partially manipulate the variances of the data.