Deep Ensembles for Graphs with Higher-order Dependencies
Graph neural networks (GNNs) continue to achieve state-of-the-art performance
on many graph learning tasks, but rely on the assumption that a given graph is
a sufficient approximation of the true neighborhood structure. In the presence
of higher-order sequential dependencies, we show that the tendency of
traditional graph representations to underfit each node's neighborhood causes
existing GNNs to generalize poorly. To address this, we propose a novel Deep
Graph Ensemble (DGE), which captures neighborhood variance by training an
ensemble of GNNs on different neighborhood subspaces of the same node within a
higher-order network structure. We show that DGE consistently outperforms
existing GNNs on semisupervised and supervised tasks on four real-world data
sets with known higher-order dependencies, even under a similar parameter
budget. We demonstrate that learning diverse and accurate base classifiers is
central to DGE's success, and discuss the implications of these findings for
future work on GNNs.