We show that the classification performance of Graph Convolutional Networks is related to the alignment between features, graph and ground truth, which we quantify using a subspace alignment measure corresponding to the Frobenius norm of the matrix of pairwise chordal distances between three subspaces associated with features, graph and ground truth. The proposed measure is based on the principal angles between subspaces and has both spectral and geometrical interpretations. We showcase the relationship between the subspace alignment measure and the classification performance through the study of limiting cases of Graph Convolutional Networks as well as systematic randomizations of both features and graph structure applied to a constructive example and several examples of citation networks of different origin. The analysis also reveals the relative importance of the graph and features for classification purposes.
Following @cite_12 , we implement a two-layer GCN with propagation rule and different activation functions for each layer, i.e., a rectified linear unit for the first layer and a softmax unit for the output layer: where @math is a vector. The model then takes the simple form: where the softmax activation function is applied row-wise and the ReLU is applied element-wise. Note there is only one hidden layer with @math units. Hence @math maps the input with @math features to the hidden layer and @math maps these hidden units to the output layer with @math units, corresponding to the number of classes of the ground truth. In this semi-supervised multi-class classification, the cross-entropy error over all labeled instances is evaluated as follows: where @math is the set of nodes that have labels. The weights of the neural network ( @math and @math ) are trained using gradient descent to minimize the loss @math . A visual summary of the GCN architecture is shown in Fig. . The reader is referred to @cite_12 for details and in-depth analysis.
@cite_12
2964015378
We present a scalable approach for semi-supervised learning on graph-structured data that is based on an efficient variant of convolutional neural networks which operate directly on graphs. We motivate the choice of our convolutional architecture via a localized first-order approximation of spectral graph convolutions. Our model scales linearly in the number of graph edges and learns hidden layer representations that encode both local graph structure and features of nodes. In a number of experiments on citation networks and on a knowledge graph dataset we demonstrate that our approach outperforms related methods by a significant margin.