Yang et al. [1]} first tried to extract knowledge from graph neural networks. They designed a module called local structure preserving (LSP) to force the local structure in the intermediate student layer to be consistent with that of teacher model. Joshi et al. [2]} extended the LSP module to global structure preservation module and showed that can extract the teacher knowledge better. Yang et al. [3]} proposed a well-designed student model as a combination of label propagation and feature based multi-layer perceptrons to make full use of the knowledge in a pre-trained teacher model. [4]} considered to extract structural knowledge from GNNs without graph data by using the multivariate Bernoulli distribution to model the graph topology. [5]} designed a regularization to adaptively transfer knowledge within a single GNN model.
