Almost all current methods about knowledge extraction on GNNs are offline (see Fig. REF (a)), i.e., transferring knowledge from a pre-trained teacher model to a student model. In some complex tasks, however, a pre-trained teacher model is not always available for various reasons, such as training cost, privacy, etc.
Inspired by recent works that the student performance could be boosted by learning from other peers [1]}, [2]}, [3]}, as shown in Fig. REF (b), we propose a novel online knowledge distillation framework specially designed for GNNs, where one student model learns the structure information extracted from another simultaneously trained counterpart in an alternating training procedure.
