Let Invariant Rationale Discovery Inspire Graph Contrastive Learning
Leading graph contrastive learning (GCL) methods perform graph augmentations
in two fashions: (1) randomly corrupting the anchor graph, which could cause
the loss of semantic information, or (2) using domain knowledge to maintain
salient features, which undermines the generalization to other domains. Taking
an invariance look at GCL, we argue that a high-performing augmentation should
preserve the salient semantics of anchor graphs regarding
instance-discrimination. To this end, we relate GCL with invariant rationale
discovery, and propose a new framework, Rationale-aware Graph Contrastive
Learning (RGCL). Specifically, without supervision signals, RGCL uses a
rationale generator to reveal salient features about graph
instance-discrimination as the rationale, and then creates rationale-aware
views for contrastive learning. This rationale-aware pre-training scheme endows
the backbone model with the powerful representation ability, further
facilitating the fine-tuning on downstream tasks. On MNIST-Superpixel and MUTAG
datasets, visual inspections on the discovered rationales showcase that the
rationale generator successfully captures the salient features (i.e.
distinguishing semantic nodes in graphs). On biochemical molecule and social
network benchmark datasets, the state-of-the-art performance of RGCL
demonstrates the effectiveness of rationale-aware views for contrastive
learning. Our codes are available at https://github.com/lsh0520/RGCL.