COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning
Graph contrastive learning (GCL) improves graph representation learning,
leading to SOTA on various downstream tasks. The graph augmentation step is a
vital but scarcely studied step of GCL. In this paper, we show that the node
embedding obtained via the graph augmentations is highly biased, somewhat
limiting contrastive models from learning discriminative features for
downstream tasks. Thus, instead of investigating graph augmentation in the
input space, we alternatively propose to perform augmentations on the hidden
features (feature augmentation). Inspired by so-called matrix sketching, we
propose COSTA, a novel COvariance-preServing feaTure space Augmentation
framework for GCL, which generates augmented features by maintaining a "good
sketch" of original features. To highlight the superiority of feature
augmentation with COSTA, we investigate a single-view setting (in addition to
multi-view one) which conserves memory and computations. We show that the
feature augmentation with COSTA achieves comparable/better results than graph
augmentation based models.