In this section, we describe the components of the CT framework (Figure REF ).
In general, SSL frameworks employ correlations in the input itself to learn robust and generalizable representations from unlabeled data. [1]}.
In the CT framework, the goal during pre-training is to force the empirical cross-correlation matrix created from the encoder embeddings two different augmentations generated by the same crystal towards the identity matrix. All the elements in the cross-correlation matrix lie between -1 and 1, with 1 representing maximum correlation. Intuitively, since the embeddings are generated from augmentations of the same crystalline system, the cross-correlation matrix must be close to the identity matrix. Using such an objective during pre-training allows the graph encoder to learn robust representations. To create the augmented instances, we use augmentation techniques, including atom masking, edge masking, and random perturbation. The embeddings for the augmented instances of the crystalline system are generated via the CGCNN graph encoder. To pre-train the CGCNN in a self-supervised manner, we use the Barlow Twins loss function. The weights of the pre-trained self-supervised model are used to initialize the graph encoder model during the fine-tuning stage for material property prediction.
