To understand the CT representations, we visualize the representations from the pre-trained and fine-tuned CT framework in comparison to the supervised CGCNN in 2D using t-SNE[1]}. The t-SNE representation maps the embedding based on the similarity in the 2D space. The comparison between the representations of the supervised CGCNN model and the CT model for the perovskites dataset is shown in Figure REF . Each point is colored by the formation energy of perovskites which is the label that the model is trained on in the fine-tuning stage.
<FIGURE>