The above intuition is presented by Tian, et al. [1]} as a proposition, which is defined and illustrated mathematically using the concept of mutual information. Based on this proposition, they studied how different choices of the augmented views affect the quality of the learned representation. They concluded that the current contrastive learning is powerful because they reduce the mutual information (MI) between views while keeping task-relevant information intact. Moreover, they proposed a semi-supervised method to learn effective views for a known given task while keeping the SSL process unsupervised.
