Wang and Isola [1]} proposed to use two properties to measure the representation quality directly without the help of downstream tasks. They are 1) alignment: the closeness between positive representation pairs, and 2) uniformity: the amount of information preserved (entropy) via the induced distribution. Denote \(p_{\text{pos}}\)  as the positive data distribution and \(p_{\text{data}}\)  as the general data distribution, alignment and uniformity are mathematically defined as below based on the representation produced by a encoder \(f_{\theta }\) :
\(\begin{aligned}\mathcal {L}_{\text{align}}(f_\theta ; \alpha ) & = \mathbb {E}_{(x, y) \sim p_{\text{pos }}}\left[\Vert f_\theta (x)-f_\theta (y)\Vert _{2}^{\alpha }\right] \\\mathcal {L}_{\text{uniform }}(f_\theta ; t) & = \log \mathbb {E}_{x, y \sim p_{\text{data }}}\left[e^{-t\Vert f_\theta (x)-f_\theta (y)\Vert _{2}^{2}}\right]\end{aligned}\) 
