Following the intuition of [1]}, we use a projection head \(g(x)\)  consisting of 2 dense layers, with a ReLU non-linearity in between, which takes the output of \(f(x)\)  as input and outputs the final latent embedding. Experiments in [1]} showed improvements in the learnt representation by including such a non-linear learnable transformation. The final embedding is represented as \(z_i = g(h_i)\)  on which the contrastive loss is applied.
