where \(z_i^t\)  is the teacher embedding vector of \(x_i\)  and \(\sigma \)  is the Gaussian kernel bandwidth.
This kind of similarity has been known to encode rich information about relations of \(x_i\)  and \(x_j\)  when the teacher embedding space is pre-trained in a fully supervised manner [1]}, but is often broken in unsupervised metric learning.
We thus supplement it with the contextual similarity, which is detailed below.
