Ablation study.
The impact of each component in STML is reported in Table REF .
The results in the table are obtained by using GoogleNet with 512 embedding dimensions.
The analysis reveals that the contextual similarity contributes most to the performance while the pairwise similarity has the minimum impact; Fig. REF  also supports this observation.
Even if both similarities are used, the performance is largely degraded when using the conventional contrastive loss [1]} instead of the relaxed version since
the conventional one quantizes our semantic similarity into a binary value, leading to significant information loss.
The momentum-based update of the teacher model and the nearest neighbor batch construction strategy also have significant impacts;
the performance degrades substantially when the teacher model is updated directly by the student counterpart or the batch construction strategy is replaced with a random sampling. The KL divergence loss has relatively little influence compared to other components.
<FIGURE><TABLE><TABLE>