Implementation details.
The student model of STML is optimized by AdamP with Nesterov momentum [1]} for 90 epochs on a single Titan RTX; the learning rate is initialized to \(10^{-4}\)  and scaled down by the cosine decay function [2]}.
Training images are randomly cropped to \(227 \times 227\)  and flipped horizontally at random while test images are resized to \(256 \times 256\)  then center cropped.
For all experiments, \(\sigma \)  in Eq. (REF ) is set to 3.
The margin \(\delta \) , momentum coefficient \(m\)  and the number of nearest neighbors \(k\)  are \(1.0\) , \(0.999\)  and 10 on the CUB and Cars datasets, and \(0.9\) , \(0.9\)  and 4 on the SOP dataset.
Note that \(k\)  in the nearest neighbor batch construction and that in Eq. (REF ) are identical.
