Embedding networks.
For comparisons to previous work, we employ GoogleNet [1]} and Inception-BN [2]} as our teacher and student embedding model architecture.
The dimension of embedding layers \(g_t\)  and \(g_s\)  are set as the output size of the last pooling, and that of \(f_s\)  follows each experimental setting.
We append an \(l_2\)  normalization layer on top of the teacher so that the Euclidean distances in Eq. (REF ) are bounded for computing semantic similarities stably.
