In traditional KD, the student network learns from the output logits of the teacher network, adopting these as soft labels for the student's training data [1]}.
Here we penalize the discrepancy between the outputs of student vs. teacher via:
\(\mathcal {L}_{\text{pred}}=l^{\text{pred}}(f^T(x), f^S(x)),\) 
