Results show that there is a correspondence between the achieved \(\mathcal {T}_{\theta ^S_0}\)  and the initialization of student as well as teacher model. They prove that the initial weights will decide or at least have an apparent effect on the final feature distribution. Analysis in [1]} also shows the importance of initialization which guesses that initial weights are closely related to their final values after training.
