For the unsupervised RemixIT, we assume that the initial teacher model was pre-trained using MixIT at a specified OOD dataset. For semi-supervised RemixIT, we pre-train using the regular permutation invariant training (PIT) [1]}. We also experiment with various online teacher updating protocols such as:
\({\begin{array}{c}\mathring{{\mathbf {\theta }}}^{(K)}_{\mathcal {T}} {\mathbf {\theta }}^{(K)}_{\mathcal {S}}, \bar{{\mathbf {\theta }}}^{(k+1)}_{\mathcal {T}} \gamma {\mathbf {\theta }}^{(k)}_{\mathcal {S}} + (1 - \gamma ) \bar{{\mathbf {\theta }}}^{(k)}_{\mathcal {T}},\end{array}}\) 
