On the other hand, teacher-student setups have shown significant improvements in several audio processing tasks [1]}. In [2]}, a student model was trained at the outputs of a pre-trained MixIT model for solving the problem of the artificially created input SNR mismatch between the train and test mixture distributions. An energy threshold was used to reduce the number of sources appearing in the noisy mixtures. Moreover, a student model can be adapted to a given test set using regression over the pre-trained teacher's estimates [3]}. Closest to our work is the self-training framework, proposed in [4]}, for semi-supervised singing voice separation where the teacher was pre-trained on an out-of-domain (OOD) supervised data and used for predicting estimated sources on the larger in-domain noisy dataset. The new self-labeled version of the dataset was filtered for low-quality separated sources and stored for offline training of a new student model using artificially generated mixtures from the estimated self-labeled estimated sources. Although all the aforementioned works assumed a frozen teacher, other works in automatic speech recognition have shown significant benefits when updating the teacher model using a moving mean teacher [5]}, [6]}, [7]}.
