@inproceedings{li-vu-2024-improving,
    title = "Improving Noisy Student Training for Low-resource Languages in End-to-End {ASR} Using {C}ycle{GAN} and Inter-domain Losses",
    author = "Li, Chia-Yu  and
      Vu, Ngoc Thang",
    editor = "Melero, Maite  and
      Sakti, Sakriani  and
      Soria, Claudia",
    booktitle = "Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.sigul-1.17",
    pages = "133--142",
    abstract = "Training a semi-supervised end-to-end speech recognition system using noisy student training has significantly improved performance. However, this approach requires a substantial amount of paired speech-text and unlabeled speech, which is costly for low-resource languages. Therefore, this paper considers a more extreme case of semi-supervised end-to-end automatic speech recognition where there are limited paired speech-text, unlabeled speech (less than five hours), and abundant external text. Firstly, we observe improved performance by training the model using our previous work on semi-supervised learning {``}CycleGAN and inter-domain losses{''} solely with external text. Secondly, we enhance {``}CycleGAN and inter-domain losses{''} by incorporating automatic hyperparameter tuning, calling {``}enhanced CycleGAN inter-domain losses.{''} Thirdly, we integrate it into the noisy student training approach pipeline for low-resource scenarios. Our experimental results, conducted on six non-English languages from Voxforge and Common Voice, show a 20{\%} word error rate reduction compared to the baseline teacher model and a 10{\%} word error rate reduction compared to the baseline best student model, highlighting the significant improvements achieved through our proposed method.",
}

Training a semi-supervised end-to-end speech recognition system using noisy student training has significantly improved performance. However, this approach requires a substantial amount of paired speech-text and unlabeled speech, which is costly for low-resource languages. Therefore, this paper considers a more extreme case of semi-supervised end-to-end automatic speech recognition where there are limited paired speech-text, unlabeled speech (less than five hours), and abundant external text. Firstly, we observe improved performance by training the model using our previous work on semi-supervised learning {``}CycleGAN and inter-domain losses{''} solely with external text. Secondly, we enhance {``}CycleGAN and inter-domain losses{''} by incorporating automatic hyperparameter tuning, calling {``}enhanced CycleGAN inter-domain losses.{''} Thirdly, we integrate it into the noisy student training approach pipeline for low-resource scenarios. Our experimental results, conducted on six non-English languages from Voxforge and Common Voice, show a 20{\%} word error rate reduction compared to the baseline teacher model and a 10{\%} word error rate reduction compared to the baseline best student model, highlighting the significant improvements achieved through our proposed method.
Li, Chia-Yu Vu, Ngoc Thang
Improving Noisy Student Training for Low-resource Languages in End-to-End ASR Using CycleGAN and Inter-domain Losses