FedNST: Federated Noisy Student Training for Automatic Speech Recognition
Federated Learning (FL) enables training state-of-the-art Automatic Speech
Recognition (ASR) models on user devices (clients) in distributed systems,
hence preventing transmission of raw user data to a central server. A key
challenge facing practical adoption of FL for ASR is obtaining ground-truth
labels on the clients. Existing approaches rely on clients to manually
transcribe their speech, which is impractical for obtaining large training
corpora. A promising alternative is using semi-/self-supervised learning
approaches to leverage unlabelled user data. To this end, we propose a new
Federated ASR method called FedNST for noisy student training of distributed
ASR models with private unlabelled user data. We explore various facets of
FedNST , such as training models with different proportions of unlabelled and
labelled data, and evaluate the proposed approach on 1173 simulated clients.
Evaluating FedNST on LibriSpeech, where 960 hours of speech data is split
equally into server (labelled) and client (unlabelled) data, showed a 22.5%
relative word error rate reduction (WERR) over a supervised baseline trained
only on server data.