ASR Model We used the Transformer model as the base network for speech recognition. The architecture configuration is based on the wav2vec 2.0 model for encoder transfer learning [1]} and MBART50 for decoder initialization [2]}, [3]}. As mentioned before, one of the main reasons to use transfer learning, apart from achieving better performance than random initialization [4]}, [5]}, is to ensure that the output layer can contain the languages to be added without additional word embeddings. Despite the experiments being limited with the languages covered by the pretrained language models, it still remains practical thanks to the current coverage reaching nearly 200 languages [6]}.
