Mix and Match: An Empirical Study on Training Corpus Composition for Polyglot Text-To-Speech (TTS)
Training multilingual Neural Text-To-Speech (NTTS) models using only
monolingual corpora has emerged as a popular way for building voice cloning
based Polyglot NTTS systems. In order to train these models, it is essential to
understand how the composition of the training corpora affects the quality of
multilingual speech synthesis. In this context, it is common to hear questions
such as "Would including more Spanish data help my Italian synthesis, given the
closeness of both languages?". Unfortunately, we found existing literature on
the topic lacking in completeness in this regard. In the present work, we
conduct an extensive ablation study aimed at understanding how various factors
of the training corpora, such as language family affiliation, gender
composition, and the number of speakers, contribute to the quality of Polyglot
synthesis. Our findings include the observation that female speaker data are
preferred in most scenarios, and that it is not always beneficial to have more
speakers from the target language variant in the training corpus. The findings
herein are informative for the process of data procurement and corpora
building.