StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesis
Text-to-Speech (TTS) has recently seen great progress in synthesizing
high-quality speech owing to the rapid development of parallel TTS systems, but
producing speech with naturalistic prosodic variations, speaking styles and
emotional tones remains challenging. Moreover, since duration and speech are
generated separately, parallel TTS models still have problems finding the best
monotonic alignments that are crucial for naturalistic speech synthesis. Here,
we propose StyleTTS, a style-based generative model for parallel TTS that can
synthesize diverse speech with natural prosody from a reference speech
utterance. With novel Transferable Monotonic Aligner (TMA) and
duration-invariant data augmentation schemes, our method significantly
outperforms state-of-the-art models on both single and multi-speaker datasets
in subjective tests of speech naturalness and speaker similarity. Through
self-supervised learning of the speaking styles, our model can synthesize
speech with the same prosodic and emotional tone as any given reference speech
without the need for explicitly labeling these categories.