@inproceedings{
wang2024artspeech,
title={ArtSpeech: Adaptive Text-to-Speech Synthesis with Articulatory Representations},
author={Zhongxu Wang and Yujia Wang and Mingzhu Li and Hua Huang},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=nagiMHx4A3}
}
We devise an articulatory representation-based text-to-speech (TTS) model, ArtSpeech, an explainable and effective network for human-like speech synthesis, by revisiting the sound production system. Current deep TTS models learn acoustic-text mapping in a fully parametric manner, ignoring the explicit physical significance of articulation movement. ArtSpeech, on the contrary, leverages articulatory representations to perform adaptive TTS, clearly describing the voice tone and speaking prosody of different speakers. Specifically, energy, F0, and vocal tract variables are utilized to represent airflow forced by articulatory organs, the degree of tension in the vocal folds of the larynx, and the coordinated movements between different organs, respectively. We further designed a multi-dimensional style mapping network to extract speaking styles from diverse articulatory representations. These speaking styles will be utilized to guide the output of the articulatory variation predictors respectively, and ultimately predict the final mel spectrogram out-put. Experiment results show that, compared to other open-source zero-shot TTS systems, ArtSpeech enhances synthesis quality and greatly boosts the similarity between the generated results and the target speaker’s voice and prosody.
ArtSpeech: Adaptive Text-to-Speech Synthesis with Articulatory Representations
Zhongxu Wang Yujia Wang Mingzhu Li Hua Huang