The text encoder is the same as the one presented in [1]}, except that we use different SALN layers. We convert normalized text into phoneme sequence, which are transformed into trainable real-valued text embeddings with a text embedding layer.
A convolutional prenet, which consists of two convolutional layers and a linear layer with residual connection, convert the text embeddings to hidden representation.
Meta-Voice adopts 4 feed-forward transformer (FFT) blocks for both the text encoder and mel decoder, following model design in FastSpeech2.
The variance adaptor has similar network architecture to FastSpeech2, where phoneme-level pitch and energy features are used.
A fully-connected (FC) layer is added on the output of FFT blocks in mel decoder to output raw predicted mel spectrogram. Inspired by [2]}, we add a residual postnet to post-process raw mel spectrogram to obtain the final predicted mel spectrogram.
Sinusoidal positional encodings [3]} is added to both the text encoder and the mel decoder following common practice.
