The style encoder used in Meta-Voice is illustrated in Fig. REF .
Speaker information is represented by speaker IDs, while prosody information is extracted from mel spectrogram with a prosody mel encoder.
One-hot speaker IDs are transformed into speaker vector via a trainable speaker look-up-table (LUT), a speaker conditional layer normalization (CLN) adaptor consumes speaker vectors to generate speaker-related SALN parameters, i.e., scale \(\gamma \) 's and shift \(\beta \) 's.
The prosody mel encoder comprises of a spectral processing block, temporal processing block, a multi-head self-attention block and a final temporal averaging layer, following the design in [1]}, which convert a mel spectrogram into a fixed-length prosody vector. We then use a prosody CLN adaptor to generate prosody-related SALN parameters.
