To fully disentangle information carried by speaker vector and prosody vector, we adopt two methods. We add a gradient reversal layer (GRL) and a speaker classifier on prosody vector, following the notion of domain adversarial training. We also add an orthogonal constraint on speaker vector and prosody vector following [1]}. Both methods are used during the large-scale pre-training phase, while only the orthogonal constraint is used during
the meta learning phase and the adaptation phase.
