For input utterances, BNF is first extracted from a pre-trained ASR model, which is trained by Connectionist Temporal Classification (CTC) loss. To be specific, we adopt the output of the final hidden layer in the encoder of ASR model as BNF, which contain fine-grained linguistic information and some accent information.
The extracted BNF is then fed into the content encoder, which is composed of multiple Conformer blocks [1]}, to obtain the final content representation. During training, to remove the accent information in the content representation, an adversarial accent classifier is applied to the content representation to discriminate between the target accent and others. And Gradient Reversal Layer (GRL) [2]} is used to connect the content encoder and the accent classifier. The loss of content modeling is defined as:
\(F_{content} &= E_{content}(BNF)\\\hat{Y}_{accent} &= C_{accent}(GRL(F_{content}))\\L_{content} &= CE(Y_{accent}, \hat{Y}_{accent})\) 
