Pseudo Siamese Disentanglement Network (PSDN) is proposed for accent modeling. PSDN consists of two streams, the target stream and the auxiliary stream. The two streams have different architectures and receive different inputs during training. The detailed architecture is shown on the right side of Figure REF .
The target stream consists of a decoder, which is a stack of Conformer [1]} blocks. For the auxiliary stream, apart from an identical decoder to the target stream, an auxiliary encoder is also included.
The input of the auxiliary encoder is Mel-spectrogram before timbre augmentation. The output is first concatenated with content and timbre representations, and then fed into the auxiliary decoder. The auxiliary stream is only enabled during training.
