
It was shown in [1]} that speech signals inherently carry both non-linguistic information and linguistic information. The non-linguistic part refer to speaker identity, which is time-independent. While the linguistic part refer to content, which changes dramatically every several frames. On this basis, we pre-train the SDN (see Fig. REF ) to obtain the representation of speaker and content. Then the two representations obtained by the SDN, are fed as prior knowledge to adapt to the multi-speaker case, further improving the speaker generalization ability of SAFN.
