Recurrent Neural Network Based Language Model Adaptation for Accent Mandarin Speech
In this paper, we propose to adapt the recurrent neural network (RNN) based language model to improve the performance of multi accent Mandarin speech recognition. N gram based language model can be easily applied to speech recognition system, but it is hard to describe the long span information in a sentence and arises a serious phenomenon of data sparsity. Instead, RNN based language model can overcome these two shortcomings, but it will take a long time to decode directly. Taking these into consideration, this paper proposes a method which combines these two types of language model (LM) together and adapts the RNN based language model to rescore lattices for different accents of Mandarin speech. The architecture of the adapted RNN LM is accent specific top layers and shared hidden layer. The accent specific top layers are used to adapt different accents and the shared hidden layer stores history information, which can be seen as a memory layer. Experiments on the RASC863 corpus show that the proposed method can improve the performance of accented Mandarin speech recognition over the baseline system.
CCPR