On Comparison of Encoders for Attention based End to End Speech Recognition in Standalone and Rescoring Mode
The streaming automatic speech recognition (ASR) models are more popular and
suitable for voice-based applications. However, non-streaming models provide
better performance as they look at the entire audio context. To leverage the
benefits of the non-streaming model in streaming applications like voice
search, it is commonly used in second pass re-scoring mode. The candidate
hypothesis generated using steaming models is re-scored using a non-streaming
model. In this work, we evaluate the non-streaming attention-based end-to-end
ASR models on the Flipkart voice search task in both standalone and re-scoring
modes. These models are based on Listen-Attend-Spell (LAS) encoder-decoder
architecture. We experiment with different encoder variations based on LSTM,
Transformer, and Conformer. We compare the latency requirements of these models
along with their performance. Overall we show that the Transformer model offers
acceptable WER with the lowest latency requirements. We report a relative WER
improvement of around 16% with the second pass LAS re-scoring with latency
overhead under 5ms. We also highlight the importance of CNN front-end with
Transformer architecture to achieve comparable word error rates (WER).
Moreover, we observe that in the second pass re-scoring mode all the encoders
provide similar benefits whereas the difference in performance is prominent in
standalone text generation mode.