Rapid progress has been made on the problem of automatic speech recognition (ASR) over the last few years. The main factors that contributes on this progress are the use of neural network based acoustic models [1]}, [2]}, [3]} and large scale training [4]}, [5]}, [6]}. Augmenting training data [7]}, [8]} and encoder pre-training via representation learning [9]}, [10]} have contributed to train such large-scale ASR models. Even with these improvements, performance of ASR models still degrades in the presence of significant background noise.
