FeaRLESS: Feature Refinement Loss for Ensembling Self-Supervised Learning Features in Robust End-to-end Speech Recognition
Self-supervised learning representations (SSLR) have resulted in robust
features for downstream tasks in many fields. Recently, several SSLRs have
shown promising results on automatic speech recognition (ASR) benchmark
corpora. However, previous studies have only shown performance for solitary
SSLRs as an input feature for ASR models. In this study, we propose to
investigate the effectiveness of diverse SSLR combinations using various fusion
methods within end-to-end (E2E) ASR models. In addition, we will show there are
correlations between these extracted SSLRs. As such, we further propose a
feature refinement loss for decorrelation to efficiently combine the set of
input features. For evaluation, we show that the proposed 'FeaRLESS learning
features' perform better than systems without the proposed feature refinement
loss for both the WSJ and Fearless Steps Challenge (FSC) corpora.