Efficient Transformer-based Speech Enhancement Using Long Frames and STFT Magnitudes
The SepFormer architecture shows very good results in speech separation. Like
other learned-encoder models, it uses short frames, as they have been shown to
obtain better performance in these cases. This results in a large number of
frames at the input, which is problematic; since the SepFormer is
transformer-based, its computational complexity drastically increases with
longer sequences. In this paper, we employ the SepFormer in a speech
enhancement task and show that by replacing the learned-encoder features with a
magnitude short-time Fourier transform (STFT) representation, we can use long
frames without compromising perceptual enhancement performance. We obtained
equivalent quality and intelligibility evaluation scores while reducing the
number of operations by a factor of approximately 8 for a 10-second utterance.