Improving Speech Enhancement through Fine-Grained Speech Characteristics
While deep learning based speech enhancement systems have made rapid progress
in improving the quality of speech signals, they can still produce outputs that
contain artifacts and can sound unnatural. We propose a novel approach to
speech enhancement aimed at improving perceptual quality and naturalness of
enhanced signals by optimizing for key characteristics of speech. We first
identify key acoustic parameters that have been found to correlate well with
voice quality (e.g. jitter, shimmer, and spectral flux) and then propose
objective functions which are aimed at reducing the difference between clean
speech and enhanced speech with respect to these features. The full set of
acoustic features is the extended Geneva Acoustic Parameter Set (eGeMAPS),
which includes 25 different attributes associated with perception of speech.
Given the non-differentiable nature of these feature computation, we first
build differentiable estimators of the eGeMAPS and then use them to fine-tune
existing speech enhancement systems. Our approach is generic and can be applied
to any existing deep learning based enhancement systems to further improve the
enhanced speech signals. Experimental results conducted on the Deep Noise
Suppression (DNS) Challenge dataset shows that our approach can improve the
state-of-the-art deep learning based enhancement systems.