Temporal and Spectral Processing Methods for Processing of Degraded Speech: A Review <s> Spectral Processing Methods <s> The transmission between a sound source and a microphone in a small hard‐walled room is characterized by multiple peaks and sharp valleys. Sound received by such a microphone often is judged to be undesirably distorted or spectrally colored. A technique is described for combining the signals from two or more microphones to produce a single output having less multipath distortion than any of the individual microphone signals. The output from each microphone is filtered to separate it into a number of bandpass signals occupying contiguous frequency bands. The microphone with greatest long‐time response in a given frequency band contributes that band to the output. Each microphone, then, is used only for those frequency bands in which its response is greater than any other microphone, so that the combined output contains less spectral coloration than the signal from any individual microphone. The performance of the process was studied by simulation on a digital computer, and sample recordings are included. <s> BIB001 </s> Temporal and Spectral Processing Methods for Processing of Degraded Speech: A Review <s> Spectral Processing Methods <s> It is well known that room reverberation can significantly impair one’s perception of sounds recorded by a microphone in that room. Acoustic recordings produced in untreated rooms are characterized by a hollow echolike quality resulting from not locating the microphone close to the source. In this paper we discuss a multimicrophone digital processing scheme for removing much of the degrading distortion. To accomplish this the individual microphone signals are divided into frequency bands whose corresponding outputs are cophased (delay differences are compensated) and added. Then the gain of each resulting band is set based on the cross correlation between corresponding microphone signals in that band. The reconstructed broadband speech is perceived with considerably reduced reverberation. <s> BIB002 </s> Temporal and Spectral Processing Methods for Processing of Degraded Speech: A Review <s> Spectral Processing Methods <s> The paper presents a new method for dereverberation of speech signals with a single microphone. For applications such as speech recognition, reverberant speech causes serious problems when a distant microphone is used in recording. This is especially severe when the reverberation time exceeds 0.5 s. We propose a method which uses the fundamental frequency (F/sub 0/) of the target speech as the primary feature for dereverberation. This method initially estimates F/sub 0/ and the harmonic structure of the speech signal and then obtains a dereverberation operator. This operator transforms the reverberant signal to its direct signal based on an inverse filtering operation. Dereverberation is achieved without prior knowledge of either the room acoustics or the target speech. Experimental results show that the dereverberation operator estimated from 5240 Japanese word utterances could effectively reduce the reverberation when the reverberation time is longer than 0.1 s. <s> BIB003 </s> Temporal and Spectral Processing Methods for Processing of Degraded Speech: A Review <s> Spectral Processing Methods <s> In speech communication systems, such as voice-controlled systems, hands-free mobile telephones, and hearing aids, the received microphone signals are degraded by room reverberation, background noise, and other interferences. This signal degradation may lead to total unintelligibility of the speech and decreases the performance of automatic speech recognition systems. In the context of this work reverberation is the process of multi-path propagation of an acoustic sound from its source to one or more microphones. The received microphone signal generally consists of a direct sound, reflections that arrive shortly after the direct sound (commonly called early reverberation), and reflections that arrive after the early reverberation (commonly called late reverberation). Reverberant speech can be described as sounding distant with noticeable echo and colouration. These detrimental perceptual effects are primarily caused by late reverberation, and generally increase with increasing distance between the source and microphone. Conversely, early reverberations tend to improve the intelligibility of speech. In combination with the direct sound it is sometimes referred to as the early speech component. Reduction of the detrimental effects of reflections is evidently of considerable practical importance, and is the focus of this dissertation. More specifically the dissertation deals with dereverberation techniques, i.e., signal processing techniques to reduce the detrimental effects of reflections. In the dissertation, novel single- and multimicrophone speech dereverberation algorithms are developed that aim at the suppression of late reverberation, i.e., at estimation of the early speech component. This is done via so-called spectral enhancement techniques that require a specific measure of the late reverberant signal. This measure, called spectral variance, can be estimated directly from the received (possibly noisy) reverberant signal(s) using a statistical reverberation model and a limited amount of a priori knowledge about the acoustic channel(s) between the source and the microphone(s). In our work an existing single-channel statistical reverberation model serves as a starting point. The model is characterized by one parameter that depends on the acoustic characteristics of the environment. We show that the spectral variance estimator that is based on this model, can only be used when the source-microphone distance is larger than the so-called critical distance. This is, crudely speaking, the distance where the direct sound power is equal to the total reflective power. A generalization of the statistical reverberation model in which the direct sound is incorporated is developed. This model requires one additional parameter that is related to the ratio between the direct sound energy and the sound energy of all reflections. The generalized model is used to derive a novel spectral variance estimator. When the novel estimator is used for dereverberation rather than the existing estimator, and the source-microphone distance is smaller than the critical distance, the dereverberation performance is significantly increased. Single-microphone systems only exploit the temporal and spectral diversity of the received signal. Reverberation, of course, also induces spatial diversity. To additionally exploit this diversity, multiple microphones must be used, and their outputs must be combined by a suitable spatial processor such as the so-called delay and sum beamformer. It is not a priori evident whether spectral enhancement is best done before or after the spatial processor. For this reason we investigate both possibilities, as well as a merge of the spatial processor and the spectral enhancement technique. An advantage of the latter option is that the spectral variance estimator can be further improved. Our experiments show that the use of multiple microphones affords a significant improvement of the perceptual speech quality. The applicability of the theory developed in this dissertation is demonstrated using a hands-free communication system. Since hands-free systems are often used in a noisy and reverberant environment, the received microphone signal does not only contain the desired signal but also interferences such as room reverberation that is caused by the desired source, background noise, and a far-end echo signal that results from a sound that is produced by the loudspeaker. Usually an acoustic echo canceller is used to cancel the far-end echo. Additionally a post-processor is used to suppress background noise and residual echo, i.e., echo which could not be cancelled by the echo canceller. In this work a novel structure and post-processor for an acoustic echo canceller are developed. The post-processor suppresses late reverberation caused by the desired source, residual echo, and background noise. The late reverberation and late residual echo are estimated using the generalized statistical reverberation model. Experimental results convincingly demonstrate the benefits of the proposed system for suppressing late reverberation, residual echo and background noise. The proposed structure and post-processor have a low computational complexity, a highly modular structure, can be seamlessly integrated into existing hands-free communication systems, and affords a significant increase of the listening comfort and speech intelligibility. <s> BIB004
Spectral processing methods are the most popular techniques for noise reduction, mainly because of their simplicity and effectiveness. Most of the spectral processing techniques rely on the basis that the human speech perception is not sensitive to short-time phase  . This is exploited in these methods, where only the spectral magnitude associated with the original signal is estimated. In case of noisy speech, the spectral processing methods can be grouped into nonparametric and statistical model-based methods  . Methods from the first category remove an estimate of the degradation from the noisy features, such as subtractive type algorithms and wavelet denoising. The statistical model based method such as MMSE estimator uses the parametric model of the signal generation process. The spectral enhancement methods achieve dereverberation by modifying the short-time magnitude spectrum of the reverberant speech. Initially, Flanagan et al. BIB001 proposed a spectral based two microphone approach for processing reverberant speech. The speech signal from each microphone was separated into several subbands. In each frequency band, the spectral amplitudes of the two signals were compared and the maximum amplitude was selected as the contribution for the reconstructed speech. This method exploits the periodic nature of the spectral distortion of speech caused by simple echo. The two microphones spaced at different locations have echoes of different delays and nulls appear at regular but different intervals in the spectra of the microphone outputs. An algorithm proposed by Allen et al. BIB002 first filters the two individual microphone signals into the frequency band and then the filtered outputs are compensated for the delay differences and added. For each band, the correlation between the two microphone signals is computed. These correlations are used as a gain factor for that band, to suppress the spectral bands with low correlations, in order to remove the reverberation effects. This is done based on the assumption that bands with high levels of coherence contain a strong direct component, whereas bands with low levels of coherence mainly contain reverberation. Another form of spectral processing method is proposed in BIB003 , using the harmonic structure of speech, based on pitch estimation. In the proposed method, the harmonic part of the speech was extracted by adaptive filtering. Averaging the ratio of DFT of the harmonic part of speech and that of the reverberant part, a dereverberation filter was calculated, which reduced reverberation in both voiced and unvoiced speech segments. The technique is suitable when sufficient number of training utterances are available and the room impulse response does not change significantly. Recently, a spectral subtraction based spectral processing technique has been developed, to suppress late reverberation effect BIB004  . In spectral subtraction based methods, the impulse response of the room h(n) can be split as shown below:  where N i is the length of the impulse response and N 1 is the threshold, which is chosen such that h a (n) consists of the direct signal and a few early reflections, and h l (n) consists of all late reflections. The threshold N 1 can be chosen, depending on the application or subjective preference BIB004 . The spectral subtraction method is developed based on the fact that the early and late impulse components are approximately uncorrelated in the time domain. Therefore, the late reverberant signal can be treated as an additive noise, and thus can be eliminated through spectral subtraction  . Accordingly, the short-time power spectral density (PSD) of the reverberant speech signal S zz (l, k) is expressed as  : where S za (l, k) and S zl (l, k) are the PSD of the early and late reverberant components, respectively. Indices l and k refer to the time frame and frequency bin, respectively. Lebart et al.  introduced a single channel speech dereverberation method, based on spectral subtraction, to reduce this effect. The method estimates the power spectrum of the reverberation, based on a statistical model of late reverberation, and then subtracts it from the power spectrum of the reverberant speech. In  , the authors assumed that reverberation time is frequency independent and that the energy related to the direct sound could be ignored. Hence, the authors assume that the signal to reverberation ratio (SRR) of the observed signal is smaller than 0 dB, which limits the use of the proposed solution to a situation in which the source-microphone distance is larger than the critical distance. This issue has been addressed in BIB004 , where the room impulse response model was generalized by considering the direct component and the reflections separately. A novel estimator was derived, which had an advantage over the late reverberant power spectral density estimator proposed by Lebart, if the source-microphone distance was smaller than the critical distance BIB004 . One of the main problems in spectral subtraction is the nonlinear processing distortion, for example, the musical noise caused by over-subtraction of the reverberation. This distortion degrades the quality of the processed speech. However, there are some well known methods like spectral floor factor, a priori SRR estimation techniques available to reduce this nonlinear distortion to a certain level. 