In this thesis, we propose an artificial auditory system that gives a robot the ability to locate and track sounds, as well as to separate simultaneous sound sources and recognising simultaneous speech. We demonstrate that it is possible to implement these capabilities using an array of microphones, without trying to imitate the human auditory system. The sound source localisation and tracking algorithm uses a steered beamformer to locate sources, which are then tracked using a multi-source particle filter. Separation of simultaneous sound sources is achieved using a variant of the Geometric Source Separation (GSS) algorithm, combined with a multi-source post-filter that further reduces noise, interference and reverberation. Speech recognition is performed on separated sources, either directly or by using Missing Feature Theory (MFT) to estimate the reliability of the speech features. The results obtained show that it is possible to track up to four simultaneous sound sources, even in noisy and reverberant environments. Real-time control of the robot following a sound source is also demonstrated. The sound source separation approach we propose is able to achieve a 13.7 dB improvement in signal-to-noise ratio compared to a single microphone when three speakers are present. In these conditions, the system demonstrates more than 80 accuracy on digit recognition, higher than most human listeners could obtain in our small case study when recognising only one of these sources. All these new capabilities will allow humans to interact more naturally with a mobile robot in real life settings.
The beamforming technique most widely used today is the Generalised Sidelobe Canceller (GSC), originally proposed by Griffiths and Jim @cite_72 . The GSC algorithm uses a fixed beamformer (delay and sum) to produce an initial estimation of the source of interest. Also, a blocking matrix is used to produce noise reference signals (that do not contain the source of interest) than can be used by a multiple-input canceller to further reduce noise at the output of the fixed beamformer. The GSC algorithm can be implemented in the frequency domain @cite_98 where its components are matrices, or in the time domain @cite_0 where the components are adaptive filters.
@cite_98 @cite_72 @cite_0
2099984896 2147665979 2586584460 2061156132
This paper proposes a new robust adaptive beamformer applicable to microphone arrays. The proposed beamformer is a generalized sidelobe canceller (GSC) with a new adaptive blocking matrix using coefficient-constrained adaptive filters (CCAFs) and a multiple-input canceller with norm-constrained adaptive filters (NCAFs). The CCAFs minimize leakage of the target-signal into the interference path of the GSC. Each coefficient of the CCAFs is constrained to avoid mistracking. The input signal to all the CCAFs is the output of a fixed beamformer. In the multiple-input canceller, the NCAFs prevent undesirable target-signal cancellation when the target-signal minimization at the blocking matrix is incomplete. The proposed beamformer is shown to be robust to target-direction errors as large as 200 with almost no degradation in interference-reduction performance, and it can be implemented with several microphones. The maximum allowable target-direction error can be specified by the user. Simulated anechoic experiments demonstrate that the proposed beamformer cancels interference by over 30 dB. Simulation with real acoustic data captured in a room with 0.3-s reverberation time shows that the noise is suppressed by 19 dB. In subjective evaluation, the proposed beamformer obtains 3.8 on a five-point mean opinion score scale, which is 1.0 point higher than the conventional robust beamformer. We consider a sensor array located in an enclosure, where arbitrary transfer functions (TFs) relate the source signal and the sensors. The array is used for enhancing a signal contaminated by interference. Constrained minimum power adaptive beamforming, which has been suggested by Frost (1972) and, in particular, the generalized sidelobe canceler (GSC) version, which has been developed by Griffiths and Jim (1982), are the most widely used beamforming techniques. These methods rely on the assumption that the received signals are simple delayed versions of the source signal. The good interference suppression attained under this assumption is severely impaired in complicated acoustic environments, where arbitrary TFs may be encountered. In this paper, we consider the arbitrary TF case. We propose a GSC solution, which is adapted to the general TF case. We derive a suboptimal algorithm that can be implemented by estimating the TFs ratios, instead of estimating the TFs. The TF ratios are estimated by exploiting the nonstationarity characteristics of the desired signal. The algorithm is applied to the problem of speech enhancement in a reverberating room. The discussion is supported by an experimental study using speech and noise signals recorded in an actual room acoustics environment. This paper considers acoustic beamforming for noise robust automatic speech recognition. A beamformer attenuates background noise by enhancing sound components coming from a direction specified by a steering vector. Hence, accurate steering vector estimation is paramount for successful noise reduction. Recently, time-frequency masking has been proposed to estimate the steering vectors that are used for a beamformer. In particular, we have developed a new form of this approach, which uses a speech spectral model based on a complex Gaussian mixture model CGMM to estimate the time-frequency masks needed for steering vector estimation, and extended the CGMM-based beamformer to an online speech enhancement scenario. Our previous experiments showed that the proposed CGMM-based approach outperforms a recently proposed mask estimator based on a Watson mixture model and the baseline speech enhancement system of the CHiME-3 challenge. This paper provides additional experimental results for our online processing, which achieves performance comparable to that of batch processing with a suitable block-batch size. This online version reduces the CHiME-3 word error rate WER on the evaluation set from 8.37 to 8.06 . Moreover, in this paper, we introduce a probabilistic prior distribution for a spatial correlation matrix a CGMM parameter, which enables more stable steering vector estimation in the presence of interfering speakers. In practice, the performance of the proposed online beamformer degrades with observations that contain only noise or and interference because of the failure of the CGMM parameter estimation. The introduced spatial prior enables the target speaker's parameter to avoid overfitting to noise or and interference. Experimental results show that the spatial prior reduces the WER from 38.4 to 29.2 in a conversation recognition task compared with the CGMM-based approach without the prior, and outperforms a conventional online speech enhancement approach. Currently, the nonadaptive delay-and-sum (DAS) beamformer is used in medical ultrasound imaging. However, due to its data-independent nature, DAS leads to images with limited resolution and contrast. In this paper, an adaptive minimum variance (MV)-based beamformer that combines the MV and coherence factor (CF) weighting is introduced and adapted to medical ultrasound imaging. MV-adaptive beamformers can improve the image quality in terms of resolution and sidelobes by suppressing off-axis signals, while keeping onaxis ones. In addition, CF weighting can improve contrast and sidelobes by emphasizing the in-phase signals and reducing the out-of-phase ones. Combining MV and CF weighting results in simultaneous improvement of imaging resolution and contrast, outperforming both DAS and MV beamformers. In addition, because of the power of CF in reducing the focusing errors, the proposed method presents satisfactory robustness against sound velocity inhomogeneities, outperforming the regularized MV beamformer. The excellent performance of the proposed beamforming approach is demonstrated by several simulated examples.