In this thesis, we propose an artificial auditory system that gives a robot the ability to locate and track sounds, as well as to separate simultaneous sound sources and recognising simultaneous speech. We demonstrate that it is possible to implement these capabilities using an array of microphones, without trying to imitate the human auditory system. The sound source localisation and tracking algorithm uses a steered beamformer to locate sources, which are then tracked using a multi-source particle filter. Separation of simultaneous sound sources is achieved using a variant of the Geometric Source Separation (GSS) algorithm, combined with a multi-source post-filter that further reduces noise, interference and reverberation. Speech recognition is performed on separated sources, either directly or by using Missing Feature Theory (MFT) to estimate the reliability of the speech features. The results obtained show that it is possible to track up to four simultaneous sound sources, even in noisy and reverberant environments. Real-time control of the robot following a sound source is also demonstrated. The sound source separation approach we propose is able to achieve a 13.7 dB improvement in signal-to-noise ratio compared to a single microphone when three speakers are present. In these conditions, the system demonstrates more than 80 accuracy on digit recognition, higher than most human listeners could obtain in our small case study when recognising only one of these sources. All these new capabilities will allow humans to interact more naturally with a mobile robot in real life settings.
More recently, approaches using more than two microphones have been developed. One approach uses a circular array of eight microphones to locate sound sources using the MUSIC algorithm @cite_101 , a signal subspace approach. In our previous work also using eight microphones @cite_17 , we presented a method for localising a single sound source where time delay of arrival (TDOA) estimation was separated from the direction of arrival (DOA) estimation. It was found that a system combining TDOA and DOA estimation in a single step improves the system's robustness, while allowing localisation (but not tracking) of simultaneous sources @cite_100 . Kagami @cite_19 reports a system using 128 microphones for 2D sound localisation of sound sources. Similarly, Wang @cite_5 use 24 fixed microphones to track a moving robot in a room. However, it would not be practical to include such a large number of microphones on a mobile robot.
@cite_101 @cite_19 @cite_5 @cite_100 @cite_17
2156514726 2139129402 2148613904 2012211082
This work describes two circular microphone arrays and a square microphone array which can be used for sound localization and sound capture. Sound capture by microphone array is achieved by sum and delay beam former (SDBF). A dedicated PCI 128-channel simultaneous input analog-to-digital (AD) board is developed for a 128 ch microphone array with a maximum sampling rate of 22. 7 spl mu s sample. Simulation of sound pressure distribution of 24 and 128 ch circular microphone array and 128 ch square microphone array are shown. Then a 24 ch circular microphone array and a 128 ch square microphone array have been developed. The 24 ch circular microphone array can capture sound from an arbitrary direction. The 128 ch square microphone array can capture sound from a specific point. Both systems are evaluated by using frequency components of the sound. The circular type system can be used on a mobile robot including humanoid robot and square type can be extended towards room coverage type application. Mobile robots in real-life settings would benefit from being able to localize sound sources. Such a capability can nicely complement vision to help localize a person or an interesting event in the environment, and also to provide enhanced processing for other capabilities such as speech recognition. We present a robust sound source localization method in three-dimensional space using an array of 8 microphones. The method is based on a frequency-domain implementation of a steered beamformer along with a probabilistic post-processor. Results show that a mobile robot can localize in real time multiple moving sources of different types over a range of 5 meters with a response time of 200 ms. When performing speaker diarization on recordings from meetings, multiple microphones of different qualities are usually available and distributed around the meeting room. Although several approaches have been proposed in recent years to take advantage of multiple microphones, they are either too computationally expensive and not easily scalable or they cannot outperform the simpler case of using the best single microphone. In this paper, the use of classic acoustic beamforming techniques is proposed together with several novel algorithms to create a complete frontend for speaker diarization in the meeting room domain. New techniques we are presenting include blind reference-channel selection, two-step time delay of arrival (TDOA) Viterbi postprocessing, and a dynamic output signal weighting algorithm, together with using such TDOA values in the diarization to complement the acoustic information. Tests on speaker diarization show a 25 relative improvement on the test set compared to using a single most centrally located microphone. Additional experimental results show improvements using these techniques in a speech recognition task. Abstract This paper presents a method for the navigation of a mobile robot using sound localization in the context of a robotic lab tour guide. Sound localization, which is achieved using an array of 24 microphones distributed on two walls of the lab, is performed whenever the robot speaks as part of the tour. The SRP-PHAT sound localization algorithm is used to estimate the current location of the robot using approximately 2 s of recorded signal. Navigation is achieved using several stops during which the estimated location of the robot is used to make course adjustments. Experiments using the acoustic robot navigation system illustrate the accuracy of the proposed technique, which resulted in an average localization error of about 7 cm close to the array and 30 cm far away from the array.