In this thesis, we propose an artificial auditory system that gives a robot the ability to locate and track sounds, as well as to separate simultaneous sound sources and recognising simultaneous speech. We demonstrate that it is possible to implement these capabilities using an array of microphones, without trying to imitate the human auditory system. The sound source localisation and tracking algorithm uses a steered beamformer to locate sources, which are then tracked using a multi-source particle filter. Separation of simultaneous sound sources is achieved using a variant of the Geometric Source Separation (GSS) algorithm, combined with a multi-source post-filter that further reduces noise, interference and reverberation. Speech recognition is performed on separated sources, either directly or by using Missing Feature Theory (MFT) to estimate the reliability of the speech features. The results obtained show that it is possible to track up to four simultaneous sound sources, even in noisy and reverberant environments. Real-time control of the robot following a sound source is also demonstrated. The sound source separation approach we propose is able to achieve a 13.7 dB improvement in signal-to-noise ratio compared to a single microphone when three speakers are present. In these conditions, the system demonstrates more than 80 accuracy on digit recognition, higher than most human listeners could obtain in our small case study when recognising only one of these sources. All these new capabilities will allow humans to interact more naturally with a mobile robot in real life settings.
Most of the work so far on localisation of source sources does not address the problem of tracking moving sources. It is proposed in @cite_57 to use a Kalman filter for tracking a moving source. However the proposed method assumes that a single source is present. In the past years, particle filtering @cite_58 (a sequential Monte Carlo method) has been increasingly popular to resolve object tracking problems. Ward @cite_60 @cite_62 and Vermaak @cite_29 use this technique for tracking single sound sources. Asoh @cite_66 even suggested to use this technique for mixing audio and video data to track speakers. But again, the technique is limited to a single source due to the problem of associating the localisation observation data to each of the sources being tracked. We refer to that problem as the source-observation assignment problem. Some attempts are made at defining multi-modal particle filters in @cite_113 , and the use of particle filtering for tracking multiple targets is demonstrated in @cite_85 @cite_70 @cite_86 . But so far, the technique has not been applied to sound source tracking. Our work demonstrates that it is possible to track multiple sound sources using particle filters by solving the source-observation assignment problem.
@cite_62 @cite_60 @cite_70 @cite_29 @cite_85 @cite_113 @cite_57 @cite_86 @cite_58 @cite_66
2107493093 2127929072 2158917775 2125806710
Traditional acoustic source localization algorithms attempt to find the current location of the acoustic source using data collected at an array of sensors at the current time only. In the presence of strong multipath, these traditional algorithms often erroneously locate a multipath reflection rather than the true source location. A recently proposed approach that appears promising in overcoming this drawback of traditional algorithms, is a state-space approach using particle filtering. In this paper we formulate a general framework for tracking an acoustic source using particle filters. We discuss four specific algorithms that fit within this framework, and demonstrate their performance using both simulated reverberant data and data recorded in a moderately reverberant office room (with a measured reverberation time of 0.39 s). The results indicate that the proposed family of algorithms are able to accurately track a moving source in a moderately reverberant room. Bayesian single-target tracking techniques can be extended to a multiple-target environment by viewing the multiple-target state as a random finite set, but evaluating the multiple-target posterior distribution is currently computationally intractable for real-time applications. A practical alternative to the optimal Bayes multitarget filter is the probability hypothesis density (PHD) filter, which propagates the first-order moment of the multitarget posterior instead of the posterior distribution itself. It has been shown that the PHD is the best-fit approximation of the multitarget posterior in an information-theoretic sense. The method avoids the need for explicit data association, as the target states are viewed as a single global target state, and the identities of the targets are not part of the tracking framework. Sequential Monte Carlo approximations of the PHD using particle filter techniques have been implemented, showing the potential of this technique for real-time tracking applications. This paper presents mathematical proofs of convergence for the particle filtering algorithm and gives bounds for the mean-square error In this paper, we formulate object tracking in a particle filter framework as a multi-task sparse learning problem, which we denote as Multi-Task Tracking (MTT). Since we model particles as linear combinations of dictionary templates that are updated dynamically, learning the representation of each particle is considered a single task in MTT. By employing popular sparsity-inducing l p, q mixed norms (p Є 2, ∞ and q = 1), we regularize the representation problem to enforce joint sparsity and learn the particle representations together. As compared to previous methods that handle particles independently, our results demonstrate that mining the interdependencies between particles improves tracking performance and overall computational complexity. Interestingly, we show that the popular L 1 tracker [15] is a special case of our MTT formulation (denoted as the L 11 tracker) when p = q = 1. The learning problem can be efficiently solved using an Accelerated Proximal Gradient (APG) method that yields a sequence of closed form updates. As such, MTT is computationally attractive. We test our proposed approach on challenging sequences involving heavy occlusion, drastic illumination changes, and large pose variations. Experimental results show that MTT methods consistently outperform state-of-the-art trackers. In recent years particle filters have become a tremendously popular tool to perform tracking for nonlinear and or nonGaussian models. This is due to their simplicity, generality and success over a wide range of challenging applications. Particle filters, and Monte Carlo methods in general, are however poor at consistently maintaining the multimodality of the target distributions that may arise due to ambiguity or the presence of multiple objects. To address this shortcoming this paper proposes to model the target distribution as a nonparametric mixture model, and presents the general tracking recursion in this case. It is shown how a Monte Carlo implementation of the general recursion leads to a mixture of particle filters that interact only in the computation of the mixture weights, thus leading to an efficient numerical algorithm, where all the results pertaining to standard particle filters apply. The ability of the new method to maintain posterior multimodality is illustrated on a synthetic example and a real world tracking problem involving the tracking of football players in a video sequence.