This paper is about alerting acoustic event detection and sound source localisation in an urban scenario. Specifically, we are interested in spotting the presence of horns, and sirens of emergency vehicles. In order to obtain a reliable system able to operate robustly despite the presence of traffic noise, which can be copious, unstructured and unpredictable, we propose to treat the spectrograms of incoming stereo signals as images, and apply semantic segmentation, based on a Unet architecture, to extract the target sound from the background noise. In a multi-task learning scheme, together with signal denoising, we perform acoustic event classification to identify the nature of the alerting sound. Lastly, we use the denoised signals to localise the acoustic source on the horizon plane, by regressing the direction of arrival of the sound through a CNN architecture. Our experimental evaluation shows an average classification rate of 94 , and a median absolute error on the localisation of 7.5 when operating on audio frames of 0.5s, and of 2.5 when operating on frames of 2.5s. The system offers excellent performance in particularly challenging scenarios, where the noise level is remarkably high.
Sound source localisation in robotics has been manly concerned with human-robot interaction applications. Recently, more traditional geometry-based methods @cite_36 @cite_16 , have been replaced by deep learning techniques @cite_5 @cite_7 , which employ cross-correlation information to model the sound source location. This work is close in spirit to both these studies. Yet, with respect to those, which focus on indoor environments and multi-speaker localisation, where the noise is either low ( @math ) or structured in the form of competing speakers, we analyse and are able to cope with outdoor scenarios characterised by the presence of a variety of unknown sources of noise of a different nature, and where the SNR can be remarkably low.
@cite_36 @cite_5 @cite_16 @cite_7
2167682942 2772736377 1498826445 2765962757
This paper describes the implementation of a novel real time robot audition system which combines a 3D sound localisation system and a voice characterisation (VC) system. The localisation system employs a 4 microphone array and uses the time delay estimation method. Accuracy is improved through the use of a correlation confidence threshold and a median filter. The VC system, which classifies between speech, non speech and silence, uses a decision tree classifier and a feature set comprising MFCCs, mean MFCCs and variance in MFCCs. The complete system has a processing time of 0.73x real time, and a range of up to 3 m. The compact design, high accuracy, and real time processing ability makes the system and the approach well suited to robotics. We propose to use neural networks for simultaneous detection and localization of multiple sound sources in human-robot interaction. In contrast to conventional signal processing techniques, neural network-based sound source localization methods require fewer strong assumptions about the environment. Previous neural network-based methods have been focusing on localizing a single sound source, which do not extend to multiple sources in terms of detection and localization. In this paper, we thus propose a likelihood-based encoding of the network output, which naturally allows the detection of an arbitrary number of sources. In addition, we investigate the use of sub-band cross-correlation information as features for better localization in sound mixtures, as well as three different network architectures based on different motivations. Experiments on real data recorded from a robot show that our proposed methods significantly outperform the popular spatial spectrum-based approaches. As a step toward the design of a robot that can take part to a conversation we propose a robotic system that, taking advantage of multiple perceptual capabilities, actively follows a conversation among several human subjects. The essential idea of our proposal is that the robot system can dynamically change the focus of its attention according to visual or audio stimuli to track the actual speaker throughout the conversation and infer her identity. This paper presents a novel machine-hearing system that exploits deep neural networks (DNNs) and head movements for robust binaural localization of multiple sources in reverberant environments. DNNs are used to learn the relationship between the source azimuth and binaural cues, consisting of the complete cross-correlation function (CCF) and interaural level differences (ILDs). In contrast to many previous binaural hearing systems, the proposed approach is not restricted to localization of sound sources in the frontal hemifield. Due to the similarity of binaural cues in the frontal and rear hemifields, front–back confusions often occur. To address this, a head movement strategy is incorporated in the localization model to help reduce the front–back errors. The proposed DNN system is compared to a Gaussian-mixture-model-based system that employs interaural time differences (ITDs) and ILDs as localization features. Our experiments show that the DNN is able to exploit information in the CCF that is not available in the ITD cue, which together with head movements substantially improves localization accuracies under challenging acoustic scenarios, in which multiple talkers and room reverberation are present.