5705405
Robust face-voice based speaker identity verification using multilevel fusion
In this paper, we propose a robust multilevel fusion strategy involving cascaded multimodal fusion of audio-lip-face motion, correlation and depth features for biometric person authentication. The proposed approach combines the information from different audio-video based modules, namely: audio-lip motion module, audio-lip correlation module, 2D+3D motion-depth fusion module, and performs a hybrid cascaded fusion in an automatic, unsupervised and adaptive manner, by adapting to the local performance of each module. This is done by taking the output-score based reliability estimates (confidence measures) of each of the module into account. The module weightings are determined automatically such that the reliability measure of the combined scores is maximised. To test the robustness of the proposed approach, the audio and visual speech (mouth) modalities are degraded to emulate various levels of train/test mismatch; employing additive white Gaussian noise for the audio and JPEG compression for the video signals. The results show improved fusion performance for a range of tested levels of audio and video degradation, compared to the individual module performances. Experiments on a 3D stereovision database AVOZES show that, at severe levels of audio and video mismatch, the audio, mouth, 3D face, and tri-module (audio-lip motion, correlation and depth) fusion EERs were 42.9%, 32%, 15%, and 7.3%, respectively, for biometric person authentication task.
25044094
Generalised Fuzzy Hidden Markov Models for Speech Recognition
A generalised fuzzy approach to statistical modelling techniques for speech recognition is proposed in this paper. Fuzzy C-means (FCM) and fuzzy entropy (FE) techniques are combined into a generalised fuzzy technique and applied to hidden Markov models (HMMs). A more robust version of the above fuzzy technique based on the noise clustering (NC) method is also proposed. Experimental results were performed on the TI46 speech data corpus. A significant result for isolatedword recognition performed on a highly confusable vocabulary consisting of the nine English E-set words is that, a 33.8% recognition error rate for the HMM-based system was reduced to 30.5%, 29.9%, 29.8% and 27.8%, respectively, by using the FCM-HMM, the FE-HMM, the NC-FE-HMM, and the NC-FCM-HMM-based systems.
10506744
Fusion of Acoustic and Linguistic Features for Emotion Detection
This paper describes a system that deploys acoustic and linguistic information from speech in order to decide whether the utterance contains negative or non-negative meaning. An earlier version of this system was Passed QA to the Interspeech-2009 Emotion Challenge evaluation. The speech data consist of short utterances of the children’s speech, and the proposed system is designed to detect anger in each given chunk. Various frame-based cepstral, prosodic and acoustic features are extracted automatically and classified by means of a support vector machine. An automatic speech recognizer transcribes the utterances and yields a separate classification, based on the degree of emotional salience of the words. The emotionally salient words are computed on word hypotheses, so that un-transcribed training data is sufficient. Late fusion is applied to make a final decision on anger vs. non-anger of the utterance.
35465720
Reliability score based multimodal fusion for biometric person authentication
In this paper, we propose a robust multilevel fusion strategy involving cascaded fusion of hybrid multimodal fusion of audio-lip-face motion, correlation and depth features for biometric person authentication. The proposed approach combines the information from different audio-video based modules, namely: audio-lip motion module, audio-lip correlation module, 2D+3D motiondepth fusion module, and performs a hybrid cascaded fusion in an automatic, unsupervised and adaptive manner, by adapting to the local performance of each module. This is done by taking the output-score based reliability estimates (confidence measures) of each of the module into account. The module weightings are determined automatically such that the reliability measure of the combined scores is maximised. To test the robustness of the proposed approach, the audio and visual speech (mouth) modalities are degraded to emulate various levels of train/test mismatch; employing additive white Gaussian noise for the audio and JPEG compression for the video signals. The results show improved fusion performance for a range of tested levels of audio and video degradation, compared to the individual module performances. Experiments on a 3D stereovision database AVOZES show that, at severe levels of audio and video mismatch, the audio, mouth, 3D face, and tri-module (audio-lip motion, correlation and depth) fusion EERs were 42.9%, 32%, 15%, and 7.3% respectively for biometric person authentication task.
43139712
Speaker Verification with Fuzzy Fusion and Genetic Optimization
987011
Similarity normalization for speaker verification by fuzzy fusion
Abstract Similarity or likelihood normalization techniques are important for speaker verification systems as they help to alleviate the variations in the speech signals. In the conventional normalization, the a priori probabilities of the cohort speakers are assumed to be equal. From this standpoint, we apply the theory of fuzzy measure and fuzzy integral to combine the likelihood values of the cohort speakers in which the assumption of equal a priori probabilities is relaxed. This approach replaces the conventional normalization term by the fuzzy integral which acts as a non-linear fusion of the similarity measures of an utterance assigned to the cohort speakers. We illustrate the performance of the proposed approach by testing the speaker verification system with both the conventional and the fuzzy algorithms using the commercial speech corpus TI46. The results in terms of the equal error rates show that the speaker verification system using the fuzzy integral is more flexible and more favorable than the conventional normalization method.
37039344
An e-Learning Tool for the Austrian Citizen Card
Austria is one of the leading countries in the field of electronic identity. The Austrian Citizen Card System is presently going live; the next step is to inform the citizen about the functionalities and advantages, as well as on how to use the card. On authority of the Federal Chancellor’s Office, the Danube University has developed an e-learning tool that will be made available to all Austrian households. The target group is the citizen rather than the early technological adopter, and the aim is to explain the use of a citizen card rather than the technical background. The main pedagogical method therefore is to reduce technical complexity to an absolute minimum. This e-learning tool is positioned to be one piece of a sophisticated e-government communication strategy.
8554783
Mapping visual notations to MOF compliant models with QVT relations
Model-centric methodologies rely on the definition of domain-specific modeling languages for being able to create domain-specific models. With MOF the OMG adopted a standard which provides the essential constructs for the definition of semantic language constructs (abstract syntax). However, there are no specifications on how to define the notations (concrete syntax) for abstract syntax elements. Usually, the concrete syntax of MOF compliant languages is described informally.
 We propose to define MOF-based metamodels for abstract syntax and concrete syntax and to connect them by model transformations specified with QVT Relations in a flexible, declarative way. Using a QVT based transformation engine one can easily implement a Model View Controller architecture by integrating modeling tools and metadata repositories
32510848
Using Augmented Virtuality for Remote Collaboration
This paper describes the concept, prototypical implementation, and usability evaluation of the Augmented Virtuality (AV)-based videoconferencing (VC) system cAR/PE!. We present a solution that allows three participants at different locations to communicate over a network in an environment simulating a traditional face-to-face meeting. Integrated into the AV environment are live video streams of the participants spatially arranged around a virtual table, a large virtual presentation screen for 2D display and application sharing, and 3D geometry (models) within the room and on top of the table. We describe the general concept and application scenario as well as the actual hardware setup, the implementation, and the use of the system in its current state. Results of two usability studies with 87 subjects are presented that show the general usability of our approach as well as good overall satisfaction. Parts of the work described here were presented as a poster at the second International Symposium on Mixed and Augmented Reality (Regenbrecht, Ott, Wagner, Lum, Kohler, et al., 2003. An Augmented Virtuality Approach to 3D Videoconferencing. Poster at 2nd Int. Symp. on Mixed and Aug. Reality, Tokyo.).
20326889
Towards a Comprehensive Teamwork Model for Highly Dynamic Domains
Cooperative behaviour of agents within highly dynamic and nondeterministic domains is an active field of research. In particular establishing responsive teamwork, where agents are able to react to dynamic changes in the environment while facing unreliable communication and sensory noise, is an open problem. Unexpectedly changing situations force agents to react and adapt under tight time-constraints. Hence they often cannot communicate or agree upon their decisions before acting upon them. We present a novel model for cooperative behaviour geared towards such domains. In our approach, the agents estimate each other’s decision and correct these estimations once they receive contradictory information. We aim at a comprehensive approach for agent teamwork featuring intuitive modelling capabilities for multi-agent activities, abstractions over activities and agents, and clear operational semantics for the new model. We show experimentally that the resulting behaviour stabilises towards teamwork and can achieve a cooperative goal.
751960
Performance Evaluation Methodologies for Abstract Data Type Implementation Techniques
Abstract The runtime management system for a programming language must allow the efficient manipulation of complex objects, such as instances of abstract data types (ADTs). We describe an implementation model for ADTs based on coroutine objects (i.e. as an alternative to ad hoc modifications of the runtime stack). We evaluate the performances of variants of our implementation model using two different approaches: a benchmarking technique which monitors actual costs for test programs and a “pseudo-execution” technique which estimates costs based on idealized instruction timings. The results of our comparison indicate that the two forms of evaluation give very similar cost estimates, supporting the use of the (cheaper) pseudo-execution technique. The overall performance results identify applications in which our coroutine implementation model has definite strengths and weaknesses. Finally, our implementation approach is compared with modified stack implementations used in other ADT systems.