Are real tongue movements easier to speech read than synthesized?
Speech perception studies with augmented reality displays in talking heads have shown that tongue reading abilities are weak initially, but that subjects become able to extract some information from intra-oral visualizations after a short training session. In this study, we investigate how the nature of the tongue movements influences the results, by comparing synthetic rule-based and actual, measured movements. The subjects were significantly better at perceiving sentences accompanied by real movements, indicating that the current coarticulation model developed for facial movements is not optimal for the tongue.
We focus on the problem of learning semantics from multimedia data associated with broadcast video documents. In this paper we propose to learn semantic concepts from multimodal sources based on style and context detectors, in combination with statistical classier ensembles. As a case study we present our method for detecting the concept of news subject monologues. This approach had the best average precision performance amongst 26 submissions in the 2003 video track of the Text Retrieval Conference benchmark. Experiments were conducted with respect to individual detector contribution, ensemble size, and ranking mechanism. It was found that the combination of detectors is decisive for the nal result, although some detectors might appear useless in isolation. Moreover, by using a probabilistic ranking, in combination with a large classier ensemble, results can be improved even further.