8657338
Improved pronunciation features for construct-driven assessment of non-native spontaneous speech
This paper describes research on automatic assessment of the pronunciation quality of spontaneous non-native adult speech. Since the speaking content is not known prior to the assessment, a two-stage method is developed to first recognize the speaking content based on non-native speech acoustic properties and then forced-align the recognition results with a reference acoustic model reflecting native and near-native speech properties. Features related to Hidden Markov Model likelihoods and vowel durations are extracted. Words with low recognition confidence can be excluded in the extraction of likelihood-related features to minimize erroneous alignments due to speech recognition errors. Our experiments on the TOEFL® Practice Online test, an English language assessment, suggest that the recognition/forced-alignment method can provide useful pronunciation features. Our new pronunciation features are more meaningful than an utterance-based normalized acoustic model score used in previous research from a construct point of view.
7600762
Automatic Scoring of Children's Read-Aloud Text Passages and Word Lists
Assessment of reading proficiency is typically done by asking subjects to read a text passage silently and then answer questions related to the text. An alternate approach, measuring reading-aloud proficiency, has been shown to correlate well with the aforementioned common method and is used as a paradigm in this paper. 
 
We describe a system that is able to automatically score two types of children's read speech samples (text passages and word lists), using automatic speech recognition and the target criterion "correctly read words per minute". Its performance is dependent on the data type (passages vs. word lists) as well as on the relative difficulty of passages or words for individual readers. Pearson correlations with human assigned scores are around 0.86 for passages and around 0.80 for word lists.
653913
Utilizing gestures to better understand dynamic structure of human communication
<b>Motivation:</b> Many researchers have highlighted the importance of gesture in natural human communication. McNeill [4] puts forward the hypothesis that gesture and speech stem from the same mental process and so tend to be both temporally and semantically related. However in contrast to speech, which surfaces as a linear progression of segments, sounds, and words, gestures appear to be nonlinear, holistic, and imagistic. Gesture adds an important dimension to language understanding due to this property of sharing a common origin with speech while using a very different mechanism for transferring information. Ignoring this information when constructing a model of human communication would limit its potential effectiveness.
 <b>Goal and Method:</b> This thesis concerns the development of methods to effectively incorporate gestural information from a human communication into a computer model to more accurately interpret the content and structure of that communication. Levelt [5] suggests that structure in human communication stems from the dynamic conscious process of language production, during which a conversant organizes the concepts to be expressed, plans the discourse, and selects appropriate words, prosody, and gestures while also correcting errors that occur in this process. Clues related to this conscious processing emerge in both the final speech stream and gestures. This thesis will attempt to utilize these clues to determine the structural elements of human-to-human dialogs, including sentence boundaries, topic boundaries, and disfluency structure. For this purpose, the data driven approach is used. This work requires three important components: corpus generation, feature extraction, and model construction.
 <b>Previous Work:</b> Some work related to each of these components has already been conducted. A data collection and processing protocol for constructing multimodal corpora has been created; details on the video and audio processing can be found in the <i>Data and Annotation</i> section of [3]. To improve the speed of producing a corpus while maintaining its quality, we have surveyed factors impacting the accuracy of forced alignments of transcriptions to audio files [2]. These alignments provide a crucial temporal synchronization between video events and spoken words (and their components) for this research effort. We have also conducted measurement studies in an attempt to understand how to model multimodal conversations. For example, we have investigated the types of gesture patterns that occur during speech repairs [1]. Recently, we constructed a preliminary model combining speech and gesture features for detecting sentence boundaries in videotaped dialogs. This model combines language and prosody models together with a simple gestural model to more effectively detect sentence boundaries [3].
 <b>Future Work:</b> To date, our multimodal corpora involve human monologues and dialogues (see http://vislab.cs.wright.edu/kdi). We are participating in the collection and preparation of a corpus of multi-party meetings (see http://vislab.cs.wright.edu/Projects/Meeting-Analysis). To facilitate the multi-channel audio processing, we are constructing a tool to support accurate audio transcription and alignment. The data from this meeting corpus will enable the development of more sophisticated gesture models allowing us to expand the set of gesture features (e.g., spatial properties of the tracked gestures). Additionally, we will investigate more advanced machine learning methods in an attempt to improve the performance of our models. We also plan to expand our models to phenomena such as topic segmentation.
18230
Indexable PLA for Efficient Similarity Search
Similarity-based search over time-series databases has been a hot research topic for a long history, which is widely used in many applications, including multimedia retrieval, data mining, web search and retrieval, and so on. However, due to high dimensionality (i.e. length) of the time series, the similarity search over directly indexed time series usually encounters a serious problem, known as the "dimensionality curse". Thus, many dimensionality reduction techniques are proposed to break such curse by reducing the dimensionality of time series. Among all the proposed methods, only Piecewise Linear Approximation (PLA) does not have indexing mechanisms to support similarity queries, which prevents it from efficiently searching over very large time-series databases. Our initial studies on the effectiveness of different reduction methods, however, show that PLA performs no worse than others. Motivated by this, in this paper, we re-investigate PLA for approximating and indexing time series. Specifically, we propose a novel distance function in the reduced PLA-space, and prove that this function indeed results in a lower bound of the Euclidean distance between the original time series, which can lead to no false dismissals during the similarity search. As a second step, we develop an effective approach to index these lower bounds to improve the search efficiency. Our extensive experiments over a wide spectrum of real and synthetic data sets have demonstrated the efficiency and effectiveness of PLA together with the newly proposed lower bound distance, in terms of both pruning power and wall clock time, compared with two state-of-the-art reduction methods, Adaptive Piecewise Constant Approximation (APCA) and Chebyshev Polynomials (CP).
28385041
A Modular System for the Classification of Time Series Data
While the field of classification is witnessing excellent achievement in recent years, not much attention is given to methods that deal with the time series data. In this paper, we propose a modular system for the classification of time series data. The proposed approach explores the diversity through various input representation techniques, each of which focuses on a certain aspect of the temporal patterns. The temporal patterns are identified by aggregation of the decisions of multiple classifiers trained through different representations of the input data. Several time series data sets are employed to examine the validity of the proposed approach. The results obtained from our experiments show that the performance of the proposed approach is effective as well as robust.
190958
Reinforcement learning and aggregation
Reinforcement learning (RL) is a learning technique that provides a means for learning an optimal control policy when the dynamics of the environment under consideration is unavailable [L.P. Kaelbling et al., 1996, R.S. Sutton and A.G. Barto, 1998]. While RL has been successfully applied in many single or multiple agents systems [S. Arai et al., 2000, H.R. Berenji and D.A. Vengerov, 2000, M. Tan, 1993, Y. Nagayuki et al., 2000], the learning quality is greatly influenced by learning algorithms and their parameters. Setting of the parameters of RL algorithms is something of a black art, and small differences in these parameters can lead to large differences in learning qualities. Determining the best algorithm and the optimal parameters can be costly in terms of time and computation. Even if the cost is acceptable, the robustness of learning is still a question. In order to address the difficulty, an aggregated multiagent reinforcement learning system (AMRLS) is proposed to deal with the RL environment as a multiagent environment. A maze world environment is used to validate the AMRLS. Experimental results illustrate that compared with normal Q(/spl lambda/)-learning and SARSA(/spl lambda/) algorithms, the AMRLS increases both the learning speed and the rate of reaching the shortest path.
43849850
Gender Recognition from Gait Using Radon Transform and Relevant Component Analysis
In this paper, a new method for gender recognition via gait silhouettes is proposed. In the feature extraction process, Radon transform on all the 180 angle degrees is applied to every silhouette to construct gait templates and the initial phase of each silhouette in an entire gait cycle is also associated to the templates representing dynamic information of walking. Then the Relevant Component Analysis (RCA) algorithm is employed on the radon-transformed templates to get a maximum likelihood estimation of the within class covariance matrix. At last, the Mahalanobis distances are calculated to measure gender dissimilarity in recognition. The Nearest Neighbor (NN) classifier is adopted to determine whether a sample in the Probe Set is male or female. Experimental results in comparison to state-of-the-art methods show considerable improvement in recognition performance of our proposed algorithm.