7031742
The effect of pruning and compression on graphical representations of the output of a speech recognizer
Large vocabulary continuous speech recognition can benefit from an efficient data structure for representing a large number of acoustic hypotheses compactly. Word graphs or lattices have been chosen as such an efficient interface between acoustic recognition engines and subsequent language processing modules. This paper first investigates the effect of pruning during acoustic decoding on the quality of word lattices and shows that by combining different pruning options (at the model level and word level), we can obtain word lattices with comparable accuracy to the original lattices and a manageable size. In order to use the word lattices as the input for a post-processing language module, they should preserve the target hypotheses and their scores while being as small as possible. In this paper, we introduce a word graph compression algorithm that significantly reduces the number of words in the graphical representation without eliminating utterance hypotheses or distorting their acoustic scores. We compare this word graph compression algorithm with several other lattice size-reducing approaches and demonstrate the relative strength of the new word graph compression algorithm for decreasing the number of words in the representation. Experiments are conducted across corpora and vocabulary sizes to determine the consistency of the pruning and compression results.
7717376
Enriching speech recognition with automatic detection of sentence boundaries and disfluencies
Effective human and automatic processing of speech requires recovery of more than just the words. It also involves recovering phenomena such as sentence boundaries, filler words, and disfluencies, referred to as structural metadata. We describe a metadata detection system that combines information from different types of textual knowledge sources with information from a prosodic classifier. We investigate maximum entropy and conditional random field models, as well as the predominant hidden Markov model (HMM) approach, and find that discriminative models generally outperform generative models. We report system performance on both broadcast news and conversational telephone speech tasks, illustrating significant performance differences across tasks and as a function of recognizer performance. The results represent the state of the art, as assessed in the NIST RT-04F evaluation
6898492
Using Conditional Random Fields for Sentence Boundary Detection in Speech
Sentence boundary detection in speech is important for enriching speech recognition output, making it easier for humans to read and downstream modules to process. In previous work, we have developed hidden Markov model (HMM) and maximum entropy (Maxent) classifiers that integrate textual and prosodic knowledge sources for detecting sentence boundaries. In this paper, we evaluate the use of a conditional random field (CRF) for this task and relate results with this model to our prior work. We evaluate across two corpora (conversational telephone speech and broadcast news speech) on both human transcriptions and speech recognition output. In general, our CRF model yields a lower error rate than the HMM and Maxent models on the NIST sentence boundary detection task in speech, although it is interesting to note that the best results are achieved by three-way voting among the classifiers. This probably occurs because each model has different strengths and weaknesses for modeling the knowledge sources.
7472011
PCFGs with Syntactic and Prosodic Indicators of Speech Repairs
A grammatical method of combining two kinds of speech repair cues is presented. One cue, prosodic disjuncture, is detected by a decision tree-based ensemble classifier that uses acoustic cues to identify where normal prosody seems to be interrupted (Lickley, 1996). The other cue, syntactic parallelism, codifies the expectation that repairs continue a syntactic category that was left unfinished in the reparandum (Levelt, 1983). The two cues are combined in a Treebank PCFG whose states are split using a few simple tree transformations. Parsing performance on the Switchboard and Fisher corpora suggests that these two cues help to locate speech repairs in a synergistic way.
1429796
A study in machine learning from imbalanced data for sentence boundary detection in speech
Abstract Enriching speech recognition output with sentence boundaries improves its human readability and enables further processing by downstream language processing modules. We have constructed a hidden Markov model (HMM) system to detect sentence boundaries that uses both prosodic and textual information. Since there are more nonsentence boundaries than sentence boundaries in the data, the prosody model, which is implemented as a decision tree classifier, must be constructed to effectively learn from the imbalanced data distribution. To address this problem, we investigate a variety of sampling approaches and a bagging scheme. A pilot study was carried out to select methods to apply to the full NIST sentence boundary evaluation task across two corpora (conversational telephone speech and broadcast news speech), using both human transcriptions and recognition output. In the pilot study, when classification error rate is the performance measure, using the original training set achieves the best performance among the sampling methods, and an ensemble of multiple classifiers from different downsampled training sets achieves slightly poorer performance, but has the potential to reduce computational effort. However, when performance is measured using receiver operating characteristics (ROC) or area under the curve (AUC), then the sampling approaches outperform the original training set. This observation is important if the sentence boundary detection output is used by downstream language processing modules. Bagging was found to significantly improve system performance for each of the sampling methods. The gain from these methods may be diminished when the prosody model is combined with the language model, which is a strong knowledge source for the sentence detection task. The patterns found in the pilot study were replicated in the full NIST evaluation task. The conclusions may be dependent on the task, the classifiers, and the knowledge combination approach.
6989862
Using Prosody for Automatic Sentence Segmentation of Multi-party Meetings
We explore the use of prosodic features beyond pauses, including duration, pitch, and energy features, for automatic sentence segmentation of ICSI meeting data We examine two different approaches to boundary classification: score-level combination of independent language and prosodic models using HMMs, and feature-level combination of models using a boosting-based method (BoosTexter) We report classification results for reference word transcripts as well as for transcripts from a state-of-the-art automatic speech recognizer (ASR) We also compare results using the lexical model plus a pause-only prosody model, versus results using additional prosodic features Results show that (1) information from pauses is important, including pause duration both at the boundary and at the previous and following word boundaries; (2) adding duration, pitch, and energy features yields significant improvement over pause alone; (3) the integrated boosting-based model performs better than the HMM for ASR conditions; (4) training the boosting-based model on recognized words yields further improvement.
5655838
Automated software engineering using concurrent class machines
Concurrent Class Machines are a novel state-machine model that directly captures a variety of object-oriented concepts, including classes and inheritance, objects and object creation, methods, method invocation and exceptions, multithreading and abstract collection types. The model can be understood as a precise definition of UML activity diagrams which, at the same time, offers an executable, object-oriented alternative to event-based statecharts. It can also be understood as a visual, combined control and data flow model for multithreaded object-oriented programs. We first introduce a visual notation and tool for Concurrent Class Machines and discuss their benefits in enhancing system design. We then equip this notation with a precise semantics that allows us to define refinement and modular refinement rules. Finally, we summarize our work on generation of optimized code, implementation and experiments, and compare with related work.
2949829
On the design of MIMO block-fading channels with feedback-link capacity constraint
In this paper, we propose a combined adaptive power control and beamforming framework for optimizing multiple-input/multiple-output (MIMO) link capacity in the presence of feedback-link capacity constraint. The feedback channel is used to carry channel state information only. It is assumed to be noiseless and causal with a feedback capacity constraint in terms of maximum number of feedback bits per fading block. We show that the hybrid design could achieve the optimal MIMO link capacity, and we derive a computationally efficient algorithm to search for the optimal design under a specific average power constraint. Finally, we shall illustrate that a minimum mean-square error spatial processor with a successive interference canceller at the receiver could be used to realize the optimal capacity. We found that feedback effectively enhances the forward channel capacity for all signal-to-noise ratio (SNR) values when the number of transmit antennas (n/sub T/) is larger than the number of receive antennas (n/sub R/). The SNR gain with feedback is contributed by focusing transmission power on active eigenchannel and temporal power waterfilling . The former factor contributed, at most, 10log/sub 10/(n/sub T//n/sub R/) dB SNR gain when n/sub T/>n/sub R/, while the latter factor's SNR gain is significant only for low SNR values.
14501514
Semi-supervised Multi-label Learning by Constrained Non-negative Matrix Factorization
We present a novel framework for multi-label learning that explicitly addresses the challenge arising from the large number of classes and a small size of training data. The key assumption behind this work is that two examples tend to have large overlap in their assigned class memberships if they share high similarity in their input patterns. We capitalize this assumption by first computing two sets of similarities, one based on the input patterns of examples, and the other based on the class memberships of the examples. We then search for the optimal assignment of class memberships to the unlabeled data that minimizes the difference between these two sets of similarities. The optimization problem is formulated as a constrained Non-negative Matrix Factorization (NMF) problem, and an algorithm is presented to efficiently find the solution. Compared to the existing approaches for multi-label learning, the proposed approach is advantageous in that it is able to explore both the unlabeled data and the correlation among different classes simultaneously. Experiments with text categorization show that our approach performs significantly better than several state-of-the-art classification techniques when the number of classes is large and the size of training data is small.
10845590
Developing structural constraints for accurate registration of overlapping range images
Abstract Automatic image registration is an attractive and unresolved problem in the machine vision literature. This paper presents two novel structural constraints, namely proximity and closeness constraints, that improve both the accuracy and robustness of an existing motion consistency based algorithm (GICP) for automatic image registration. It also defines the conditions when such constraints are fired at specific points of the registration of two overlapping range images. The proximity constraint says that neighbouring points should be neighbouring before and after a rigid motion. The closeness constraint implements a local closest point search from the second image to the first image. While the GICP algorithm uses an exhaustive search for the closest points, in this paper, we employ the optimised k–d tree data structure to accelerate the closest point search. A large number of experiments based on real range images demonstrate that the combination of rigid motion constraints with these novel proximity and closeness constraints leads to a more accurate and robust evaluation of possible correspondences leading, in turn, to more accurate, robust and efficient automatic image registration results.
5965790
On scalable and locality-aware web document sharing
We propose a scalable Web document sharing infrastructure model called Browsers-Aware Proxy Server. In this design, a proxy server connecting to a group of networked clients maintains an index file of data objects of all clients' browser caches. If a user request misses in its local browser cache and the proxy cache, the browsers-aware proxy server will search the index file attempting to find it in another client's browser cache before sending the request to an upper level proxy or the Web server. If such a request does hit in a remote client, this client will directly forward the data object to the requesting client; or the proxy server fetches the data object from this client and then forwards it to the requesting client. The contributions of this caching model are twofold. First, we show that the amount of sharable data in browser caches is significant and can be utilized for document sharing among clients to improve Web caching performance and scalability. Second, the browsers-aware model can effectively and further improve Web prefetching performance. The browsers-aware model and its supported prefetching technique build a strong locality-aware Internet environment to make Web accesses fast with low communication costs. Conducting trace-driven simulations, we show the effectiveness of the browsers-aware model, and its unique advantages to facilitate prefetching.