7958668
Boltzmann Machine Learning with the Latent Maximum Entropy Principle
We present a new statistical learning paradigm for Boltzmann machines based on a new inference principle we have proposed: the latent maximum entropy principle (LME). LME is different both from Jaynes' maximum entropy principle and from standard maximum likelihood estimation. We demonstrate the LME principle by deriving new algorithms for Boltzmann machine parameter estimation, and show how a robust and rapidly convergent new variant of the EM algorithm can be developed. Our experiments show that estimation based on LME generally yields better results than maximum likelihood estimation when inferring models from small amounts of data.
6812524
Stochastic Analysis of Lexical and Semantic Enhanced Structural Language Model
In this paper, we present a directed Markov random field model that integrates trigram models, structural language models (SLM) and probabilistic latent semantic analysis (PLSA) for the purpose of statistical language modeling. The SLM is essentially a generalization of shift-reduce probabilistic push-down automata thus more complex and powerful than probabilistic context free grammars (PCFGs). The added context-sensitiveness due to trigrams and PLSAs and violation of tree structure in the topology of the underlying random field model make the inference and parameter estimation problems plausibly intractable, however the analysis of the behavior of the lexical and semantic enhanced structural language model leads to a generalized inside-outside algorithm and thus to rigorous exact EM type re-estimation of the composite language model parameters.
40909266
Exploiting Syntactic, Semantic, and Lexical Regularities in Language Modeling via Directed Markov Random Fields
We present a directed Markov random field (MRF) model that combines n-gram models, probabilistic context-free grammars (PCFGs), and probabilistic latent semantic analysis (PLSA) for the purpose of statistical language modeling. Even though the composite directed MRF model potentially has an exponential number of loops and becomes a context-sensitive grammar, we are nevertheless able to estimate its parameters in cubic time using an efficient modified Expectation-Maximization (EM) method, the generalized inside–outside algorithm, which extends the inside–outside algorithm to incorporate the effects of the n-gram and PLSA language models. We generalize various smoothing techniques to alleviate the sparseness of n-gram counts in cases where there are hidden variables. We also derive an analogous algorithm to find the most likely parse of a sentence and to calculate the probability of initial subsequence of a sentence, all generated by the composite language model. Our experimental results on the Wall Street Journal corpus show that we obtain significant reductions in perplexity compared to the state-of-the-art baseline trigram model with Good–Turing and Kneser–Ney smoothing techniques.
210649
Combining Statistical Language Models via the Latent Maximum Entropy Principle
We present a unified probabilistic framework for statistical language modeling which can simultaneously incorporate various aspects of natural language, such as local word interaction, syntactic structure and semantic document information. Our approach is based on a recent statistical inference principle we have proposed—the latent maximum entropy principle—which allows relationships over hidden features to be effectively captured in a unified model. Our work extends previous research on maximum entropy methods for language modeling, which only allow observed features to be modeled. The ability to conveniently incorporate hidden variables allows us to extend the expressiveness of language models while alleviating the necessity of pre-processing the data to obtain explicitly observed features. We describe efficient algorithms for marginalization, inference and normalization in our extended models. We then use these techniques to combine two standard forms of language models: local lexical models (Markov N-gram models) and global document-level semantic models (probabilistic latent semantic analysis). Our experimental results on the Wall Street Journal corpus show that we obtain a 18.5% reduction in perplexity compared to the baseline tri-gram model with Good-Turing smoothing.
1771247
Using query-specific variance estimates to combine Bayesian classifiers
Many of today's best classification results are obtained by combining the responses of a set of base classifiers to produce an answer for the query. This paper explores a novel "query specific" combination rule: After learning a set of simple belief network classifiers, we produce an answer to each query by combining their individual responses, using weights based inversely on their respective variances around their responses. These variances are based on the uncertainty of the network parameters, which in turn depend on the training datasample. In essence, this variance quantifies the base classifier's confidence of its response to this query. Our experimental results show that these "mixture-using-variance belief net classifiers" MUVS work effectively, especially when the base classifiers are learned using balanced bootstrap samples and when their results are combined using James-Stein shrinkage. We also found that our variance-based combination rule performed better than both bagging and AdaBoost, even on the set of base classifiers produced by AdaBoost itself. Finally, this framework is extremely efficient, as both the learning and the classification components require only straight-line code.
662359
Language and Task Independent Text Categorization with Simple Language Models
We present a simple method for language independent and task independent text categorization learning, based on character-level n-gram language models. Our approach uses simple information theoretic principles and achieves effective performance across a variety of languages and tasks without requiring feature selection or extensive pre-processing. To demonstrate the language and task independence of the proposed technique, we present experimental results on several languages---Greek, English, Chinese and Japanese---in several text categorization problems---language identification, authorship attribution, text genre classification, and topic detection. Our experimental results show that the simple approach achieves state of the art performance in each case.
17967492
Network Game and Boosting
We propose an ensemble learning method called Network Boosting which combines weak learners together based on a random graph (network). A theoretic analysis based on the game theory shows that the algorithm can learn the target hypothesis asymptotically. The comparison results using several datasets of the UCI machine learning repository and synthetic data are promising and show that Network Boosting has much resistance to the noisy data than AdaBoost through the cooperation of classifiers in the classifier network.
15695627
Representing images using points on image surfaces
This paper presents a new approach to represent an image by "verge points," which are defined as high-curvature points on the image surface. This representation offers a compact and reversible way to preserve the essence of the original image. Various applications, such as compression, edge detection, image enhancement, and image editing, can be achieved based on this representation. In this paper, the whole procedure for verge point representation is presented. Based on these verge points, image reconstruction can be easily achieved via iterative linear interpolation. These extracted verge points with compatible properties are further linked into verge curves to offer more compact representation. Progressive representation is also developed based on a multiscale extraction scheme. Some potential applications are then presented to demonstrate the versatility of this representation.
27992571
Segmentation of SAR images
Abstract This paper presents a new algorithm for segmentation of SAR images based on threshold estimation using the histogram. The speckle distribution in the SAR image is modeled by a Gamma function. Thus, the SAR image histogram exhibits a combination of Gamma distributions. The maximum likelihood technique is therefore used to estimate the histogram parameters. This technique requires knowledge of the number of modes of the histogram, the number of looks of the SAR image, and the initial parameters of the histogram. The second derivative of the histogram is used to estimate the number of modes. We use two methods to estimate the number of looks. Initial parameters are estimated at the maximum of the Gamma function. Thresholds are selected at the valleys of a multi-modal histogram by minimizing the discrimination error between the classes of pixels in the image. The algorithm is applied to several RADARSAT SAR images with different number of looks. The results obtained are promising.
2070714
A large quantity of novel human antisense transcripts detected by LongSAGE
MOTIVATION
Taking advantage of the high sensitivity and specificity of LongSAGE tag for transcript detection and genome mapping, we analyzed the 632 813 unique human LongSAGE tags deposited in public databases to identify novel human antisense transcripts.


RESULTS
Our study identified 45 321 tags that match the antisense strand of 9804 known mRNA sequences, 6606 of which contain antisense ESTs and 3198 are mapped only by SAGE tags. Quantitative analysis showed that the detected antisense transcripts are present at levels lower than their counterpart sense transcripts. Experimental results confirmed the presence of antisense transcripts detected by the antisense tags. We also constructed an antisense tag database that can be used to identify the antisense SAGE tags originated from the antisense strand of known mRNA sequences included in the RefSeq database.


CONCLUSIONS
Our study highlights the benefits of exploring SAGE data for comprehensive identification of human antisense transcripts and demonstrates the prevalence of antisense transcripts in the human genome.
35665703
Implementation of QoS-Provisioning system for voice over IP
In this paper, we address issues related to implementing Voice-over-IP (VoIP) services in packet switching networks. VoIP has been identified as a critical real-time application in the network QoS research community and has been implemented in commercial products. To provide competent quality of service for VoIP, the call admission control (CAC) mechanism has to be introduced to prevent packet losing and over-queuing. Several well-designed CAC mechanisms, such as the Site- Utilization-Based CAC and the link-utilization-based CAC mechanisms, are in place. However, the existing commercial VoIP systems have not been able to adequately apply and support these CAC mechanisms, and hence unable to provide QoS guarantees to VoIP. We have designed and implemented a QoS-Provisioning system that can be seamlessly integrated to the existing VoIP system to overcome its weakness in offering QoS guarantees. As a result, our system has been realized at Internet2 Voice Over IP Testbed in Texas A&M University.