1771247
Using query-specific variance estimates to combine Bayesian classifiers
Many of today's best classification results are obtained by combining the responses of a set of base classifiers to produce an answer for the query. This paper explores a novel "query specific" combination rule: After learning a set of simple belief network classifiers, we produce an answer to each query by combining their individual responses, using weights based inversely on their respective variances around their responses. These variances are based on the uncertainty of the network parameters, which in turn depend on the training datasample. In essence, this variance quantifies the base classifier's confidence of its response to this query. Our experimental results show that these "mixture-using-variance belief net classifiers" MUVS work effectively, especially when the base classifiers are learned using balanced bootstrap samples and when their results are combined using James-Stein shrinkage. We also found that our variance-based combination rule performed better than both bagging and AdaBoost, even on the set of base classifiers produced by AdaBoost itself. Finally, this framework is extremely efficient, as both the learning and the classification components require only straight-line code.
210649
Combining Statistical Language Models via the Latent Maximum Entropy Principle
We present a unified probabilistic framework for statistical language modeling which can simultaneously incorporate various aspects of natural language, such as local word interaction, syntactic structure and semantic document information. Our approach is based on a recent statistical inference principle we have proposed—the latent maximum entropy principle—which allows relationships over hidden features to be effectively captured in a unified model. Our work extends previous research on maximum entropy methods for language modeling, which only allow observed features to be modeled. The ability to conveniently incorporate hidden variables allows us to extend the expressiveness of language models while alleviating the necessity of pre-processing the data to obtain explicitly observed features. We describe efficient algorithms for marginalization, inference and normalization in our extended models. We then use these techniques to combine two standard forms of language models: local lexical models (Markov N-gram models) and global document-level semantic models (probabilistic latent semantic analysis). Our experimental results on the Wall Street Journal corpus show that we obtain a 18.5% reduction in perplexity compared to the baseline tri-gram model with Good-Turing smoothing.
662359
Language and Task Independent Text Categorization with Simple Language Models
We present a simple method for language independent and task independent text categorization learning, based on character-level n-gram language models. Our approach uses simple information theoretic principles and achieves effective performance across a variety of languages and tasks without requiring feature selection or extensive pre-processing. To demonstrate the language and task independence of the proposed technique, we present experimental results on several languages---Greek, English, Chinese and Japanese---in several text categorization problems---language identification, authorship attribution, text genre classification, and topic detection. Our experimental results show that the simple approach achieves state of the art performance in each case.
12212021
Variational Bayesian image modelling
We present a variational Bayesian framework for performing inference, density estimation and model selection in a special class of graphical models---Hidden Markov Random Fields (HMRFs). HMRFs are particularly well suited to image modelling and in this paper, we apply them to the problem of image segmentation. Unfortunately, HMRFs are notoriously hard to train and use because the exact inference problems they create are intractable. Our main contribution is to introduce an efficient variational approach for performing approximate inference of the Bayesian formulation of HMRFs, which we can then apply to the density estimation and model selection problems that arise when learning image models from data. With this variational approach, we can conveniently tackle the problem of image segmentation. We present experimental results which show that our technique outperforms recent HMRF-based segmentation methods on real world images.
40909266
Exploiting Syntactic, Semantic, and Lexical Regularities in Language Modeling via Directed Markov Random Fields
We present a directed Markov random field (MRF) model that combines n-gram models, probabilistic context-free grammars (PCFGs), and probabilistic latent semantic analysis (PLSA) for the purpose of statistical language modeling. Even though the composite directed MRF model potentially has an exponential number of loops and becomes a context-sensitive grammar, we are nevertheless able to estimate its parameters in cubic time using an efficient modified Expectation-Maximization (EM) method, the generalized inside–outside algorithm, which extends the inside–outside algorithm to incorporate the effects of the n-gram and PLSA language models. We generalize various smoothing techniques to alleviate the sparseness of n-gram counts in cases where there are hidden variables. We also derive an analogous algorithm to find the most likely parse of a sentence and to calculate the probability of initial subsequence of a sentence, all generated by the composite language model. Our experimental results on the Wall Street Journal corpus show that we obtain significant reductions in perplexity compared to the state-of-the-art baseline trigram model with Good–Turing and Kneser–Ney smoothing techniques.
261372
Learning Mixture Models with the Latent Maximum Entropy Principle
We present a new approach to estimating mixture models based on a new inference principle we have proposed: the latent maximum entropy principle (LME). LME is different both from Jaynes' maximum entropy principle and from standard maximum likelihood estimation. We demonstrate the LME principle by deriving new algorithms for mixture model estimation, and show how robust new variants of the EM algorithm can be developed. Our experiments show that estimation based on LME generally yields better results than maximum likelihood estimation, particularly when inferring latent variable models from small amounts of data.
10317661
Nonrigid Shape Correspondence Using Landmark Sliding, Insertion and Deletion
The growing usage of statistical shape analysis in medical imaging calls for effective methods for highly accurate shape correspondence. This paper presents a novel landmark-based method to correspond a set of 2D shape instances in a nonrigid fashion. Different from prior methods, the proposed method combines three important factors in measuring the shape-correspondence error: landmark-correspondence error, shape-representation error, and shape-representation compactness. In this method, these three important factors are explicitly handled by the landmark sliding, insertion, and deletion operations, respectively. The proposed method is tested on several sets of structural shape instances extracted from medical images. We also conduct an empirical study to compare the developed method to the popular Minimum Description Length method.
14809978
Image contrast enhancement based on intensity-pair distribution
Current contrast enhancement algorithms sometimes come with undesired drawbacks, like the loss of tiny details, enhancement of image noise, occasional over-enhancement, and unnatural look of the processed images. In this paper, we propose a new approach for contrast enhancement based on the use of a so-called intensity-pair distribution. This distribution possesses both local information and global information of the image content. By analyzing the content of intensity-pair distribution, a set of expansion forces are generated for contrast enhancement while another set of anti-expansion forces are generated to suppress image noise. To avoid over-enhancement and preserve the natural look of the processed images, a magnitude mapping function is also proposed. Experimental results show that the proposed algorithm does provide a flexible and reliable way for contrast enhancement.
17967492
Network Game and Boosting
We propose an ensemble learning method called Network Boosting which combines weak learners together based on a random graph (network). A theoretic analysis based on the game theory shows that the algorithm can learn the target hypothesis asymptotically. The comparison results using several datasets of the UCI machine learning repository and synthetic data are promising and show that Network Boosting has much resistance to the noisy data than AdaBoost through the cooperation of classifiers in the classifier network.
2070714
A large quantity of novel human antisense transcripts detected by LongSAGE
MOTIVATION
Taking advantage of the high sensitivity and specificity of LongSAGE tag for transcript detection and genome mapping, we analyzed the 632 813 unique human LongSAGE tags deposited in public databases to identify novel human antisense transcripts.


RESULTS
Our study identified 45 321 tags that match the antisense strand of 9804 known mRNA sequences, 6606 of which contain antisense ESTs and 3198 are mapped only by SAGE tags. Quantitative analysis showed that the detected antisense transcripts are present at levels lower than their counterpart sense transcripts. Experimental results confirmed the presence of antisense transcripts detected by the antisense tags. We also constructed an antisense tag database that can be used to identify the antisense SAGE tags originated from the antisense strand of known mRNA sequences included in the RefSeq database.


CONCLUSIONS
Our study highlights the benefits of exploring SAGE data for comprehensive identification of human antisense transcripts and demonstrates the prevalence of antisense transcripts in the human genome.
4791841
Salient Boundary Detection using Ratio Contour
This paper presents a novel graph-theoretic approach, named ratio contour, to extract perceptually salient boundaries from a set of noisy boundary fragments detected in real images. The boundary saliency is defined using the Gestalt laws of closure, proximity, and continuity. This paper first constructs an undirected graph with two different sets of edges: solid edges and dashed edges. The weights of solid and dashed edges measure the local saliency in and between boundary fragments, respectively. Then the most salient boundary is detected by searching for an optimal cycle in this graph with minimum average weight. The proposed approach guarantees the global optimality without introducing any biases related to region area or boundary length. We collect a variety of images for testing the proposed approach with encouraging results.