89684
Semi-Supervised Conditional Random Fields for Improved Sequence Segmentation and Labeling
We present a new semi-supervised training procedure for conditional random fields (CRFs) that can be used to train sequence segmentors and labelers from a combination of labeled and unlabeled training data. Our approach is based on extending the minimum entropy regularization framework to the structured prediction case, yielding a training objective that combines unlabeled conditional entropy with labeled conditional likelihood. Although the training objective is no longer concave, it can still be used to improve an initial model (e.g. obtained from supervised training) by iterative ascent. We apply our new training algorithm to the problem of identifying gene and protein mentions in biological texts, and show that incorporating unlabeled data improves the performance of the supervised CRF in this case.
6812524
Stochastic Analysis of Lexical and Semantic Enhanced Structural Language Model
In this paper, we present a directed Markov random field model that integrates trigram models, structural language models (SLM) and probabilistic latent semantic analysis (PLSA) for the purpose of statistical language modeling. The SLM is essentially a generalization of shift-reduce probabilistic push-down automata thus more complex and powerful than probabilistic context free grammars (PCFGs). The added context-sensitiveness due to trigrams and PLSAs and violation of tree structure in the topology of the underlying random field model make the inference and parameter estimation problems plausibly intractable, however the analysis of the behavior of the lexical and semantic enhanced structural language model leads to a generalized inside-outside algorithm and thus to rigorous exact EM type re-estimation of the composite language model parameters.
662359
Language and Task Independent Text Categorization with Simple Language Models
We present a simple method for language independent and task independent text categorization learning, based on character-level n-gram language models. Our approach uses simple information theoretic principles and achieves effective performance across a variety of languages and tasks without requiring feature selection or extensive pre-processing. To demonstrate the language and task independence of the proposed technique, we present experimental results on several languages---Greek, English, Chinese and Japanese---in several text categorization problems---language identification, authorship attribution, text genre classification, and topic detection. Our experimental results show that the simple approach achieves state of the art performance in each case.
1771247
Using query-specific variance estimates to combine Bayesian classifiers
Many of today's best classification results are obtained by combining the responses of a set of base classifiers to produce an answer for the query. This paper explores a novel "query specific" combination rule: After learning a set of simple belief network classifiers, we produce an answer to each query by combining their individual responses, using weights based inversely on their respective variances around their responses. These variances are based on the uncertainty of the network parameters, which in turn depend on the training datasample. In essence, this variance quantifies the base classifier's confidence of its response to this query. Our experimental results show that these "mixture-using-variance belief net classifiers" MUVS work effectively, especially when the base classifiers are learned using balanced bootstrap samples and when their results are combined using James-Stein shrinkage. We also found that our variance-based combination rule performed better than both bagging and AdaBoost, even on the set of base classifiers produced by AdaBoost itself. Finally, this framework is extremely efficient, as both the learning and the classification components require only straight-line code.
210649
Combining Statistical Language Models via the Latent Maximum Entropy Principle
We present a unified probabilistic framework for statistical language modeling which can simultaneously incorporate various aspects of natural language, such as local word interaction, syntactic structure and semantic document information. Our approach is based on a recent statistical inference principle we have proposed—the latent maximum entropy principle—which allows relationships over hidden features to be effectively captured in a unified model. Our work extends previous research on maximum entropy methods for language modeling, which only allow observed features to be modeled. The ability to conveniently incorporate hidden variables allows us to extend the expressiveness of language models while alleviating the necessity of pre-processing the data to obtain explicitly observed features. We describe efficient algorithms for marginalization, inference and normalization in our extended models. We then use these techniques to combine two standard forms of language models: local lexical models (Markov N-gram models) and global document-level semantic models (probabilistic latent semantic analysis). Our experimental results on the Wall Street Journal corpus show that we obtain a 18.5% reduction in perplexity compared to the baseline tri-gram model with Good-Turing smoothing.
12212021
Variational Bayesian image modelling
We present a variational Bayesian framework for performing inference, density estimation and model selection in a special class of graphical models---Hidden Markov Random Fields (HMRFs). HMRFs are particularly well suited to image modelling and in this paper, we apply them to the problem of image segmentation. Unfortunately, HMRFs are notoriously hard to train and use because the exact inference problems they create are intractable. Our main contribution is to introduce an efficient variational approach for performing approximate inference of the Bayesian formulation of HMRFs, which we can then apply to the density estimation and model selection problems that arise when learning image models from data. With this variational approach, we can conveniently tackle the problem of image segmentation. We present experimental results which show that our technique outperforms recent HMRF-based segmentation methods on real world images.
27992571
Segmentation of SAR images
Abstract This paper presents a new algorithm for segmentation of SAR images based on threshold estimation using the histogram. The speckle distribution in the SAR image is modeled by a Gamma function. Thus, the SAR image histogram exhibits a combination of Gamma distributions. The maximum likelihood technique is therefore used to estimate the histogram parameters. This technique requires knowledge of the number of modes of the histogram, the number of looks of the SAR image, and the initial parameters of the histogram. The second derivative of the histogram is used to estimate the number of modes. We use two methods to estimate the number of looks. Initial parameters are estimated at the maximum of the Gamma function. Thresholds are selected at the valleys of a multi-modal histogram by minimizing the discrimination error between the classes of pixels in the image. The algorithm is applied to several RADARSAT SAR images with different number of looks. The results obtained are promising.
6116737
Domain Knowledge Extracting in a Chinese Natural Language Interface to Databases: NChiql
This paper presents the method of domain knowledge extracting in NChiql, a Chinese natural language interface to databases. After describing the overall extracting strategy in NChiql, we mainly discuss the basic semantic information extracting method, called DSE. A semantic conceptual graph is employed to specify two types of modification and three types of verbal relationship among the entities, relationships and attributes. Compared with related works. DSE has more strongly extracting ability.
9573239
A new algorithm for inexact graph matching
The graph is an essential data structure for representing relational information. When graphs are used to represent objects, comparing objects amounts to graph matching. Inexact graph matching is the process of finding the best possible matching between two graphs when exact matching is impossible. We propose a new algorithm for the inexact matching problem. The new algorithm decomposes the matching process into K phases, where the value of K ranges from 1 to the minimum of the numbers of nodes in the two graphs to be matched. The efficiency of the new algorithm results from the use of small values of K, significantly reducing the search space while still producing very good matchings (most of them optimal) between graphs. The algorithm is compared with the error-correcting subgraph isomorphism algorithm based on A*.
14809978
Image contrast enhancement based on intensity-pair distribution
Current contrast enhancement algorithms sometimes come with undesired drawbacks, like the loss of tiny details, enhancement of image noise, occasional over-enhancement, and unnatural look of the processed images. In this paper, we propose a new approach for contrast enhancement based on the use of a so-called intensity-pair distribution. This distribution possesses both local information and global information of the image content. By analyzing the content of intensity-pair distribution, a set of expansion forces are generated for contrast enhancement while another set of anti-expansion forces are generated to suppress image noise. To avoid over-enhancement and preserve the natural look of the processed images, a magnitude mapping function is also proposed. Experimental results show that the proposed algorithm does provide a flexible and reliable way for contrast enhancement.
9413098
A rotation invariant printed Chinese character recognition system
Abstract In this paper, we present an approach to construct an optical character recognition (OCR) system for recognizing rotated printed Chinese characters. Our system includes preprocessing, feature extraction, candidates selection, and target character recognition. The proposed system has a three-stage structure designed mainly to reduce the time complexity in the recognition process. Our goal is to recognize the complete set of frequently used 13 053 printed Chinese characters with arbitrary orientations. The overall recognition rate of our system reaches 97.4%.