18767470
New event detection based on indexing-tree and named entity
New Event Detection (NED) aims at detecting from one or multiple streams of news stories that which one is reported on a new event (i.e. not reported previously). With the overwhelming volume of news available today, there is an increasing need for a NED system which is able to detect new events more efficiently and accurately. In this paper we propose a new NED model to speed up the NED task by using news indexing-tree dynamically. Moreover, based on the observation that terms of different types have different effects for NED task, two term reweighting approaches are proposed to improve NED accuracy. In the first approach, we propose to adjust term weights dynamically based on previous story clusters and in the second approach, we propose to employ statistics on training data to learn the named entity reweighting model for each class of stories. Experimental results on two Linguistic Data Consortium (LDC) datasets TDT2 and TDT3 show that the proposed model can improve both efficiency and accuracy of NED task significantly, compared to the baseline system and other existing systems.
4342705
Weighted Ontology-Based Search Exploiting Semantic Similarity
This paper is concerned with the problem of semantic search. By semantic search, we mean searching for instances from knowledge base. Given a query, we are to retrieve ‘relevant’ instances, including those that contain the query keywords and those that do not contain the keywords. This is contrast to the traditional approaches of generating a ranked list of documents that contain the keywords. Specifically, we first employ keyword based search method to retrieve instances for a query; then a proposed method of semantic feedback is performed to refine the search results; and then we conduct re-retrieval by making use of relations and instance similarities. To make the search more effective, we use weighted ontology as the underlying data model in which importances are assigned to different concepts and relations. As far as we know, exploiting instance similarities in search on weighted ontology has not been investigated previously. For the task of instance similarity calculation, we exploit both concept hierarchy and properties. We applied our methods to a software domain. Empirical evaluation indicates that the proposed methods can improve the search performance significantly.
26966022
Term Committee Based Event Identification within News Topics
Most previous research focus on organizing news set into flat collections of stories. However, a topic in news is more than a mere collection of stories: it is characterized by a definite structure of inter-related events. Stories within a topic usually share some terms which are related to the topic other than a specific event, so stories of different events are usually very similar to each other within a topic. To deal with this problem, we propose a new event identification method based on the term committee. We first capture some tight term clusters as term committees of potential events, and then use them to reweight the key terms in a story. The experimental results on two Linguistic Data Consortium (LDC) datasets show that the proposed method for event identification outperforms previous methods significantly.
17724018
Logical structure based semantic relationship extraction from semi-structured documents
Addressed in this paper is the issue of semantic relationship extraction from semi-structured documents. Many research efforts have been made so far on the semantic information extraction. However, much of the previous work focuses on detecting `isolated' semantic information by making use of linguistic analysis or linkage information in web pages and limited research has been done on extracting semantic relationship from the semi-structured documents. In this paper, we propose a method for semantic relationship extraction by using the logical information in the semi-structured document (semi-structured document usually has various types of structure information, e.g. a semi-structured document may be hierarchical laid out). To the best of our knowledge, extracting semantic relationships by using logical information has not been investigated previously. A probabilistic approach has been proposed in the paper. Features used in the probabilistic model have been defined.
18105952
Adapting Prime Number Labeling Scheme for Directed Acyclic Graphs
Directed Acyclic Graph(DAG) could be used for modeling subsumption hierarchies. Several labeling schemes have been proposed or tailored for indexing DAG in order to efficiently explore relationships in such hierarchy. However few of them can satisfy all the requirements in response time, space, and effect of updates simultaneously. In this paper, the prime number labeling scheme is extended for DAG. The scheme invests intrinsic mapping between integer divisibility and subsumption hierarchy, which simplifies the transitive closure computations and diminishes storage redundancy, as well as inherits the dynamic labeling ability from original scheme. Performance is further improved by introducing some optimization techniques. Our extensive experimental results show that prime number labeling scheme for DAG outperforms interval-based and prefix-based labeling schemes in most cases.
4300632
Keyword Extraction Using Support Vector Machine
This paper is concerned with keyword extraction. By keyword extraction, we mean extracting a subset of words/phrases from a document that can describe the ‘meaning' of the document. Keywords are of benefit to many text mining applications. However, a large number of documents do not have keywords and thus it is necessary to assign keywords before enjoying the benefit from it. Several research efforts have been done on keyword extraction. These methods make use of the ‘global context information', which makes the performance of extraction restricted. A thorough and systematic investigation on the issue is thus needed. In this paper, we propose to make use of not only ‘global context information', but also ‘local context information' for extracting keywords from documents. As far as we know, utilizing both ‘global context information' and ‘local context information' in keyword extraction has not been sufficiently investigated previously. Methods for performing the tasks on the basis of Support Vector Machines have also been proposed in this paper. Features in the model have been defined. Experimental results indicate that the proposed SVM based method can significantly outperform the baseline methods for keyword extraction. The proposed method has been applied to document classification, a typical text mining processing. Experimental results show that the accuracy of document classification can be significantly improved by using the keyword extraction method.
10126460
Mining common topics from multiple asynchronous text streams
Text streams are becoming more and more ubiquitous, in the forms of news feeds, weblog archives and so on, which result in a large volume of data. An effective way to explore the semantic as well as temporal information in text streams is topic mining, which can further facilitate other knowledge discovery procedures. In many applications, we are facing multiple text streams which are related to each other and share common topics. The correlation among these streams can provide more meaningful and comprehensive clues for topic mining than those from each individual stream. However, it is nontrivial to explore the correlation with the existence of asynchronism among multiple streams, i.e. documents from different streams about the same topic may have different timestamps, which remains unsolved in the context of topic mining. In this paper, we formally address this problem and put forward a novel algorithm based on the generative topic model. Our algorithm consists of two alternate steps: the first step extracts common topics from multiple streams based on the adjusted timestamps by the second step; the second step adjusts the timestamps of the documents according to the time distribution of the discovered topics by the first step. We perform these two steps alternately and a monotone convergence of our objective function is guaranteed. The effectiveness and advantage of our approach were justified by extensive empirical studies on two real data sets consisting of six research paper streams and two news article streams, respectively.
29748731
Support vector machine networks for multi-class classification
The support vector machine (SVM) has recently attracted growing interest in pattern classification due to its competitive performance. It was originally designed for two-class classification, and many researchers have been working on extensions to multiclass. In this paper, we present a new framework that adapts the SVM with neural networks and analyze the source of misclassification in guiding our preprocessing for optimization in multiclass classification. We perform experiments on the ORL database and the results show that our framework can achieve high recognition rates.
1936453
A hierarchical refinement algorithm for fully automatic gridding in spotted DNA microarray image processing
Gridding, the first step in spotted DNA microarray image processing, usually requires human intervention to achieve acceptable accuracy. We present a new algorithm for automatic gridding based on hierarchical refinement to improve the efficiency, robustness and reproducibility of microarray data analysis. This algorithm employs morphological reconstruction along with global and local rotation detection, non-parametric optimal thresholding and local fine-tuning without any human intervention. Using synthetic data and real microarray images of different sizes and with different degrees of rotation of subarrays, we demonstrate that this algorithm can detect and compensate for alignment and rotation problems to obtain reliable and robust results.
15545048
ELB-Q: A New Method for Improving the Robustness in DNA Microarray Image Quantification
Reliable and robust quantification of signal intensities is a critical step in microarray-based biomedical studies. However, traditional techniques for microarray image processing would face significant challenges if the number of pixels used for the quantification of the local background and the foreground decreases dramatically. We have developed a new method, ELB-Q, which, by design, is well suited for the image quantification of microarrays with very high density of spot layout (large number of spots arranged in unit area). In ELB-Q, a large extended local background (ELB) inter spot region excluding those ldquonoise of the backgroundrdquo pixels is used for estimating the local background, and the quantification of spot intensities (mean and median) in the putative target spot regions is performed after further excluding background pixels in these areas based on the cutoff values established during the ELB calculation. ELB-Q takes advantage of the abundant spatial information around each spot of interest, makes no assumption of the shape and size of the spots, and needs no sophisticated adjustment. We show results of image processing using ELB-Q on both the simulated data and real DNA microarrays, which compare favorably in robustness and accuracy against those obtained with Gene Pix Pro 6.0 (Axon Instruments, 1999) and the Markov random field (MRF) modeling approach (O. Demirkaya , Bioinformatics, vol. 21, pp. 2994-3000, 2005). The ELB-Q software is developed in Matlab, and is available upon request.
2257586
Applying neighborhood consistency for fast clustering and kernel density estimation
Nearest neighborhood consistency is an important concept in statistical pattern recognition, which underlies the well-known k-nearest neighbor method. In this paper, we combine this idea with kernel density estimation based clustering, and derive the fast mean shift algorithm (FMS). FMS greatly reduces the complexity of feature space analysis, resulting satisfactory precision of classification. More importantly, we show that with FMS algorithm, we are in fact relying on a conceptually novel approach of density estimation, the fast kernel density estimation (FKDE) for clustering. The FKDE combines smooth and non-smooth estimators and thus inherits advantages from both. Asymptotic analysis reveals the approximation of the FKDE to standard kernel density estimator. Data clustering and image segmentation experiments demonstrate the efficiency of FMS.