5650583
Bipartite graph partitioning and data clustering
Many data types arising from data mining applications can be modeled as bipartite graphs, examples include terms and documents in a text corpus, customers and purchasing items in market basket analysis and reviewers and movies in a movie recommender system. In this paper, we propose a new data clustering method based on partitioning the underlying bipartite graph. The partition is constructed by minimizing a normalized sum of edge weights between unmatched pairs of vertices of the bipartite graph. We show that an approximate solution to the minimization problem can be obtained by computing a partial singular value decomposition (SVD) of the associated edge weight matrix of the bipartite graph. We point out the connection of our clustering algorithm to correspondence analysis used in multivariate analysis. We also briefly discuss the issue of assigning data objects to multiple clusters. In the experimental results, we apply our clustering algorithm to the problem of document clustering to illustrate its effectiveness and efficiency.
807817
A spectral method to separate disconnected and nearly-disconnected web graph components
Separation of connected components from a graph with disconnected graph components mostly use breadth-first search (BFS) or depth-first search (DFS) graph algorithms. Here we propose a new algebraic method to separate disconnected and nearly-disconnected components. This method is based on spectral graph partitioning, following a key observation that disconnected components will show up, after properly sorted, as step-function like curve in the lowest eigenvectors of the Laplacian matrix of the graph. Following an perturbative analysis framework, we systematically analyzed the graph structures, first on the disconnected subgraph case, and second on the effects of adding edges sparsely connecting different subgraphs as a perturbation. Several new results are derived, providing insights to spectral methods and related clustering objective function. Examples are given illustrating the concepts and results our methods. Comparing to the standard graph algorithms, this method has the same O(‖E ‖ + ‖V‖log(‖V‖)) complexity, but is easier to implement (using readily available eigensolvers). Further more the method can easily identify articulation points and bridges on nearly-disconnected graphs. Segmentation of a real example of Web graph for query amazon is given. We found that each disconnected or nearly-disconnected components forms a cluster on a clear topic.
4990172
Structure search and stability enhancement of Bayesian networks
Learning Bayesian network structure from large-scale data sets, without any expert-specified ordering of variables, remains a difficult problem. We propose systematic improvements to automatically learn Bayesian network structure from data. (1) We propose a linear parent search method to generate candidate graph. (2) We propose a comprehensive approach to eliminate cycles using minimal likelihood loss, a short cycle first heuristic, and a cut-edge repairing. (3) We propose structure perturbation to assess the stability of the network and a stability-improvement method to refine the network structure. The algorithms are easy to implement and efficient for large networks. Experimental results on two data sets show that our new approach outperforms existing methods.
1876538
PSoL: a positive sample only learning algorithm for finding non-coding RNA genes
MOTIVATION
Small non-coding RNA (ncRNA) genes play important regulatory roles in a variety of cellular processes. However, detection of ncRNA genes is a great challenge to both experimental and computational approaches. In this study, we describe a new approach called positive sample only learning (PSoL) to predict ncRNA genes in the Escherichia coli genome. Although PSoL is a machine learning method for classification, it requires no negative training data, which, in general, is hard to define properly and affects the performance of machine learning dramatically. In addition, using the support vector machine (SVM) as the core learning algorithm, PSoL can integrate many different kinds of information to improve the accuracy of prediction. Besides the application of PSoL for predicting ncRNAs, PSoL is applicable to many other bioinformatics problems as well.


RESULTS
The PSoL method is assessed by 5-fold cross-validation experiments which show that PSoL can achieve about 80% accuracy in recovery of known ncRNAs. We compared PSoL predictions with five previously published results. The PSoL method has the highest percentage of predictions overlapping with those from other methods.
45840953
Dynamic Cluster Formation Using Level Set Methods
Density-based clustering has the advantages for (i) allowing arbitrary shape of cluster and (ii) not requiring the number of clusters as input. However, when clusters touch each other, both the cluster centers and cluster boundaries (as the peaks and valleys of the density distribution) become fuzzy and difficult to determine. In higher dimension, the boundaries become wiggly and over-fitting often occurs. We introduce the notion of cluster intensity function (CIF) which captures the important characteristics of clusters. When clusters are well-separated, CIFs are similar to density functions. But as clusters touch each other, CIFs still clearly reveal cluster centers, cluster boundaries, and, degree of membership of each data point to the cluster that it belongs. Clustering through bump hunting and valley seeking based on these functions are more robust than that based on kernel density functions which are often oscillatory or over-smoothed. These problems of kernel density estimation are resolved using Level Set Methods and related techniques. Comparisons with two existing density-based methods, valley seeking and DBSCAN, are presented to illustrate the advantages of our approach.
12184758
Integrating program component executables on distributed memory architectures via MPH
Summary form only given. A growing trend in developing large and complex applications on today's Teraflop computers is to integrate stand-alone and/or semiindependent program components into a comprehensive simulation package. One example is the climate system model, which consists of atmosphere, ocean, land-surface and sea-ice. Each component is semiindependent and has been developed at different institutions. We study how this multicomponent multiexecutable application can run effectively on distributed memory architectures. We identify five effective execution modes and develop the MPH library to support application developments utilizing these modes. MPH performs component-name registration, resource allocation and initial component handshaking in a flexible way.
13344452
Context query in information retrieval
There is an important query requirement missing for search engines. With the wide variation of domain knowledge and user interest, a user would like to retrieve documents in which one query term is discussed in the context of another. Based on existing query mechanisms, what can be specified at most is the co-occurrence of multiple terms in a query. This is insufficient because the co-occurrence of two terms does not necessarily mean that one is discussed in the context of the other. In this paper we propose the context query for Web searching. A new query operator, called the 'in' operator, is used to specify context inclusion between two terms. Heuristic rules to identify context inclusion are suggested and implementation of the 'in' operator in search engines is proposed. Results show that both the precision and ranking relevance of Web searching are improved significantly.
36544447
What Are People Looking for in Your Web Page?
Web server log analyses usually analyze the pattern of the access. We believe that it is also very important to understand the goal of the access. In this paper, we propose to combine the log analysis with content analysis to identify information goals on individual accessed pages. We analyze the web server log to extract information goals on entry pages from anchor texts and query terms, and propagate them along users’ access paths to other linked pages. The experiment shows that our approach could find popular terms on web pages, temporal changes in these terms could reflect users’ interest shifts, and unexpected terms could sometimes indicate a design problem.
209398610
Beyond the traditional query operators.
9546816
A Framework of HTML Content Selection and Adaptive Delivery
In this paper, we propose a content selection and adaptive delivery framework to adapt content delivery to different client capabilities and preferences. The framework provides language support to content publishers with XML-based markup language to describe the content and control the transformation and delivery process. Automatic content classification and selective delivery based on match between content class and user capability or preference are also implemented. Thus, when content descriptions from server are not available, adaptive content delivery is still possible. The framework is generic and scalable. Performance studies on content classification and selection model show that it is effective in saving network resources and improving clients’ access experiences.
21949746
An Improved Usage-Based Ranking
A good ranking is critical to gain a positive searching experience. With usage data collected from past searching activities, it could be improved from current approaches which are largely based on text or link information. In this paper, we proposed a usage-based ranking algorithm. Basically, it calculates the rank score on time duration considering the propagated effect, which is an improvement on the simple selection frequency determined method. Besides, it also has some heuristics to further improve the accuracy of top-positioned results.