4990172
Structure search and stability enhancement of Bayesian networks
Learning Bayesian network structure from large-scale data sets, without any expert-specified ordering of variables, remains a difficult problem. We propose systematic improvements to automatically learn Bayesian network structure from data. (1) We propose a linear parent search method to generate candidate graph. (2) We propose a comprehensive approach to eliminate cycles using minimal likelihood loss, a short cycle first heuristic, and a cut-edge repairing. (3) We propose structure perturbation to assess the stability of the network and a stability-improvement method to refine the network structure. The algorithms are easy to implement and efficient for large networks. Experimental results on two data sets show that our new approach outperforms existing methods.
17669185
Unsupervised Learning: Self-aggregation in Scaled Principal Component Space
We demonstrate that data clustering amounts to a dynamic process of self-aggregation in which data objects move towards each other to form clusters, revealing the inherent pattern of similarity. Self-aggregation is governed by connectivity and occurs in a space obtained by a nonlinear scaling of principal component analysis (PCA). The method combines dimensionality reduction with clustering into a single framework. It can apply to both square similarity matrices and rectangular association matrices.
841061
Contraction graphs for representation and analysis of RNA secondary structure
We introduce the contraction graph as a novel graphical representation of RNA secondary structure. A contraction operation - defined along base pairs, single stranded regions, and tertiary interactions - allows for representing RNA secondary structure at different levels of detail. Directionality of the graph and flow conservation of the contraction operation allow for explicit representation of the 5' and 3' ends of the molecule. Furthermore, existing representations such as tree and dual graphs are specific instances in this framework. We anticipate that this unified representation will facilitate automated motif finding and functional classification based on direct searches and comparisons of RNA secondary structure topology.
45840953
Dynamic Cluster Formation Using Level Set Methods
Density-based clustering has the advantages for (i) allowing arbitrary shape of cluster and (ii) not requiring the number of clusters as input. However, when clusters touch each other, both the cluster centers and cluster boundaries (as the peaks and valleys of the density distribution) become fuzzy and difficult to determine. In higher dimension, the boundaries become wiggly and over-fitting often occurs. We introduce the notion of cluster intensity function (CIF) which captures the important characteristics of clusters. When clusters are well-separated, CIFs are similar to density functions. But as clusters touch each other, CIFs still clearly reveal cluster centers, cluster boundaries, and, degree of membership of each data point to the cluster that it belongs. Clustering through bump hunting and valley seeking based on these functions are more robust than that based on kernel density functions which are often oscillatory or over-smoothed. These problems of kernel density estimation are resolved using Level Set Methods and related techniques. Comparisons with two existing density-based methods, valley seeking and DBSCAN, are presented to illustrate the advantages of our approach.
12184758
Integrating program component executables on distributed memory architectures via MPH
Summary form only given. A growing trend in developing large and complex applications on today's Teraflop computers is to integrate stand-alone and/or semiindependent program components into a comprehensive simulation package. One example is the climate system model, which consists of atmosphere, ocean, land-surface and sea-ice. Each component is semiindependent and has been developed at different institutions. We study how this multicomponent multiexecutable application can run effectively on distributed memory architectures. We identify five effective execution modes and develop the MPH library to support application developments utilizing these modes. MPH performs component-name registration, resource allocation and initial component handshaking in a flexible way.
807817
A spectral method to separate disconnected and nearly-disconnected web graph components
Separation of connected components from a graph with disconnected graph components mostly use breadth-first search (BFS) or depth-first search (DFS) graph algorithms. Here we propose a new algebraic method to separate disconnected and nearly-disconnected components. This method is based on spectral graph partitioning, following a key observation that disconnected components will show up, after properly sorted, as step-function like curve in the lowest eigenvectors of the Laplacian matrix of the graph. Following an perturbative analysis framework, we systematically analyzed the graph structures, first on the disconnected subgraph case, and second on the effects of adding edges sparsely connecting different subgraphs as a perturbation. Several new results are derived, providing insights to spectral methods and related clustering objective function. Examples are given illustrating the concepts and results our methods. Comparing to the standard graph algorithms, this method has the same O(‖E ‖ + ‖V‖log(‖V‖)) complexity, but is easier to implement (using readily available eigensolvers). Further more the method can easily identify articulation points and bridges on nearly-disconnected graphs. Segmentation of a real example of Web graph for query amazon is given. We found that each disconnected or nearly-disconnected components forms a cluster on a clear topic.
13344452
Context query in information retrieval
There is an important query requirement missing for search engines. With the wide variation of domain knowledge and user interest, a user would like to retrieve documents in which one query term is discussed in the context of another. Based on existing query mechanisms, what can be specified at most is the co-occurrence of multiple terms in a query. This is insufficient because the co-occurrence of two terms does not necessarily mean that one is discussed in the context of the other. In this paper we propose the context query for Web searching. A new query operator, called the 'in' operator, is used to specify context inclusion between two terms. Heuristic rules to identify context inclusion are suggested and implementation of the 'in' operator in search engines is proposed. Results show that both the precision and ranking relevance of Web searching are improved significantly.
26301648
Data Integrity Framework and Language Support for Active Web Intermediaries
With the maturity of content adaptation technologies (such as I-CAP and OPES) in web intermediaries and of content personalization in the network, data integrity has already become a key concern when such technologies are deployed. To address this issue, we propose a data integrity framework with the following functionalities: (i) a server can specify its authorizations, (ii) active web intermediaries can provide value-added services in accordance with the server’s intentions, and more importantly, (iii) a client is facilitated to verify the received message with the server’s authorizations and intermediaries’ footprints. With the actual implementation of our data integrity framework in web (content and proxy) servers and clients, we show the feasibility and practicability of our proposed design through its low performance overhead and possible data (re)use.
10076076
Quantitative Modeling for Web Objects' Cacheability
In this paper, we propose a mathematical model, called the E-Cacheability Index, to quantify and compare the effectiveness of caching web objects on Internet. Unlike previous related work on web object cacheability, our study is based on a comprehensive study of all object attributes related to the HTTP protocol transfer, content reuse, freshness re-validation as well as the inter-relationship/dependency among these attributes in the final caching decision. This model is important because it not only estimates the cacheability effectiveness of web content, but also provides hints on web caching optimization.
19735283
A generalized site ranking model for Web IR
Normally, the unit for a ranking model in a Web IR system is a Web page, which is, sometimes, just an information fragment. A larger unit considering the linkage information may be desired to reduce the cognitive overload for users to identify the complete information from the interconnected Web. We propose a ranking model to measure the relevance of the whole Web site. We take some illustrations to show the idea and provide evidences to indicate its effectiveness.
1490908
Word segmentation and recognition for web document framework
It is observed that a better approach to Web information understanding is to base on its document framework, which is mainly consisted of (i) the title and the URL name of the page, (ii) the titles and the URL names of the Web pages that it points to, (iii) the alternative information source for the embedded Web objects, and (iv) its linkage to other Web pages of the same document. Investigation reveals that a high percentage of words inside the document framework are “compound words” which cannot be understood by ordinary dictionaries. They might be abbreviations or acronyms, or concatenations of several (partial) words. To recover the content hierarchy of Web documents, we propose a new word segmentation and recognition mechanism to understand the information derived from the Web document framework. A maximal bi-directional matching algorithm with heuristic rules is used to resolve ambiguous segmentation and meaning in compound words. An adaptive training process is further employed to build a dictionary of recognisable abbreviations and acronyms. Empirical results show that over 75% of the compound words found in the Web document framework can be understood by our mechanism. With the training process, the success rate of recognising compound words can be increased to about 90%.