18215853
Study for fusion of different sources to determine relevance
The relevance of a Web document could be measured not only by its text content, but also by some other factors such as the link connectivity and usage patterns. In previous data fusion researches, the text is the only source to determine the relevance, and only the different runs (e.g. by different retrieval models, different query or document representations) on this same source are combined. It is the purpose of this paper to investigate whether the different sources can be combined to determine the relevance with a better accuracy than any single source. We conducted a preliminary experiment to test its feasibility and effectiveness and a positive result was obtained.
13552447
Reverse mapping of referral links from storage hierarchy for Web documents
Due to the lack of back-pointers, information about the referral parent(s) of a given Web page is not usually available to Web surfers. This significantly reduces the effectiveness of Web surfing and information discovery. We propose a mechanism to locate the referral parent(s) for a given fragment of a Web document (in terms of an URL address) from the client side. The basic idea is to explore the hierarchical hyperlink structure of a Web document fragment from its storage path directory information that is embedded in its URL. Two mechanisms were proposed: exhaustibility first search (EFS) and generalization first search (GFS). Results showed that through direct mapping of the storage hierarchy path to the URL address and then performing reverse tracing along the hyperlink structure, the chance of discovering a referral parent ranged from 41% to 51%. Compared to EFS, GFS was found to reduce the average search space from 26 pages to 4 pages.
19735283
A generalized site ranking model for Web IR
Normally, the unit for a ranking model in a Web IR system is a Web page, which is, sometimes, just an information fragment. A larger unit considering the linkage information may be desired to reduce the cognitive overload for users to identify the complete information from the interconnected Web. We propose a ranking model to measure the relevance of the whole Web site. We take some illustrations to show the idea and provide evidences to indicate its effectiveness.
31062388
Content Selection Model for Adaptive Content Delivery
In order to adapt content delivery to different client capabilities and preferences, we propose a content selection model to automatically classify HTML content based on its functionality, then map client descriptions on preferences and device capabilities into our classification scheme, and finally selectively deliver the content which users want and which devices can handle. The experiment shows that our content selection model could reduce HTML object size, object latency and page latency. Therefore, it is effective in saving network resources and improving clients’ access experiences.
26301648
Data Integrity Framework and Language Support for Active Web Intermediaries
With the maturity of content adaptation technologies (such as I-CAP and OPES) in web intermediaries and of content personalization in the network, data integrity has already become a key concern when such technologies are deployed. To address this issue, we propose a data integrity framework with the following functionalities: (i) a server can specify its authorizations, (ii) active web intermediaries can provide value-added services in accordance with the server’s intentions, and more importantly, (iii) a client is facilitated to verify the received message with the server’s authorizations and intermediaries’ footprints. With the actual implementation of our data integrity framework in web (content and proxy) servers and clients, we show the feasibility and practicability of our proposed design through its low performance overhead and possible data (re)use.
36544447
What Are People Looking for in Your Web Page?
Web server log analyses usually analyze the pattern of the access. We believe that it is also very important to understand the goal of the access. In this paper, we propose to combine the log analysis with content analysis to identify information goals on individual accessed pages. We analyze the web server log to extract information goals on entry pages from anchor texts and query terms, and propagate them along users’ access paths to other linked pages. The experiment shows that our approach could find popular terms on web pages, temporal changes in these terms could reflect users’ interest shifts, and unexpected terms could sometimes indicate a design problem.
15511813
Adaptive dimension reduction for clustering high dimensional data
It is well-known that for high dimensional data clustering, standard algorithms such as EM and K-means are often trapped in a local minimum. Many initialization methods have been proposed to tackle this problem, with only limited success. In this paper we propose a new approach to resolve this problem by repeated dimension reductions such that K-means or EM are performed only in very low dimensions. Cluster membership is utilized as a bridge between the reduced dimensional subspace and the original space, providing flexibility and ease of implementation. Clustering analysis performed on highly overlapped Gaussians, DNA gene expression profiles and Internet newsgroups demonstrate the effectiveness of the proposed algorithm.
1831347
Minimum redundancy feature selection from microarray gene expression data
Selecting a small subset of genes out of the thousands of genes in microarray data is important for accurate classification of phenotypes. Widely used methods typically rank genes according to their differential expressions among phenotypes and pick the top-ranked genes. We observe that feature sets so obtained have certain redundancy and study methods to minimize it. Feature sets obtained through the minimum redundancy - maximum relevance framework represent broader spectrum of characteristics of phenotypes than those obtained through standard ranking methods; they are more robust, generalize well to unseen data, and lead to significantly improved classifications in extensive experiments on 5 gene expressions data sets.
1519914
Nonnegative Matrix Factorization and Probabilistic Latent Semantic Indexing: Equivalence Chi-Square Statistic, and a Hybrid Method
Non-negative Matrix Factorization (NMF) and Probabilistic Latent Semantic Indexing (PLSI) have been successfully applied to document clustering recently. In this paper, we show that PLSI and NMF optimize the same objective function, although PLSI and NMF are different algorithms as verified by experiments. This provides a theoretical basis for a new hybrid method that runs PLSI and NMF alternatively, each jumping out of local minima of the other method successively, thus achieving better final solution. Extensive experiments on 5 real-life datasets show relations between NMF and PLSI, and indicate the hybrid method lead to significant improvements over NMF-only or PLSI-only methods. We also show that at first order approximation, NMF is identical to χ2-statistic.
1074209
Comparative mapping of sequence-based and structure-based protein domains
BackgroundProtein domains have long been an ill-defined concept in biology. They are generally described as autonomous folding units with evolutionary and functional independence. Both structure-based and sequence-based domain definitions have been widely used. But whether these types of models alone can capture all essential features of domains is still an open question.MethodsHere we provide insight on domain definitions through comparative mapping of two domain classification databases, one sequence-based (Pfam) and the other structure-based (SCOP). A mapping score is defined to indicate the significance of the mapping, and the properties of the mapping matrices are studied.ResultsThe mapping results show a general agreement between the two databases, as well as many interesting areas of disagreement. In the cases of disagreement, the functional and evolutionary characteristics of the domains are examined to determine which domain definition is biologically more informative.
30201789
Term norm distribution and its effects on Latent Semantic Indexing
Latent Semantic Indexing (LSI) uses the singular value decomposition to reduce noisy dimensions and improve the performance of text retrieval systems. Preliminary results have shown modest improvements in retrieval accuracy and recall, but these have mainly explored small collections. In this paper we investigate text retrieval on a larger document collection (TREC) and focus on distribution of word norm (magnitude). Our results indicate the inadequacy of word representations in LSI space on large collections. We emphasize the query expansion interpretation of LSI and propose an LSI term normalization that achieves better performance on larger collections.