An experimental study on the retrieval effectiveness of collaborative tags
This study investigated the retrieval effectiveness of tagging through experimental tests. We developed a test collection, a list of topics and corresponding relevance judgments using data collected from Pubmed and CiteULike. Two retrieval test runs were carried out, one without tags (baseline) and one with tags (tag run). General improvements in retrieval performance were observed from the tag run in terms of mean average precision and the difference is statistically significant though small. As revealed by the 11-point precision recall chart, the tag run retrieved more relevant documents at the top of the result list overall. The initial findings suggest that the use of tags in information retrieval has promise, especially in ranking the top retrieved documents.
1 Summary For Case Western Reserve University’s debut participation in TREC, we chose to participate in the Enterprise Track expert search task. Our motivation for participation stems from our work developing expert search capability for our prototype vertical digital library, MEMS World Online 1 . Our work incorporates two unique aspects. First, our relevance ranking mechanism relies on term position within each document rather than the number of term occurrences. This mechanism takes into account both term document rank and term co-occurrence proximity. Second, the expert score of closely related colleagues has a small eect on the score of each related expert. This follows the intuition that experts on a particular topic within a single organization tend to closely collaborate with one another. We also make some use of WordNet synonyms. We submitted a total of three runs to this years expert search task.