We implemented and evaluated a two-stage retrieval method for personalized academic search in which the initial search results are re-ranked using an author-topic profile. In academic search tasks, the user's own data can help optimizing the ranking of search results to match the searcher's specific individual needs. The author-topic profile consists of topic-specific terms, stored in a graph. We re-rank the top-1000 retrieved documents using ten features that represent the similarity between the document and the author-topic graph. We found that the re-ranking gives a small but significant improvement over the reproduced best method from the literature. Storing the profile as a graph has a number of advantages: it is flexible with respect to node and relation types; it is a visualization of knowledge that is interpretable by the user, and it offers the possibility to view relational characteristics of individual nodes.
We use the iSearch data @cite_29 for our experiments. The collection consists of 65 natural search tasks (topics) from 23 scholars from university departments of physics. The search task description form had five fields that the searchers filled in before they started to search for answers: In our experiments we use these fields to populate the author--topic profile. The fields (a--d) contain roughly between 10 and 40 words each; field e is much shorter, listing between of 3 and 10 search terms.
@cite_29
1928047241
The poster discusses the characteristics needed in an information retrieval (IR) test collection to facilitate the evaluation of integrated search, i.e. search across a range of different sources but with one search box and one ranked result list, and describes and analyses a new test collection constructed for this purpose. The test collection consists of approx. 18,000 monographic records, 160,000 papers and journal articles in PDF and 275,000 abstracts with a varied set of metadata and vocabularies from the physics domain, 65 topics based on real work tasks and corresponding graded relevance assessments. The test collection may be used for systems- as well as user-oriented evaluation.