Towards searching as a learning process : A review of current perspectives and future directions <s> Future directions for assessing learning from online search behaviour <s> Search trails mined from browser or toolbar logs comprise queries and the post-query pages that users visit. Implicit endorsements from many trails can be useful for search result ranking, where the presence of a page on a trail increases its query relevance. Follow-ing a search trail requires user effort, yet little is known about the benefit that users obtain from this activity versus, say, sticking with the clicked search result or jumping directly to the destination page at the end of the trail. In this paper, we present a log-based study estimating the user value of trail following. We compare the relevance, topic coverage, topic diversity, novelty, and utility of full trails over that provided by sub-trails, trail origins (landing pages), and trail destinations (pages where trails end). Our findings demonstrate significant value to users in following trails, especially for certain query types. The findings have implications for the design of search systems, including trail recommendation systems that display trails on search result pages. <s> BIB001 </s> Towards searching as a learning process : A review of current perspectives and future directions <s> Future directions for assessing learning from online search behaviour <s> Traditionally, search engines have ignored the reading difficulty of documents and the reading proficiency of users in computing a document ranking. This is one reason why Web search engines do a poor job of serving an important segment of the population: children. While there are many important problems in interface design, content filtering, and results presentation related to addressing children's search needs, perhaps the most fundamental challenge is simply that of providing relevant results at the right level of reading difficulty. At the opposite end of the proficiency spectrum, it may also be valuable for technical users to find more advanced material or to filter out material at lower levels of difficulty, such as tutorials and introductory texts. We show how reading level can provide a valuable new relevance signal for both general and personalized Web search. We describe models and algorithms to address the three key problems in improving relevance for search using reading difficulty: estimating user proficiency, estimating result difficulty, and re-ranking based on the difference between user and result reading level profiles. We evaluate our methods on a large volume of Web query traffic and provide a large-scale log analysis that highlights the importance of finding results at an appropriate reading level for the user. <s> BIB002 </s> Towards searching as a learning process : A review of current perspectives and future directions <s> Future directions for assessing learning from online search behaviour <s> Imagine a physician and a patient doing a search on antibiotic resistance. Or a chess amateur and a grandmaster conducting a search on Alekhine's Defence. Although the topic is the same, arguably the two users in each case will satisfy their information needs with very different texts. Yet today search engines mostly adopt the one-size-fits-all solution, where personalization is restricted to topical preference. We found that users do not uniformly prefer simple texts, and that the text comprehensibility level should match the user's level of preparedness. Consequently, we propose to model the comprehensibility of texts as well as the users' reading proficiency in order to better explain how different users choose content for further exploration. We also model topic-specific reading proficiency, which allows us to better explain why a physician might choose to read sophisticated medical articles yet simple descriptions of SLR cameras. We explore different ways to build user profiles, and use collaborative filtering techniques to overcome data sparsity. We conducted experiments on large-scale datasets from a major Web search engine and a community question answering forum. Our findings confirm that explicitly modeling text comprehensibility can significantly improve content ranking (search results or answers, respectively). <s> BIB003 </s> Towards searching as a learning process : A review of current perspectives and future directions <s> Future directions for assessing learning from online search behaviour <s> Information about a user's domain knowledge and interest can be important signals for many information retrieval tasks such as query suggestion or result ranking. State-of-the-art user models rely on coarse-grained representations of the user's previous knowledge about a topic or domain. In this paper, we study query refinement using eye-tracking in order to gain precise and detailed insight into which terms the user was exposed to in a search session and which ones they showed a particular interest in. We measure fixations on the term level, allowing for a detailed model of user attention. To allow for a wide-spread exploitation of our findings, we generalize from the restrictive eye-gaze tracking to using more accessible signals: mouse cursor traces. Based on the public API of a popular search engine, we demonstrate how query suggestion candidates can be ranked according to traces of user attention and interest, resulting in significantly better performance than achieved by an attention-oblivious industry solution. Our experiments suggest that modelling term-level user attention can be achieved with great reliability and holds significant potential for supporting a range of traditional IR tasks. <s> BIB004
Focusing on the goal of comprehensive search will require research advances in most, if not all, of the key areas discussed here. In particular, modelling and assessing comprehensive search progress will require richer representations of learning and expertise that capture not only the nature and progress of domain expertise over time, but also potentially how this additional expertise is acquired, for example, if diverse perspectives are encountered along the learning path, or if multiple sources of evidence are gathered in support of a conclusion. Recent research that has mined the rich user interaction evidence in search trails BIB001 from log data may be one approach to characterizing important features of the learning process during search. In order to provide search results that are most appropriate for a specific user's desired level of understanding, researchers have also begun investigating search engine ranking algorithms that account for the complexity of content compared with the domain expertise of learners. These are based on user profiles and ranking algorithms that have been trained on features derived partly from extensive log data. A recent example of this is the study by Collins-Thompson et al. BIB002 , which found that text readability was a valuable relevance signal for web search, leading to significantly improved rankings. In related work, Tan et al. BIB003 defined a comprehensibility measure of content posted on the Yahoo! Answers question-answering site in order to provide personalized ranking that ranked simple vs complex technical answers depending on the user's domain expertise. In another promising direction, researchers have begun exploring additional sources of behavioural data, for example, eye-tracking devices BIB004 that can augment log files with gaze position and movement data to provide a more complete view of where and how the user is focusing their attention and learning effort. In sum, researchers are making promising advances towards models of learning intent, motivation, domain expertise and content complexity that can enable richer representations of queries, users and content for assessing learning opportunities and outcomes. Furthermore, these findings provide early evidence that implicit and explicit signals from search interaction can be used to identify learning tasks, assess motivation and predict the degree and growth of a user's domain expertise. In future work, what we learn from log-based assessment measures -while not without its limitations -could be used as an additional rich source of evidence for improving future result quality and overall search experience for personalized, learning-related search tasks.