Towards searching as a learning process : A review of current perspectives and future directions <s> Measuring change in expertise over time <s> A search tactic is a set of search moves that are temporally and semantically related. The current study examined the tactics of medical students searching a factual database in microbiology. The students answered problems and searched the database on three occasions over a 9-month period. Their search moves were analyzed in terms of the changes in search terms used from one cycle to the next, using two different analysis methods.Common patterns were found in the students' search tactics; the most common approach was the specification of a concept, followed by the addition of one or more concepts, gradually narrowing the retrieved set before it was displayed. It was also found that the search tactics changed over time as the students' domain knowledge changed. These results have important implications for designers in developing systems that will support users' preferred ways of formulating searches. In addition, the research methods used (the coding scheme and the two data analysis methods--zero-order state transition matrices and maximal repeating patterns [MRP] analysis) are discussed in terms of their validity in future studies of search tactics. <s> BIB001 </s> Towards searching as a learning process : A review of current perspectives and future directions <s> Measuring change in expertise over time <s> Domain experts search differently than people with little or no domain knowledge. Previous research suggests that domain experts employ different search strategies and are more successful in finding what they are looking for than non-experts. In this paper we present a large-scale, longitudinal, log-based analysis of the effect of domain expertise on web search behavior in four different domains (medicine, finance, law, and computer science). We characterize the nature of the queries, search sessions, web sites visited, and search success for users identified as experts and non-experts within these domains. Large-scale analysis of real-world interactions allows us to understand how expertise relates to vocabulary, resource use, and search task under more realistic search conditions than has been possible in previous small-scale studies. Building upon our analysis we develop a model to predict expertise based on search behavior, and describe how knowledge about domain expertise can be used to present better results and query suggestions to users and to help non-experts gain expertise. <s> BIB002 </s> Towards searching as a learning process : A review of current perspectives and future directions <s> Measuring change in expertise over time <s> This study uses regression modeling to predict a user's domain knowledge level (DK) from implicit evidence provided by certain search behaviors. A user study (n=35) with recall-oriented search tasks in the genomic domain was conducted. A number of regression models of a person's DK, were generated using different behavior variable selection methods. The best model highlights three behavior variables as DK predictors: the number of documents saved, the average query length, and the average ranking position of documents opened. The model is validated using the split sampling method. Limitations and future research directions are discussed. <s> BIB003 </s> Towards searching as a learning process : A review of current perspectives and future directions <s> Measuring change in expertise over time <s> The Internet is the largest source of information in the world. Search engines help people navigate the huge space of available data in order to acquire new skills and knowledge. In this paper, we present an in-depth analysis of sessions in which people explicitly search for new knowledge on the Web based on the log files of a popular search engine. We investigate within-session and cross-session developments of expertise, focusing on how the language and search behavior of a user on a topic evolves over time. In this way, we identify those sessions and page visits that appear to significantly boost the learning process. Our experiments demonstrate a strong connection between clicks and several metrics related to expertise. Based on models of the user and their specific context, we present a method capable of automatically predicting, with good accuracy, which clicks will lead to enhanced learning. Our findings provide insight into how search engines might better help users learn as they search. <s> BIB004
Learning and expertise are closely connected: expertise is an evolving characteristic of users that reflects learning over time, rather than being a static property. Wildemuth BIB001 examined how a user's domain expertise, that is, their knowledge and experience in a particular profession or discipline, was reflected in their choice of search strategies. She found that domain novices tended to exhibit increasingly similar search strategies to those of more expert users as the novices learned more about the topic. Other previous work on domain knowledge and expertise BIB002 has characterized domain expertise and search behaviour in terms of metrics that can be derived from search logs, such as: Website visitation: Experts tend to visit different and more diverse websites in a subject area than novices. One measure of this is the average number of unique top-level websites on a search result page (SERP) or across clicked documents. Focus: Experts tend to explore more narrow topical spaces than non-experts. The degree to which a SERP is covered by a single topic is one measure of focus. Topics may be derived based on existing class hierarchies such as the Open Directory Project. Entropy: While focus measures degree of coverage by a single topic, entropy captures diversity across multiple topics on search results pages. Experts typically have higher focus, less diversity, and thus, lower entropy across topics. Branchiness: White et al. BIB002 found that domain experts tend to follow more 'branchy' paths than novices, returning to a previously visited point and then following a new unexplored direction. Specific path features include the search depth and quality of results along typical paths. Dwell time: This is the time a user spends reading a search result document. White et al. BIB002 found that experts tend to spend less time per page during a search, perhaps because they are more adept at making assessments of technical or specialized content. A more general category of features measures the pace of interaction for search and browsing (e.g. time until the first question is asked, or time until the first document is saved). Query and session complexity: The use of domain-specific, technical, or specialized terms in queries can be associated with expert vs non-expert searchers. More generally, characteristics such as the number of queries submitted and query reformulation patterns may also be used. White et al. BIB002 identified expert users in medicine, finance, law and computer science by their visitation patterns to a pre-determined set of specialist websites, which were identified in turn from discussion with domain experts in each subject area. The non-expert users selected for comparison were those who also showed interest in the same topics according to their search behaviour. In related work, Zhang et al. BIB003 characterized domain expertise in terms of similar classes of features derived from search behaviour. Previous work on modelling domain expertise in searching typically focused on longer-term behaviour patterns across sessions. However, in one of the first large-scale search log-based studies to examine session-level features of tasks where people are explicitly searching for new knowledge, Eickhoff et al. BIB004 looked at within-session changes for these expertise metrics. They focused on two specific types of knowledge acquisition: procedural knowledge (how to do something) vs declarative knowledge (knowing facts about something). The authors found evidence both for learning progress within single sessions and for persistence of learning across sessions. Significant proportions of new query terms came from result page snippets and recently visited pages, showing that the search process itself contributed to augmenting the user's domain knowledge.