Evaluating recommender systems from the user’s perspective: survey of the state of the art <s> Elicitation of feature preferences <s> This paper presents the candidate/critique model of interactive problem solving, in which an automated problem solver communicates candidate solutions to the user and the user critiques those solutions. The system starts with minimal information about the user’s preferences, and preferences are elicited and inferred incrementally by analyzing the critiques. The system’s goal is to present “good” candidates to the user, but to do so it must learn as much as possible about his preferences in order to improve its choice of candidates in subsequent iterations. This system contrasts with traditional decision-analytic and planning frameworks in which a complete model is elicited beforehand or is constructed by a human expert. The paper presents the Automated Travel Assistant, an implemented prototype of the model that interactively builds flight itineraries using realtime airline information. The ATA is available on the World Wide Web and has had over 4000 users between May and October 1996. <s> BIB001 </s> Evaluating recommender systems from the user’s perspective: survey of the state of the art <s> Elicitation of feature preferences <s> The Apt Decision agent learns user preferences in the domain of rental real estate by observing the user's critique of apartment features. Users provide a small number of criteria in the initial interaction, receive a display of sample apartments, and then react to any feature of any apartment independently, in any order. Users learn which features are important to them as they discover the details of specific apartments. The agent uses interactive learning techniques to build a profile of user preferences, which can then be saved and used in further retrievals. Because the user's actions in specifying preferences are also used by the agent to create a profile, the result is an agent that builds a profile without redundant or unnecessary effort on the user's part. <s> BIB002 </s> Evaluating recommender systems from the user’s perspective: survey of the state of the art <s> Elicitation of feature preferences <s> Recommender systems represent user preferences for the purpose of suggesting items to purchase or examine. They have become fundamental applications in electronic commerce and information access, providing suggestions that effectively prune large information spaces so that users are directed toward those items that best meet their needs and preferences. A variety of techniques have been proposed for performing recommendation, including content-based, collaborative, knowledge-based and other techniques. To improve performance, these methods have sometimes been combined in hybrid recommenders. This paper surveys the landscape of actual and possible hybrid recommenders, and introduces a novel hybrid, EntreeC, a system that combines knowledge-based recommendation and collaborative filtering to recommend restaurants. Further, we show that semantic ratings obtained from the knowledge-based part of the system enhance the effectiveness of collaborative filtering. <s> BIB003 </s> Evaluating recommender systems from the user’s perspective: survey of the state of the art <s> Elicitation of feature preferences <s> This paper presents an overview of the field of recommender systems and describes the current generation of recommendation methods that are usually classified into the following three main categories: content-based, collaborative, and hybrid recommendation approaches. This paper also describes various limitations of current recommendation methods and discusses possible extensions that can improve recommendation capabilities and make recommender systems applicable to an even broader range of applications. These extensions include, among others, an improvement of understanding of users and items, incorporation of the contextual information into the recommendation process, support for multicriteria ratings, and a provision of more flexible and less intrusive types of recommendations. <s> BIB004 </s> Evaluating recommender systems from the user’s perspective: survey of the state of the art <s> Elicitation of feature preferences <s> In a previously reported user study, we found that users were able to perform decision tradeoff tasks more efficiently and commit considerably fewer errors with the example critiquing interface than with the ranked list. We concluded that example-based search tools were likely to be useful particularly for extending the scope of consumer e-commerce to more complex products where decision making is critical. This paper presents results from a follow-up user study quantifying the benefits of tradeoff support. Users were able to refine the quality of their preference structures and improve decision accuracy by up to 57% after performing tradeoff tasks. Tradeoff support also significantly increased users' confidence in their choices. Together, these two studies show that example critiquing enables users to more accurately find what they want and be confident in their choices, while only requiring a level of effort that is comparable to the ranked list interface. <s> BIB005 </s> Evaluating recommender systems from the user’s perspective: survey of the state of the art <s> Elicitation of feature preferences <s> I. Modeling Technologies.- User Models for Adaptive Hypermedia and Adaptive Educational Systems.- User Profiles for Personalized Information Access.- Data Mining for Web Personalization.- Generic User Modeling Systems.- Web Document Modeling.- II. Adaptation Technologies.- Personalized Search on the World Wide Web.- Adaptive Focused Crawling.- Adaptive Navigation Support.- Collaborative Filtering Recommender Systems.- Content-Based Recommendation Systems.- Case-Based Recommendation.- Hybrid Web Recommender Systems.- Adaptive Content Presentation for the Web.- Adaptive 3D Web Sites.- III. Applications.- Adaptive Information for Consumers of Healthcare.- Personalization in E-Commerce Applications.- Adaptive Mobile Guides.- Adaptive News Access.- IV. Challenges.- Adaptive Support for Distributed Collaboration.- Recommendation to Groups.- Privacy-Enhanced Web Personalization.- Open Corpus Adaptive Educational Hypermedia.- Semantic Web Technologies for the Adaptive Web.- Usability Engineering for the Adaptive Web. <s> BIB006 </s> Evaluating recommender systems from the user’s perspective: survey of the state of the art <s> Elicitation of feature preferences <s> Recommender Systems (RS) suggest useful and interesting items to users in order to increase user satisfaction and online conversion rates. They typically exploit explicit or implicit user feedback such as ratings, buying records or clickstream data and apply statistical methods to derive recommendations. This paper focuses on explicitly formulated customer requirements as the sole type of user feedback. Its contribution lies in comparing different techniques such as knowledge- and utility-based methods, collaborative filtering, association rule mining as well as hybrid variants when user models consist solely of explicit customer requirements. We examine how this type of user feedback can be exploited for personalization in e-commerce scenarios. Furthermore, examples of actual online shops are developed where such contextual user information is available, demonstrating how more efficient RS configurations can be implemented. Results indicate that, especially for new users, explicit customer requirements are a useful source of feedback for personalization and hybrid configurations of collaborative and knowledge-based techniques achieve best results. <s> BIB007
Thus far, we have mainly considered methods that build users' interest profiles based on ratings, which have been commonly used via rating-based recommenders. In other types of systems such case-based and utility-based recommenders, the focus lies more on the elicitation of user preferences on product features. For instance, BIB003 and BIB004 discussed how the recommendation technology relies on the similarity between items' features and how to construct a multi-criteria preference model. Consider the example of a flight recommender; rather than asking users to rate flights that they have taken in the past, the system can ask users' criteria on airline, departure time, intermediate airport, and so on. Another way of eliciting users' preferences was to ask them to specify a set of example items that they preferred in the past, the method used by case-based recommenders. For example, Pandora (www. pandora.com), an internet radio and music recommender website, lets users start a radio station by entering the name of a song, an artist, or album. The system uses this initial feed's features to deduce what the user prefers and then recommend similar items. As a matter of fact, feature-based preference elicitation has been more popularly applied to high-risk and high-involvement products, such as cars, computers, houses, cameras, with which users rarely have experience (so it is difficult to obtain their ratings), while it is feasible to ask them to identify criteria on specific features. Furthermore, besides eliciting users' preferences on features' values (e.g., brand, price, size, weight), most related works have aimed at identifying the relative importance (i.e., weight) between different features. For example, in the ATA system, which is a flight selection/recommendation agent BIB001 ), a user's preference structure is assumed to be weighted additive independent. Each constraint (e.g., on the arrival time) is associated with a weight to indicate its importance. The Apt decision agent (used in the domain of rental apartments) also builds user model as a weighted feature vector BIB002 . Each feature of an apartment has a base weight determined as a part of domain analysis. The system allows users to freely change the weight on individual features by dragging the feature onto a slot (from −1 to 6). In a series of preference-based product search systems developed by BIB005 , 2006 BIB006 , users are also supported to provide the weight of each participating attribute (e.g., from 1 "least important" to 5 "most important"). If a user expresses a preference, but does not know at this point how important this preference is, the default value is set to "somewhat important". A preference structure is hence a set of (attribute value, weight) pairs of all participating attributes. When obtaining users' preferences, the system applies the weighted additive sum rule to compute recommendations. They performed a user study to test the interfaces in the real estate domain and found that most users were active in specifying and adjusting the weights during the whole interaction process. This type of weighted preference structure has also been proven to enable users to conduct tradeoff navigation and make effective recommendation critiques. Similar phenomena were later verified in other product domains, such as digital cameras and computers. Furthermore, such feature-based preference elicitation methods relying on explicit user feedback are especially suited for the context of providing recommendations to anonymous or first-time online users who have no item ratings available for learning BIB007 . We thus suggest: Guideline 5 Consider allowing users to indicate the relative importance of the features of a product/item that they prefer. This approach will likely help establish a more accurate preference model for decision support recommenders in high-risk product domains.