Evaluating recommender systems from the user’s perspective: survey of the state of the art <s> User contribution <s> University of TorontoThe authors summarize 35 years of empirical research ongoal-setting theory. They describe the core ﬁndings of thetheory, the mechanisms by which goals operate, modera-tors of goal effects, the relation of goals and satisfaction,and the role of goals as mediators of incentives. Theexternal validity and practical signiﬁcance of goal-settingtheory are explained, and new directions in goal-settingresearch are discussed. The relationships of goal setting toother theories are described as are the theory’s limitations. <s> BIB001 </s> Evaluating recommender systems from the user’s perspective: survey of the state of the art <s> User contribution <s> Under-contribution is a problem for many online communities. Social psychology theories of social loafing and goal-setting can provide mid-level design principles to address this problem. We tested the design principles in two field experiments. In one, members of an online movie recommender community were reminded of the uniqueness of their contributions and the benefits that follow from them. In the second, they were given a range of individual or group goals for contribution. As predicted by theory, individuals contributed when they were reminded of their uniqueness and when they were given specific and challenging goals, but other predictions were not borne out. The paper ends with suggestions and challenges for mining social science theories as well as implications for design. <s> BIB002
As BIB002 discovered, most users are social loafers rather than contributors. Motivated by such problems, researchers referred to social psychology to gain some insights; they particularly focused on the effect of community on users' behavior since people are increasingly aware of other users and contributors. One of the questions they asked was: if a user is not motivated to contribute ratings for her/his own sake, would s/he do it for the sake of the community?  developed a collective effort model (CEM) and identified some conditions under which people will contribute more to a group. For example, when users believe that their effort is important to the group's performance and their contributions to the group can be identifiable, they will work harder. The second question the researchers asked was: if people are motivated to contribute, should they be given specific goals or just be told to do their best? Two sets of large-scale user studies were conducted in the MovieLens setting BIB002 . The findings offered a rare opportunity to observe how social psychology theories can be adopted to understand design issues for recommender systems. Specifically, in their experiment, a rating campaign was launched among 830 active MovieLens users (those who have rated at least three movies). After receiving an email invitation, 397 members logged in and rated at least one movie. These users had actually rated an average of 5.4 movies in the 6 months before the invitation. One important finding from this study is that members who were told that they were going to make a unique contribution provided significantly more ratings on average (i.e., 31.53 ratings versus 25.16 for those who were not told that their contribution was unique). Another result is that users contributed lower amounts of ratings when they were told that their contribution would only benefit themselves (13.19 ratings), compared to the condition in which their contribution was stated to benefit both themselves and others (19.21 ratings). Based on this study, our design suggestion is: Guideline 8 Consider making users aware of their unique contributions to communities. This approach will likely motivate users to contribute. At Amazon, there is a feature called "be the first reviewer for this product", which can be regarded as an application of this guideline. It might make users aware of their contribution to the community. A second study reported in the same article BIB002 ) investigated whether goal-setting theories BIB001 could boost members' contributions to a community or not. According to this theory, assigning people with challenging, specific goals can lead them to achieve more. However, pushing users past a certain point will result in their performance leveling off. The experiment tested the theory's benefits and limitations in a similar email campaign, in which 834 new users participated. An interesting result of this study is that participants who were given specific goals contributed significantly more than those who were given the "do your best" goal. Moreover, they provided more ratings if they were told that they belonged to a group of "explorers", where the whole group was required to rate at least 32 movies. From this study's results, we thus propose: Guideline 9 Consider setting specific goals for users to achieve. This approach will also motivate users to contribute. In addition, if users can identify themselves within specific communities (e.g., "explorer", "comedy fan club"), they will likely contribute more.