One goal of online social recommendation systems is to harness the wisdom of crowds in order to identify high quality content. Yet the sequential voting mechanisms that are commonly used by these systems are at odds with existing theoretical and empirical literature on optimal aggregation. This literature suggests that sequential voting will promote herding---the tendency for individuals to copy the decisions of others around them---and hence lead to suboptimal content recommendation. Is there a problem with our practice, or a problem with our theory? Previous attempts at answering this question have been limited by a lack of objective measurements of content quality. Quality is typically defined endogenously as the popularity of content in absence of social influence. The flaw of this metric is its presupposition that the preferences of the crowd are aligned with underlying quality. Domains in which content quality can be defined exogenously and measured objectively are thus needed in order to better assess the design choices of social recommendation systems. In this work, we look to the domain of education, where content quality can be measured via how well students are able to learn from the material presented to them. Through a behavioral experiment involving a simulated massive open online course (MOOC) run on Amazon Mechanical Turk, we show that sequential voting systems can surface better content than systems that elicit independent votes.
To deal with the huge amount of potentially interesting content on the web today, users seek the help of curators to recommend which content to consume. The two most common forms of curation are expert-based (the editor of a newspaper decides which articles to place on the front page), and algorithmic-based (a search algorithm determines the ranking of websites for a given query). In recent years, content aggregators which use explicit vote-based feedback to curate content for future users have grown exponentially in popularity. The goal of this paper is to provide a descriptive analysis of these crowdsourced curation mechanisms. In particular, we study crowd-curation mechanisms that rank articles according to a score which is a function of userfeedback. We precisely quantify the dynamics of which articles become popular in such these systems. While crowdcuration can be relatively eective for cardinal objectives like discovering and promoting content of high quality, they do not perform well for ordinal objectives such as nding the best articles. Our analysis suggests that user preferences and behavior are a far greater determinant of curation quality than the actual details of the curation mechanism. Finally, we show that certain shifts in user voting behavior can have positive impacts on these systems, suggesting that active moderation of user behavior is important for high quality curation in crowd-sourced systems.