Massive Open Online Courses (MOOCs) use peer assessment to grade open ended questions at scale, allowing students to provide feedback. Relative to teacher based grading, peer assessment on MOOCs traditionally delivers lower quality feedback and fewer learner interactions. We present the identified peer review (IPR) framework, which provides non-blind peer assessment and incentives driving high quality feedback. We show that, compared to traditional peer assessment methods, IPR leads to significantly longer and more useful feedback as well as more discussion between peers.
Feedback improves performance by changing students' locus of attention, focusing them on productive aspects of their work @cite_20 @cite_11 . However, not all peers provide great feedback and some leave limited comments with no coherent message for improvement, or they are rogue reviews @cite_1 . Rogue reviews are insufficient reviews caused by laziness, collusion, dishonesty, ret aliation, competition, or malevolence @cite_23 . To improve on this, PeerStudio @cite_4 peer assessment system is designed to encourage more feedback comments by showing short tips for writing comments just below the comment box. For example, if a response has no constructive feedback, it may remind students with phrases like: ''Quick check: Is your feedback actionable?'' by triggering heuristics word count on feedback @cite_4 . Students see such comments as more useful than rubrics in reviewing @cite_22 . Similar techniques are used to improve the quality of product reviews online @cite_24 . Our framework uses a simple interface design reflecting these lessons by integrating four pointed questions with separate response areas.
@cite_4 @cite_22 @cite_1 @cite_24 @cite_23 @cite_20 @cite_11
2038924341 2168105487 2116119142 2087294982 1607629406 2030441548
Rapid feedback is a core component of mastery learning, but feedback on open-ended work requires days or weeks in most classes today. This paper introduces PeerStudio, an assessment platform that leverages the large number of students' peers in online classes to enable rapid feedback on in-progress work. Students submit their draft, give rubric-based feedback on two peers' drafts, and then receive peer feedback. Students can integrate the feedback and repeat this process as often as they desire. In MOOC deployments, the median student received feedback in just twenty minutes. Rapid feedback on in-progress work improves course outcomes: in a controlled experiment, students' final grades improved when feedback was delivered quickly, but not if delayed by 24 hours. More than 3,600 students have used PeerStudio in eight classes, both massive and in-person. This research demonstrates how large classes can leverage their scale to encourage mastery through rapid feedback and revision. When students work with peers, they learn more actively, build richer knowledge structures, and connect material to their lives. However, not every peer learning experience online sees successful adoption. This paper articulates and addresses three adoption challenges for global-scale peer learning. First, peer interactions struggle to bootstrap critical mass. However, class incentives can signal importance and spur initial usage. Second, online classes have limited peer visibility and awareness, so students often feel alone even when surrounded by peers. We find that highlighting interdependence and strengthening norms can mitigate this issue. Third, teachers can readily access "big" aggregate data but not "thick" contextual data that helps build intuitions, so software should guide teachers' scaffolding of peer interactions. We illustrate these challenges through studying 8,500 students' usage of two peer learning platforms, Talkabout and PeerStudio. This paper measures efficacy through sign-up and participation rates and the structure and duration of student interactions. Scientific peer review, open source software development, wikis, and other domains use distributed review to improve quality of created content by providing feedback to the work's creator. Distributed review is used to assess or improve the quality of a work (e.g., an article). However, it can also provide learning benefits to the participants in the review process. We developed an online review system for beginning computer programming students; it gathers multiple anonymous peer reviews to give students feedback on their programming work. We deployed the system in an introductory programming class and evaluated it in a controlled study. We find that: peer reviews are accurate compared to an accepted evaluation standard, that students prefer reviews from other students with less experience than themselves, and that participating in a peer review process results in better learning outcomes. User-supplied reviews are widely and increasingly used to enhance e-commerce and other websites. Because reviews can be numerous and varying in quality, it is important to assess how helpful each review is. While review helpfulness is currently assessed manually, in this paper we consider the task of automatically assessing it. Experiments using SVM regression on a variety of features over Amazon.com product reviews show promising results, with rank correlations of up to 0.66. We found that the most useful features include the length of the review, its unigrams, and its product rating. With demonstrated benefits to higher level learning, peer review in the classroom has been well researched and popular since at least the 1990s. However, little or no prior studies exist into the peer review process for online courses. Further, we found no prior research specifically addressing the operational aspects of online peer review. This research addresses that gap by comparing the issues involved in managing peer review for an online course with those for a traditional classroom course. In an exploratory case study, two sections of the same introductory level course were taught by the same professor in the same academic term, one section in the traditional classroom and one as an online section. Both sections covered the same material in the same order. Online students had access to narrated PowerPoint recordings that tracked in-class lectures. The same assignments and exams were used. The two sections used a joint discussion board for posting questions and answers about the course material. In short, the two courses were almost identical, except for the steps necessary to make peer review operate in an online environment.