<cite>(Tetreault and Chodorow, 2008b)</cite> showed that trained human raters can achieve very high agreement (78%) on this task. We replicate this experiment not with trained raters but with the AMT to answer two research questions: 1. Can untrained raters be as effective as trained 46 raters? 2. If so, how many raters does it take to match trained raters?