We randomly picked 200 unlabeled bags 5 from each of the two datasets (Riedel et al., 2010;<cite> Surdeanu et al., 2012</cite> ) generated by DS, and we manually annotate all relation mentions in these bags.