As shown in previous work <cite>(Su et al., 2018)</cite> , on NYT dataset, due to a significant amount of false negatives, the PR curve on the held-out set may not be an accurate measure of performance. Therefore, we mainly employ manual evaluation.