Interestingly, the Freebase held-out metric<cite> (Mintz et al., 2009</cite>; Hoffmann et al., 2011 ) turns out to be heavily biased toward distantly labeled data (e.g., increasing human feedback hurts precision; see Section 4.6).