We define S e as the sentences where some system extracted a relation and S F as the sentences that match the arguments of a fact in ∆. The sentential precision and recall is computed on a randomly sampled set of sentences from S e ∪ S F , in which each sentence is manually labeled whether it expresses any relation in R. Figure 3 shows the precision/recall curves for <cite>MULTIR</cite> with and without pseudo-relevance feedback computed on the test dataset of 1000 sentence used by <cite>Hoffmann et al. (2011)</cite> .