Filtration/masking can be seen as very late fusion. For slot-filling or fact-aware language modelling, the retriever may provide relevant documents out of which a term incidence vectorA vector \(v\)  of the length of the vocabulary filled with ones and zeroes. Then \(v_i = 1\)  if the \(i\) -th term occurred in the retrieved documents, otherwise zero. is constructed. The output of the model is then masked and normalized to prevent outputs unrelated to the query \((\hat{y} = \frac{c\, \odot \, \xi }{|c\, \odot \, \xi |})\) .
An example of this is to constrain beam-search decoding to valid names only as done in the recently proposed GENRE model for entity retrieval tasks [1]}.
