In this section we briefly introduce Dynamic Memory Networks [1]}, [2]}, which we use to aggregate BERT outputs. DMNs take as input a sequence of words \(w = (w_1, ..., w_T)\) , which usually represent multiple sentences such as a document or a passage, and a question \(q = (q_1, ..., q_N)\) . They are composed of four modules (cf. Figure REF ):
