Our proposed approach dynamically builds up a "memory" of contextualized embeddings and applies a pooling operation to distill a global contextualized embedding for each word. It requires an embed() function that produces a contextualized embedding for a given word in a sentence context (see <cite>Akbik et al. (2018)</cite> ).