
Modify the input embeddings for a single-turn MRC model to incorporate history. This is done by either appending the entire conversation to the question, such as Roberta+KD which uses RoBERTa [1]} as the base model and truncates query if it exceeds the limit, or add special embeddings to highlight conversational history for the model, such as HAE [2]} embeds history answer embeddings with each context token if it is present in any of the history turns (detailed in section REF ). This approach does not effectively use the model to capture interactions between every dialog-turn and context.

Use separate model for each conversational turn to capture one-to-one interaction between history and context, and merge the per-turn contextualized embeddings into aggregated history-aware embeddings. Two models follow this trend. BERTw/2-ctx uses BERT models to capture contextual interaction for every question (history and current) and answer (2N+1 sequences for N turns) and concatenates all sequences together. Finally, it runs Bi-GRU [3]} over the aggregated sequence to capture inter-turn interactions before sending for prediction. On the other hand, HAM [4]} ignores the history questions and uses the current question as a query with positional History Answer Embeddings (section REF ), thus generating one output sequence per conversation turn. Fig. REF  illustrates HAM encoder. The final sequence is generated using token-level soft-attention based aggregation across all per-turn contextualized sequences.

