
Appending selected history questions and/or answers (in raw form or text span indices) to the current question before encoding. QA tokens across turns should be distinguishable or separated when appending. Models DrQA+PGNet [1]}, SDNet [2]} and RoBERTa + AT + KD [3]} append all history QA pairs separated by tokens like symbols \([Q]\)  or \([A]\)  such that new \( Q_k^* = \lbrace [Q], Q_1, [A], A_1, ..., [Q], Q_{k-1}, [A],A_{k-1}, [Q], Q_k\rbrace \) . On the other hand, QuAC baseline model BiDAF++ w/ 2-ctx [4]} and GraphFlow [5]} append only the history questions to the current question and encode relative dialog-turn number within each question embedding to differentiate. quac validate that this dialog-turn encoding strategy performs better in practice.

Encoding context tokens with history answer marker embeddings (HAE) before passing on for reasoning. These embeddings indicate if the context token is present in any conversational history answer or not, such as in BiDAF++ w/ 2-ctx [6]}, GraphFlow [5]}, BERT+HAE [8]} and HAM [9]}. HAM encodes a dialog-turn encoded variant of HAE called Positional HAE. It maintains a lookup table of history embeddings for every relative position from the current conversation and embeds the corresponding embedding if the token is found in that history answer, e.g. for the current question \(q_k\)  if a token is found in history answer \(a_{k-2}\)  then Positional HAE embedding at index 2 is encoded, otherwise embedding at index 0 is encoded. This setting is illustrated in Fig. REF .

Integrating intermediate representations generated in the reasoning modules of selected history conversation turns to grasp the deep latent semantics of the history, rather than acting on raw inputs. This approach is also called the FLOW based approach. The models that follow this approach are FlowQA [10]}, FlowDelta [11]}, and GraphFlow [5]}. GraphFlow encodes conversational histories into context graphs which are used by the reasoning module for contextual analysis.

