
Dataset preparation: Conversations are prepared over passages collected across 7 different domains, each with its source dataset, such as news articles derived from CNN [1]}. Amongst the 7 domains, two are used for out-of-domain evaluation (only for evaluation, not training), while the other five aid in-domain evaluation (both training and evaluation). The dialog is prepared in a two annotator setting with one questioning and another answering, both referring to the entire context.

 Questions: Questions are factoid but require sufficient co-referencing and pragmatic reasoning [2]}.

Answers: Answers are free-form, with their corresponding rationale highlighted in the passage. However, datasetcomparison identified that the answers are slightly modified versions of the rationale, and therefore optimizing an extractive model to predict the answer span with maximum F1 overlap to the gold answer can achieve up to 97.8 F1.

Dialog features: The dialogs mostly involve drilling-down for details (about 60% of all questions) but lack other dialog features like topic-shift, clarification, or definition.

Evaluation: Macro-average F1 score of word overlap is used as an evaluation metric and is computed separately for in-domain and out-of-domain.

