Open-Domain Conversational Question Answering with Historical Answers

Open-domain conversational question answering can be viewed as two tasks: passage retrieval and conversational question answering, where the former relies on selecting candidate passages from a large corpus and the latter requires better understanding of a question with contexts to predict the answers. This paper proposes ConvADR-QA that leverages historical answers to boost retrieval performance and further achieves better answering performance. In our proposed framework, the retrievers use a teacher-student framework to reduce noises from previous turns. Our experiments on the benchmark dataset, OR-QuAC, demonstrate that our model outperforms existing baselines in both extractive and generative reader settings, well justifying the effectiveness of historical answers for open-domain conversational question answering.


Introduction
Conversational information seeking and conversational question answering (CQA) are fundamental tasks of dialogue systems (Gao et al., 2018). The conversational agents are expected to serve as nature interfaces for users' information need, providing information and answers via multi-turn natural language interactions. The multi-turn natural of CQA makes it challenging as the queries are contextualized, requiring the systems to resolve coreference and ambiguities. With recent advances in language understanding and dialogue modeling, along with the curation of large-scale datasets, e.g., QuAC  and CoQA (Reddy et al., 2019), we have seen substantial progress in CQA.
While the state-of-the-art (SOTA) models have achieved performance comparable or even superior than human performance on QA and CQA datasets, this setting is highly limited as it requires the source document containing evidence to be given, which is unlikely the case in real-world scenarios. To address this issue, researchers have expanded the scheme of CQA to an open-domain setting, where the document containing evidence must be retrieved from a large candidate pool (Qu et al., 2020). In the open-domain setting, there are usually millions of candidate documents, making the conventional method which jointly encodes the query and the document infeasible (Chen et al., 2017). The dominant technique to tackle the challenge is dense retrieval Qu et al., 2020;, which encodes a query and documents as dense representations separately and performs nearest neighbor search that is efficient and scalable to millions of documents. It has been shown to outperform traditional sparse retrieval methods on multiple QA benchmarks. However, applying dense retrieval for conversations may need to consider the dialogue context and structure, which is not trivial. Qu et al. (2020) proposed ORConvQA to include previous questions in the same dialogue. where the context-dependent nature of questions is shown useful. ConvDR (Yu et al., 2021) further improved the retrieval performance by knowledge distillation on reformulated questions with an ad-hoc teacher model. Nevertheless, simply concatenating historical questions is suboptimal. Our hypothesis is that rather than relying on the model to infer helpful knowledge from historical questions, we provide direct signals by adding historical answers to the input. Hence, we propose ConvADR-QA (Conversational Answeraware Dense Retrieval) to leverage historical answers for better retrieval and then answering performance for open-domain CQA.
2 Related Work CQA A unique challenge to CQA is that the questions are context-dependent. Hence, most prior work focused on various history modeling techniques (Huang et al., 2018;Yeh and Chen, 2019;Qu et al., 2019b;Chen et al., 2020).  proposed to mark the previous answers in the passage by adding an answer embedding to the input embeddings. Qu et al. (2019a) extended this method to the large pre-trained language models. However, Chiang et al. (2020) showed that prior conversational models do not fully understand the content, implying that CQA still needs further investigation. While our method also leverages historical answers as additional input signal, our major contribution is that we apply this technique to dense retrieval instead of question answering for better practicality in a open-domain setting.
Open-Domain QA Without a given target passage, most work for this task was built upon the dense retrieval framework for retrieving relevant passages for QA. DPR  first showed that dense retrieval outperforms sparse retrieval methods. GAR (Mao et al., 2021a) introduced pseudo relevance feedback by augmenting queries with generated texts. RIDER (Mao et al., 2021b) proposed a simple passage reranking method which promotes the passages containing the predicted answers. While these methods consider the predicted answers, they aim at improving single-turn question answering. We instead focus on enhancing model's ability on handling multiturn conversational questions.
Open-Domain CQA Researchers have put increasing attention on open-domain CQA with the TREC Conversational Assistance Track (Dalton et al., 2020(Dalton et al., , 2021. However, these datasets have limited supervision, making dense retrieval hardly applicable due to its data-hungry nature. Qu et al. (2020) introduced the first large-scale open-domain CQA data, OR-QuAC, by extending QuAC  to a open-domain setting. They also proposed ORConvQA, a pipeline system with a DPR retriever and an extractive reader, as a baseline system. ConvDR (Yu et al., 2021) proposed to reformulate questions into their context-independent rewrites with the CANARD dataset (Elgohary et al., 2019), then applied knowledge distillation using a ad-hoc teacher model. Our method is built upon these two methods by incorporating historical answers to aid the retriever.  proposed a graph-guided retrieval method which constructs a graph using passages with historical answers and potential answers. Our work does not introduce extra parameters and complex modeling, and we demonstrate that we can achieve better results with Previous wrong answers lead to the wrong direction Figure 1: Demonstration of how previous answers affect the quality of an answer. a simpler design for better practicality. The All-History strategy from TopiOCQA (Adlakha et al., 2022) is very similar to ours. However, their experimental setting is not realistic as they used ground truth answers as historical answers, which is corresponding to our oracle setting.

ConvADR-QA
where p i can be viewed as a sequence of tokens p 1 i , . . . , p l i . Given the t-th question q t and all historical questions {q i } t−1 i=1 in a conversation, the task of open-domain CQA is to predict a t from C. In an extractive setting, a t is a span p s i , . . . , p e i from a passage p i . The difficulty of open-domain CQA is that the current question usually requires context information from previous turns, which makes it harder for the system to capture the latent information compared with the open-domain QA task. Previous work on open-domain conversational search addressed the problem by concatenating the current and historical questions without answers (Qu et al., 2020;Yu et al., 2021). Our motivation is that historical answers can also provide the important signal for the current question to obtain the answers illustrated in Figure 1.
To better leverage the historical answers for open-domain CQA, we propose ConvADR-QA illustrated in Figure 2, which includes a retriever for obtaining relevant passages from a large collection and a reader for CQA.

Retriever
Following the prior work , we apply a dense retrieval  method, which has shown dominant performance over sparse ones. Specifically, the model uses a dual-encoder architecture to map passages and questions to the same embedding space. The input of our question encoder is the concatenation of historical questions and answers: The retrieval score is then defined as the dot product of the passage embedding and the question embedding: In the training process, each question contains one gold passage p + and a set of negative passages P − , ConvADR-QA is then optimized using the negative log likelihood loss:

Knowledge Distillation
In conversational search, dense retrieval is challenging since the current question requires information from previous turns, which aggravate the discrepancy between question embeddings and passage embeddings. Yu et al. (2021) recently addressed the problem using a teacher-student framework to distill knowledge from an ad-hoc teacher model. The input of the teacher model is a manually-rewritten context-independent query q k , and the knowledge distillation (KD) loss is defined as the mean square error (MSE) loss between the teacher's and the student's question embeddings: The retrieval loss of our multi-task learning setting is the sum of NLL loss and KD loss:

Reader
The task of the reader is to extract a span from passages as the final answer. We use a standard BERT model for the machine comprehension task (Devlin et al., 2019). Given the t-th question q t and top-K candidate passages {p i } K i=1 retrieved by our retriever, the reader first extracts a span for each passage by choosing the highest score of start and end tokens. The score of the m-th token is defined as follows: We choose the final answer by multiplying the retriever score S rt and the sum of start/end token score as the reader score S rd : S(q t , p) = S rt (q t , p) · S rd (q t ; p).  (2021), we use three commonly used metrics, MRR@5, Recall@5 and MAP@10, to evaluate the retrieval performance. In addition, we use word-level F1 and human equivalence score (HEQ) provided by the QuAC challenge to evaluate the overall performance of our system. The definitions of above metrics are detailed in Appendix B.

Baselines
We compare our model with recently proposed baselines for open-domain CQA, ORConvQA (Qu et al., 2020), Graph-Guided , and RAG  for both extractive and generative settings.
• ORConvQA: It is an end-to-end system for the open-domain CQA task, which includes a retriever, a reranker and a reader. The retriever use the dense retrieval method where the input of the query encoder is the concatenation of the current and historical questions.
• Graph-Guided: Li et al. (2021) proposed a graph-guided retrieval method that models the relations among answers across conversational turns, which is the first work attempting at utilizing historical answers for opendomain CQA. This model utilizes a graph built from the hyperlink-connected passages containing historical answers to better retrieve relevant passages.
• RAG: It is a generation model that can access to pre-trained parametric memory and non-parametric memory like wikipedia. It has shown good performance on the opendomain QA task, we further adapt it to the open-domain CQA task by doing the following modifications: (1) finetuning the base model, where the input of the question encoder is the concatenation of the current and historical questions, (2) using passages from OR-QuAC as our knowledge source (nonparametric memory).
In addition to the existing open-domain CQA approaches, we further implement two baseline where we use ConvDR as the retriever model. ConvDR is a conversational dense retriever, which uses the few-shot strategy to mimic the embeddings of manual oracle queries from an ad hoc dense retriever. It is also the current SOTA model in the retrieval stage. We adopt it to open-domain CQA by enabling it with QA capability using two existing models to generate answers: (1) Reader of ORCon-vQA (Qu et al., 2020), which adapts a BERT-based extractive QA model to a multi-document setting, (2) FiD (Izacard and Grave, 2021), which uses a sequence-to-sequence model to generate the answer given the input is the question and retrieved passages, which has shown great performance at combining evidences from multiple passages.

Results
Table 1 summarizes our experimental results. It is obvious that our proposed ConvADR-QA outperforms almost all existing baseline models in both retrieval and answering stages, achieving new SOTA performance of open-domain CQA. We can observe that in both extractive and generative QA settings, our model which leverages predicted answers achieves better performance over the one without answers. Moreover, the graph-guided approach also utilizes historical answers in a more complex way, but performs worse than our ConvADR-QA, demonstrating that our model leverages answer signal more effectively. We also report the oracle results using gold historical answers. It shows that the model with gold answers outperforms the one with predicted answers in most of the metrics except HEQ-D. Note that the oracle results can be viewed as the upper bound of our method, as the gold answers are not available during inference. The results well justify our hypothesis that historical answers are informative for open-domain CQA.
Notably, we can notice that the quality of predicted answers can significantly affect the retrieval performance. Our experiment shows that MRR@5 drops when using FiD as the reader, demonstrating that a QA model with weak performance could potentially hurt retrieval performance. Our hypothesis is that due to its lower answering quality, the errors would propagate through the conversation and mislead the retriever, indicating that further improvement on reader performance could also improve the retrieval performance of our method. In sum, the experimental results show the effectiveness of our model for open-domain CQA in both extractive and generative settings. An example is presented in Table 3 for qualitative analysis, where it can be shown that the previous answers affect the following prediction results. More analysis can be found in Appendix C.

Error Propagation Analysis
To inspect the impact of the errors propagated through the conversation and reduce the robustness. we conduct analysis on accuracy against number of turns in Figure 3. It shows that the benefits of adding the answers outweigh the error propagation, where ConvADR-QA outperforms ConvDR in earlier turns, which tends to drop as the dialogue gets Q1: What is Roberto Mangabeira Unger,'s programmatic thought? A1: Key in Unger's thinking is the need to re-imagine social institutions before attempting to revise them. ConvDR: The beginning of Unger's academic career began with the books Knowledge and Politics and Law in Modern Society, ConvADR-QA: Key in Unger's thinking is the need to reimagine social institutions before attempting to revise them.
Q2: Can you explain the mechanism of thinking? A2: In building this program, however, we must not entertain complete revolutionary overhaul, lest we be plagued by three false assumptions: ConvDR: CANNOTANSWER ConvADR-QA: In building this program, however, we must not entertain complete revolutionary overhaul, lest we be plagued by three false assumptions: Q3: What are the three false assumptions? A3: Typological Fallacy: ConvDR: Unger finds three weaknesses that crippled the theory: foremost, the theory claimed that equilibrium would be spontaneously generated in a market economy. ConvADR-QA: Typological Fallacy: the fallacy that there is closed list of institutional alternatives in history, such as "feudalism" or "capitalism".

Conclusions
This work introduces ConvADR-QA, an opendomain CQA model that leverages historical answers. The experiments on a benchmark dataset demonstrate that our proposed method outperforms all baselines for both retrieval and answering performance. Our results justify not only the importance of historical answers in a conversation but also the generalizability to different types of readers.

A Reproducibility
Our source code and the trained model was published at GitHub together with running scripts for better reproducibility. All models are trained with 2 Nvidia Quadro P6000. For the retriever, we set the training batch size to 4, the number of epochs to 3, and the learning rate to 1e-5. For the reader, we set the training batch to 2, the number of epochs to 3, the max sequence length to 512, the max question length to 125 and the learning rate to 3e-5.

B Evaluation Metrics
We use following metrics for evaluating our proposed model.
• MRR: Reciprocal Rank (RR) calculates the reciprocal of the rank where the first relevant passage was retrieved, and MRR averages the reciprocal rank across all questions.
• Recall: The proportion of the questions that the answer is in the retrieved passages.
• MAP: Mean Average Precision is the mean of the average precision scores for each question.
• F1: It measures the overlap of the predicted answer span and the ground truth answer span at the word level.
• HEQ: HEQ-Q measures the percentage of questions where the system matches or surpasses human performance in terms of F1 score. HEQ-D measures the percentage of dialogues in which all questions have an HEQ-Q = 1.

C Qualitative Study
The results generated by ConvDR and ConvADR-QA are presented in Table 3, where the underline texts indicate the spans appearing in the ground truth answers. It can be found that historical answers can be an important signal to obtain the answers.

Q1
What was unique about Joe Greene's attitude? A1 ferocious and intimidating.

ConvDR
Greene's nickname remained popular due to his exploits on the playing field, where he was described as ferocious and intimidating. ConvADR-QA Greene's nickname remained popular due to his exploits on the playing field, where he was described as ferocious and intimidating.

Q2
Did he intimidate other players? A2 He instilled fear in opponents with the intensity of his play.

ConvDR
Lynn Swann, a wide receiver, considered Greene a mentor. ConvADR-QA He instilled fear in opponents with the intensity of his play.

Q3
Who did he play for? A3 Pittsburgh ConvDR Houston Oilers ConvADR-QA Houston Oilers

Q4
Did his teams do well? A4 CANNOTANSWER ConvDR Greene was named the captain of the defense in 1977.

Q5
What else should I know about his playing style? A5 In his early years with the Steelers, Greene was at times uncontrollable, and often let his temper get the best of him.

ConvDR
Greene was named the captain of the defense in 1977. ConvADR-QA In his early years with the Steelers, Greene was at times uncontrollable, and often let his temper get the best of him.

Q1
What is Roberto Mangabeira Unger,'s programmatic thought? A1 Key in Unger's thinking is the need to re-imagine social institutions before attempting to revise them.

ConvDR
The beginning of Unger's academic career began with the books Knowledge and Politics and Law in Modern Society, ConvADR-QA Key in Unger's thinking is the need to re-imagine social institutions before attempting to revise them.

Q2
Can you explain the mechanism of thinking? A2 In building this program, however, we must not entertain complete revolutionary overhaul, lest we be plagued by three false assumptions: ConvDR CANNOTANSWER ConvADR-QA In building this program, however, we must not entertain complete revolutionary overhaul, lest we be plagued by three false assumptions:

Q3
What are the three false assumptions? A3 Typological Fallacy:

ConvDR
Unger finds three weaknesses that crippled the theory: foremost, the theory claimed that equilibrium would be spontaneously generated in a market economy. ConvADR-QA Typological Fallacy: the fallacy that there is closed list of institutional alternatives in history, such as "feudalism" or "capitalism".