Conversational QA involves retrieval-augmented generation (RAG) in open-domain setting, or when the provided documents are longer than the context window of LLM. The dense retrievers are usually trained to retrieve the top-\(k\) relevant chunks given a single question (e.g., Lin et al., 2023; Wang et al., 2022; Izacard et al., 2022). In conversational QA, the follow-up questions (e.g., with pronouns referring to entities mentioned in the previous conversation) may have insufficient information for retrieval, while feeding them along with the dialogue history can be redundant and lead to sub-optimal results.