While dense retrieval has been shown effective and efficient across tasks and languages, it remains difficult to create effective fully zero-shot dense retrieval systems when no relevance label is available.
In this paper, we recognize the difficulty of zero-shot learning and encoding relevance. Instead, we propose to pivot throughHypotheticalDocumentEmbeddings (HyDE) .
Given a query,HyDEfirst zero-shot instructs an instruction-following language model (e.g.InstructGPT)  to generate ahypotheticaldocument.
The document captures relevance patterns but is unreal and may contain false details. Then, an unsupervised contrastively learned encoder (e.g.Contriever)  encodes the document into an embedding vector. This vector identifies a neighborhood in the corpus embedding space, where similarrealdocuments are retrieved based on vector similarity. This second step ground the generated document to the actual corpus, with the encoder’s dense bottleneck filtering out the incorrect details.
Our experiments show thatHyDEsignificantly outperforms the state-of-the-art unsupervised dense retrieverContrieverand shows strong performance comparable to fine-tuned retrievers, across various tasks (e.g. web search, QA, fact verification)  and languages (e.g. sw, ko, ja) .111No models were trained or fine-tuned in making this pre-print. Our open source code is available athttps://github.com/texttron/hyde.