Humanities scholars commonly provide evidence forclaimsthat they make about a work of literature (e.g., a novel)  in the form ofquotationsfrom the work.
We collect a large-scale dataset (RELiC)  of 78K literary quotations and surrounding critical analysis and use it to formulate the novel task ofliterary evidence retrieval, in which models are given anexcerpt of literary analysissurrounding amasked quotationand asked to retrieve the quoted passage from the set of all passages in the work. Solving this retrieval task requires a deep understanding of complex literary and linguistic phenomena, which proves challenging to methods that overwhelmingly rely on lexical and semantic similarity matching. We implement a RoBERTa-based dense passage retriever for this task that outperforms existing pretrained information retrieval baselines; however, experiments and analysis by human domain experts indicate that there is substantial room for improvement over our dense retriever.