File size: 1,013 Bytes
421284f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
# Spider-NQ: Context Encoder
This is the context encoder of the model fine-tuned on Natural Questions (and initialized from Spider) discussed in our paper [Learning to Retrieve Passages without Supervision](https://arxiv.org/abs/2112.07708).
## Usage
We used weight sharing for the query encoder and passage encoder, so the same model should be applied for both.
**Note**! We format the passages similar to DPR, i.e. the title and the text are separated by a `[SEP]` token, but token
type ids are all 0-s.
An example usage:
```python
from transformers import AutoTokenizer, DPRContextEncoder
tokenizer = AutoTokenizer.from_pretrained("NAACL2022/spider-nq-ctx-encoder")
model = DPRContextEncoder.from_pretrained("NAACL2022/spider-nq-ctx-encoder")
title = "Sauron"
context = "Sauron is the title character and main antagonist of J. R. R. Tolkien's \"The Lord of the Rings\"."
input_dict = tokenizer(title, context, return_tensors="pt")
del input_dict["token_type_ids"]
outputs = model(**input_dict)
```
|