simonhughes22
commited on
Commit
•
9afe510
1
Parent(s):
1b467f3
Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,42 @@
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
---
|
4 |
+
# Cross-Encoder for Hallucination Detection
|
5 |
+
This model was trained using [SentenceTransformers](https://sbert.net) [Cross-Encoder](https://www.sbert.net/examples/applications/cross-encoder/README.html) class. This model is based on [microsoft/deberta-v3-base](https://huggingface.co/microsoft/deberta-v3-base).
|
6 |
+
|
7 |
+
## Training Data
|
8 |
+
The model was trained on the NLI data and a variety of datasets evaluating summarization accuracy for factual consistency, including [FEVER](https://huggingface.co/datasets/fever), [Vitamin C](https://huggingface.co/datasets/tals/vitaminc) and [PAWS](https://huggingface.co/datasets/paws).
|
9 |
+
|
10 |
+
## Performance
|
11 |
+
TODO
|
12 |
+
## Usage
|
13 |
+
|
14 |
+
Pre-trained models can be used like this:
|
15 |
+
```python
|
16 |
+
from sentence_transformers import CrossEncoder
|
17 |
+
model = CrossEncoder('cross-encoder/nli-deberta-v3-large')
|
18 |
+
scores = model.predict([('A man is eating pizza', 'A man eats something'), ('A black race car starts up in front of a crowd of people.', 'A man is driving down a lonely road.')])
|
19 |
+
|
20 |
+
#Convert scores to labels
|
21 |
+
label_mapping = ['contradiction', 'entailment', 'neutral']
|
22 |
+
labels = [label_mapping[score_max] for score_max in scores.argmax(axis=1)]
|
23 |
+
```
|
24 |
+
|
25 |
+
## Usage with Transformers AutoModel
|
26 |
+
You can use the model also directly with Transformers library (without SentenceTransformers library):
|
27 |
+
```python
|
28 |
+
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
29 |
+
import torch
|
30 |
+
|
31 |
+
model = AutoModelForSequenceClassification.from_pretrained('cross-encoder/nli-deberta-v3-large')
|
32 |
+
tokenizer = AutoTokenizer.from_pretrained('cross-encoder/nli-deberta-v3-large')
|
33 |
+
|
34 |
+
features = tokenizer(['A man is eating pizza', 'A black race car starts up in front of a crowd of people.'], ['A man eats something', 'A man is driving down a lonely road.'], padding=True, truncation=True, return_tensors="pt")
|
35 |
+
|
36 |
+
model.eval()
|
37 |
+
with torch.no_grad():
|
38 |
+
scores = model(**features).logits
|
39 |
+
label_mapping = ['contradiction', 'entailment', 'neutral']
|
40 |
+
labels = [label_mapping[score_max] for score_max in scores.argmax(dim=1)]
|
41 |
+
print(labels)
|
42 |
+
```
|