Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,29 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: cc-by-nc-4.0
|
3 |
+
---
|
4 |
+
|
5 |
+
This is a **Factual Consistency Evaluation** model, introduced in the [TrueTeacher paper (Gekhman et al, 2023)](https://arxiv.org/pdf/2305.11171.pdf).
|
6 |
+
|
7 |
+
The model is optimized for evaluating factual consistency in **summarization**.
|
8 |
+
|
9 |
+
It is the main model from the paper (see **"T5-11B w. ANLI + TrueTeacher full"** in **Table 1**) which is based on a **T5-11B** fine-tuned with a mixture of the following datasets:
|
10 |
+
- TrueTeacher ([Gekhman et al., 2023](https://arxiv.org/pdf/2305.11171.pdf))
|
11 |
+
- ANLI ([Nie et al., 2020](https://aclanthology.org/2020.acl-main.441.pdf))
|
12 |
+
|
13 |
+
|
14 |
+
The input format for the model is: "premise: GROUNDING_DOCUMENT hypothesis: HYPOTHESIS_SUMMARY".
|
15 |
+
|
16 |
+
The model predicts a binary label ('1' - Factualy Consistent, '0' - Factualy Inconsistent).
|
17 |
+
|
18 |
+
If you use this model for a research publication, please cite the TrueTeacher paper (using the bibtex entry below) and the dataset papers mentioned above.
|
19 |
+
|
20 |
+
```
|
21 |
+
@misc{gekhman2023trueteacher,
|
22 |
+
title={TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models},
|
23 |
+
author={Zorik Gekhman and Jonathan Herzig and Roee Aharoni and Chen Elkind and Idan Szpektor},
|
24 |
+
year={2023},
|
25 |
+
eprint={2305.11171},
|
26 |
+
archivePrefix={arXiv},
|
27 |
+
primaryClass={cs.CL}
|
28 |
+
}
|
29 |
+
```
|