zorik commited on
Commit
ed578a0
1 Parent(s): e974f49

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +29 -0
README.md ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-4.0
3
+ ---
4
+
5
+ This is a **Factual Consistency Evaluation** model, introduced in the [TrueTeacher paper (Gekhman et al, 2023)](https://arxiv.org/pdf/2305.11171.pdf).
6
+
7
+ The model is optimized for evaluating factual consistency in **summarization**.
8
+
9
+ It is the main model from the paper (see **"T5-11B w. ANLI + TrueTeacher full"** in **Table 1**) which is based on a **T5-11B** fine-tuned with a mixture of the following datasets:
10
+ - TrueTeacher ([Gekhman et al., 2023](https://arxiv.org/pdf/2305.11171.pdf))
11
+ - ANLI ([Nie et al., 2020](https://aclanthology.org/2020.acl-main.441.pdf))
12
+
13
+
14
+ The input format for the model is: "premise: GROUNDING_DOCUMENT hypothesis: HYPOTHESIS_SUMMARY".
15
+
16
+ The model predicts a binary label ('1' - Factualy Consistent, '0' - Factualy Inconsistent).
17
+
18
+ If you use this model for a research publication, please cite the TrueTeacher paper (using the bibtex entry below) and the dataset papers mentioned above.
19
+
20
+ ```
21
+ @misc{gekhman2023trueteacher,
22
+ title={TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models},
23
+ author={Zorik Gekhman and Jonathan Herzig and Roee Aharoni and Chen Elkind and Idan Szpektor},
24
+ year={2023},
25
+ eprint={2305.11171},
26
+ archivePrefix={arXiv},
27
+ primaryClass={cs.CL}
28
+ }
29
+ ```