Files changed (1) hide show
  1. README.md +19 -12
README.md CHANGED
@@ -7,28 +7,24 @@ tags:
7
  - Patronus AI
8
  - evaluation
9
  - hallucination-detection
10
- license: llama3
11
  language:
12
  - en
13
- base_model: PatronusAI/Llama-3-Patronus-Lynx-8B-Instruct
14
- pipeline_tag: text-generation
15
  ---
16
 
17
- # QuantFactory/Llama-3-Patronus-Lynx-8B-Instruct-GGUF
18
- This is quantized version of [PatronusAI/Llama-3-Patronus-Lynx-8B-Instruct](https://huggingface.co/PatronusAI/Llama-3-Patronus-Lynx-8B-Instruct) created using llama.cpp
19
 
20
- # Model Description
21
-
22
- Lynx is an open-source hallucination evaluation model. Patronus-Lynx-8B-Instruct was trained on a mix of datasets including CovidQA, PubmedQA, DROP, RAGTruth.
23
  The datasets contain a mix of hand-annotated and synthetic data. The maximum sequence length is 8000 tokens.
24
 
25
 
26
  ## Model Details
27
 
28
- - **Model Type:** Patronus-Lynx-8B-Instruct is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct model.
29
  - **Language:** Primarily English
30
  - **Developed by:** Patronus AI
31
- - **License:** [https://llama.meta.com/llama3/license](https://llama.meta.com/llama3/license)
 
32
 
33
  ### Model Sources
34
 
@@ -70,7 +66,7 @@ The model will output the score as 'PASS' if the answer is faithful to the docum
70
 
71
  ## Training Details
72
 
73
- The model was finetuned for 3 epochs using H100s on dataset of size 2400. We use [lion](https://github.com/lucidrains/lion-pytorch) optimizer with lr=5.0e-7. For more details on data generation, please check out our Github repo.
74
 
75
  ### Training Data
76
 
@@ -80,8 +76,19 @@ We train on 2400 samples consisting of CovidQA, PubmedQA, DROP and RAGTruth samp
80
 
81
  The model was evaluated on [PatronusAI/HaluBench](https://huggingface.co/datasets/PatronusAI/HaluBench).
82
 
83
- It outperforms GPT-3.5-Turbo, GPT-4-Turbo, GPT-4o and Claude Sonnet.
 
 
 
84
 
 
 
 
 
 
 
 
 
85
 
86
  ## Model Card Contact
87
  [@sunitha-ravi](https://huggingface.co/sunitha-ravi)
 
7
  - Patronus AI
8
  - evaluation
9
  - hallucination-detection
10
+ license: cc-by-nc-4.0
11
  language:
12
  - en
 
 
13
  ---
14
 
15
+ # Model Card for Model ID
 
16
 
17
+ Lynx is an open-source hallucination evaluation model. Patronus-Lynx-70B-Instruct was trained on a mix of datasets including CovidQA, PubmedQA, DROP, RAGTruth.
 
 
18
  The datasets contain a mix of hand-annotated and synthetic data. The maximum sequence length is 8000 tokens.
19
 
20
 
21
  ## Model Details
22
 
23
+ - **Model Type:** This is a quantized version of Patronus-Lynx-70B-Instruct.
24
  - **Language:** Primarily English
25
  - **Developed by:** Patronus AI
26
+ - **Paper:** [https://arxiv.org/abs/2407.08488](https://arxiv.org/abs/2407.08488)
27
+ - **License:** [ https://creativecommons.org/licenses/by-nc/4.0/](https://creativecommons.org/licenses/by-nc/4.0/)
28
 
29
  ### Model Sources
30
 
 
66
 
67
  ## Training Details
68
 
69
+ The model was finetuned for 3 epochs using 32 H100s on dataset of size 2400. We use [lion](https://github.com/lucidrains/lion-pytorch) optimizer with lr=5.0e-7. For more details on data generation, please check out our Github repo.
70
 
71
  ### Training Data
72
 
 
76
 
77
  The model was evaluated on [PatronusAI/HaluBench](https://huggingface.co/datasets/PatronusAI/HaluBench).
78
 
79
+ It outperforms GPT-3.5-Turbo, GPT-4-Turbo, GPT-4o and Claude-3-Sonnet.
80
+
81
+ ## Citation
82
+ If you are using the model, cite using
83
 
84
+ ```
85
+ @article{ravi2024lynx,
86
+ title={Lynx: An Open Source Hallucination Evaluation Model},
87
+ author={Ravi, Selvan Sunitha and Mielczarek, Bartosz and Kannappan, Anand and Kiela, Douwe and Qian, Rebecca},
88
+ journal={arXiv preprint arXiv:2407.08488},
89
+ year={2024}
90
+ }
91
+ ```
92
 
93
  ## Model Card Contact
94
  [@sunitha-ravi](https://huggingface.co/sunitha-ravi)