Update README.md

README.md

@@ -6,37 +6,49 @@ library_name: transformers
 tags:
 - mergekit
 - merge
-
+license: apache-2.0
+datasets:
+- prometheus-eval/Preference-Collection
+- prometheus-eval/Feedback-Collection
+language:
+- en
 ---
-# prometheus-8b-linear
-
-This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
-
-## Merge Details
-### Merge Method
+# prometheus-2-llama-3-8b
 
-This model was merged using the [linear](https://arxiv.org/abs/2203.05482) merge method.
+Replication of [prometheus-7b-v2.0](https://huggingface.co/prometheus-eval/prometheus-7b-v2.0) using [Llama 3 8B Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) as a base model.
 
-### Models Merged
+As in their paper, two different models were trained on their preference and feedback datasets then linearly merged at equal weight.
 
-The following models were included in the merge:
-* [chargoddard/prometheus-llama-3-8b-preference](https://huggingface.co/chargoddard/prometheus-llama-3-8b-preference)
-* [chargoddard/prometheus-llama-3-8b-absolute](https://huggingface.co/chargoddard/prometheus-llama-3-8b-absolute)
+Training hyperparameters:
+* 1 epoch
+* Learning rate 1e-5
+* Effective batch size 128
+* Cosine annealing
+* ~5% warmup
 
-### Configuration
 
-The following YAML configuration was used to produce this model:
+Uses Llama 3 Instruct prompt format and the same prompts as prometheus-7b-v2.0. See that readme for info.
 
-```yaml
-merge_method: linear
-models:
-  - model: chargoddard/prometheus-llama-3-8b-preference
-    parameters:
-      weight: 0.5
-  - model: chargoddard/prometheus-llama-3-8b-absolute
-    parameters:
-      weight: 0.5
-dtype: bfloat16
+# Citations
 
 
+```bibtex
+@misc{kim2023prometheus,
+    title={Prometheus: Inducing Fine-grained Evaluation Capability in Language Models},
+    author={Seungone Kim and Jamin Shin and Yejin Cho and Joel Jang and Shayne Longpre and Hwaran Lee and Sangdoo Yun and Seongjin Shin and Sungdong Kim and James Thorne and Minjoon Seo},
+    year={2023},
+    eprint={2310.08491},
+    archivePrefix={arXiv},
+    primaryClass={cs.CL}
+}
 ```
+```bibtex
+@misc{kim2024prometheus,
+    title={Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models},
+    author={Seungone Kim and Juyoung Suk and Shayne Longpre and Bill Yuchen Lin and Jamin Shin and Sean Welleck and Graham Neubig and Moontae Lee and Kyungjae Lee and Minjoon Seo},
+    year={2024},
+    eprint={2405.01535},
+    archivePrefix={arXiv},
+    primaryClass={cs.CL}
+}
+```