Add Model Evals (#1)
Browse files- Add Model Evals (0eeb6f4c637ccdc4a08ea9843c22011911147950)
- Add Lambada OpenAI to evals (48fe0daccff2135f4c0c7b12195ad93032b1b774)
Co-authored-by: USVSN Sai Prashanth <[email protected]>
README.md
CHANGED
|
@@ -1 +1,18 @@
|
|
| 1 |
-
Wandb Runs: https://wandb.ai/eleutherai/pythia-rlhf/runs/644tyaq0?workspace=user-yongzx
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Wandb Runs: https://wandb.ai/eleutherai/pythia-rlhf/runs/644tyaq0?workspace=user-yongzx
|
| 2 |
+
|
| 3 |
+
Model Evals:
|
| 4 |
+
| Task |Version|Filter| Metric |Value | |Stderr|
|
| 5 |
+
|-------------|-------|------|--------|-----:|---|-----:|
|
| 6 |
+
|arc_challenge|Yaml |none |acc |0.2287|± |0.0123|
|
| 7 |
+
| | |none |acc_norm|0.2619|± |0.0128|
|
| 8 |
+
|arc_easy |Yaml |none |acc |0.5248|± |0.0102|
|
| 9 |
+
| | |none |acc_norm|0.4533|± |0.0102|
|
| 10 |
+
|logiqa |Yaml |none |acc |0.2089|± |0.0159|
|
| 11 |
+
| | |none |acc_norm|0.2765|± |0.0175|
|
| 12 |
+
|piqa |Yaml |none |acc |0.6855|± |0.0108|
|
| 13 |
+
| | |none |acc_norm|0.6823|± |0.0109|
|
| 14 |
+
|sciq |Yaml |none |acc |0.8050|± |0.0125|
|
| 15 |
+
| | |none |acc_norm|0.7080|± |0.0144|
|
| 16 |
+
|winogrande |Yaml |none |acc |0.5335|± |0.0140|
|
| 17 |
+
|lambada_openai|Yaml |none |perplexity|9.8265|± |0.3139|
|
| 18 |
+
| | |none |acc |0.5135|± |0.0070|
|