Add Model Evals (#1)
Browse files- Add Model Evals (43bd4c107a985cf87aa6f3d55de01ec3b88cdb51)
Co-authored-by: USVSN Sai Prashanth <[email protected]>
README.md
CHANGED
|
@@ -1 +1,18 @@
|
|
| 1 |
-
wandb: https://wandb.ai/eleutherai/pythia-rlhf/runs/8p0wfi7m?workspace=user-yongzx
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
wandb: https://wandb.ai/eleutherai/pythia-rlhf/runs/8p0wfi7m?workspace=user-yongzx
|
| 2 |
+
|
| 3 |
+
Model Evals:
|
| 4 |
+
| Task |Version|Filter| Metric |Value | |Stderr|
|
| 5 |
+
|--------------|-------|------|----------|-----:|---|-----:|
|
| 6 |
+
|arc_challenge |Yaml |none |acc |0.2654|± |0.0129|
|
| 7 |
+
| | |none |acc_norm |0.2875|± |0.0132|
|
| 8 |
+
|arc_easy |Yaml |none |acc |0.6149|± |0.0100|
|
| 9 |
+
| | |none |acc_norm |0.5391|± |0.0102|
|
| 10 |
+
|lambada_openai|Yaml |none |perplexity|5.6120|± |0.1509|
|
| 11 |
+
| | |none |acc |0.6146|± |0.0068|
|
| 12 |
+
|logiqa |Yaml |none |acc |0.1951|± |0.0155|
|
| 13 |
+
| | |none |acc_norm |0.2796|± |0.0176|
|
| 14 |
+
|piqa |Yaml |none |acc |0.7160|± |0.0105|
|
| 15 |
+
| | |none |acc_norm |0.7182|± |0.0105|
|
| 16 |
+
|sciq |Yaml |none |acc |0.8610|± |0.0109|
|
| 17 |
+
| | |none |acc_norm |0.7930|± |0.0128|
|
| 18 |
+
|winogrande |Yaml |none |acc |0.5754|± |0.0139|
|