Commit
•
6e7d32a
1
Parent(s):
ad4b8fa
Update README.md
Browse files
README.md
CHANGED
|
@@ -10,6 +10,61 @@ model-index:
|
|
| 10 |
results: []
|
| 11 |
---
|
| 12 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
| 14 |
should probably proofread and complete it, then remove this comment. -->
|
| 15 |
|
|
|
|
| 10 |
results: []
|
| 11 |
---
|
| 12 |
|
| 13 |
+
DHARMA TINY EVAL
|
| 14 |
+
============
|
| 15 |
+
|
| 16 |
+
#### Language Hi
|
| 17 |
+
|
| 18 |
+
| Model | ARC-Easy | bigbench | truthful_qa | BoolQ | winogrande | agieval | ARC-Challenge | MMLU | openbookqa |
|
| 19 |
+
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
| 20 |
+
| open-aditi-hi-v2 | 0.6245 | 0.4959 | 0.3866 | 0.7192 | 0.5353 | 0.2945 | 0.4828 | 0.3457 | 0.5279 |
|
| 21 |
+
| open-aditi-hi-v3 | 0.6803 | 0.4553 | 0.2788 | 0.7385 | 0.5390 | 0.2178 | 0.4914 | 0.3346 | 0.5688 |
|
| 22 |
+
| open-aditi-hi-v4 | 0.6989 | 0.4526 | 0.2714 | 0.7231 | 0.5167 | 0.2331 | 0.5302 | 0.3123 | 0.5316 |
|
| 23 |
+
| open-aditi-v6-gemma | 0.7212 | 0.4146 | 0.3234 | 0.6923 | 0.4870 | 0.2638 | 0.4957 | 0.3680 | 0.4349 |
|
| 24 |
+
| open-aditi-v6-llama3 | 0.5688 | 0.4119 | 0.2268 | 0.6500 | 0.4498 | 0.2331 | 0.4310 | 0.3420 | 0.3792 |
|
| 25 |
+
| open-aditi-hi-v1 | 0.4572 | 0.3767 | 0.2230 | 0.6346 | 0.4647 | 0.1840 | 0.3405 | 0.3271 | 0.3532 |
|
| 26 |
+
| OpenHermes-2.5-Mistral-7B | 0.3309 | 0.4201 | 0.3197 | 0.6077 | 0.4981 | 0.2331 | 0.3276 | 0.3086 | 0.3086 |
|
| 27 |
+
| OpenHathi-7B-Hi-v0.1-Base | 0.2862 | 0.3333 | 0.5130 | 0.6077 | 0.4907 | 0.2301 | 0.3017 | 0.2677 | 0.1933 |
|
| 28 |
+
| Airavata | 0.2751 | 0.1274 | 0.2268 | 0.0615 | 0.3866 | 0.1104 | 0.2845 | 0.1450 | 0.3383 |
|
| 29 |
+
| gemma-7b-it | 0.1227 | 0.0786 | 0.0743 | 0.1808 | 0.1561 | 0.0491 | 0.1078 | 0.0818 | 0.0855 |
|
| 30 |
+
| Qwen1.5-72B-Chat-AWQ | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
|
| 31 |
+
| Qwen1.5-7B-Chat | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
|
| 32 |
+
|
| 33 |
+
#### Language En
|
| 34 |
+
|
| 35 |
+
| Model | ARC-Easy | bigbench | truthful_qa | BoolQ | winogrande | agieval | ARC-Challenge | MMLU | openbookqa |
|
| 36 |
+
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
| 37 |
+
| OpenHermes-2.5-Mistral-7B | 0.8922 | 0.5745 | 0.3197 | 0.8346 | 0.6989 | 0.4908 | 0.7802 | 0.5911 | 0.7621 |
|
| 38 |
+
| open-aditi-hi-v2 | 0.8625 | 0.5149 | 0.3532 | 0.8192 | 0.6877 | 0.4571 | 0.7500 | 0.5613 | 0.7732 |
|
| 39 |
+
| open-aditi-hi-v4 | 0.8959 | 0.5041 | 0.2862 | 0.8423 | 0.6914 | 0.4571 | 0.7716 | 0.5651 | 0.7138 |
|
| 40 |
+
| open-aditi-hi-v3 | 0.8773 | 0.4986 | 0.3048 | 0.8385 | 0.6766 | 0.4663 | 0.7371 | 0.5613 | 0.7249 |
|
| 41 |
+
| Qwen1.5-7B-Chat | 0.8922 | 0.5122 | 0.2007 | 0.8000 | 0.6654 | 0.4294 | 0.7759 | 0.5799 | 0.7621 |
|
| 42 |
+
| open-aditi-v6-gemma | 0.8699 | 0.4959 | 0.2602 | 0.7385 | 0.5465 | 0.4540 | 0.7371 | 0.5167 | 0.6654 |
|
| 43 |
+
| open-aditi-v6-llama3 | 0.8810 | 0.4634 | 0.1822 | 0.7577 | 0.5353 | 0.4110 | 0.7457 | 0.5688 | 0.6506 |
|
| 44 |
+
| open-aditi-hi-v1 | 0.8104 | 0.3902 | 0.2491 | 0.6962 | 0.5539 | 0.3681 | 0.6379 | 0.5056 | 0.5911 |
|
| 45 |
+
| Airavata | 0.7026 | 0.4282 | 0.3123 | 0.7192 | 0.5651 | 0.3313 | 0.5172 | 0.3792 | 0.5093 |
|
| 46 |
+
| OpenHathi-7B-Hi-v0.1-Base | 0.4684 | 0.3062 | 0.4758 | 0.6346 | 0.5167 | 0.2577 | 0.3017 | 0.2788 | 0.2714 |
|
| 47 |
+
| gemma-7b-it | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
|
| 48 |
+
|
| 49 |
+
Task: BoolQ Metric: score
|
| 50 |
+
|
| 51 |
+
Task: ARC-Easy Metric: score
|
| 52 |
+
|
| 53 |
+
Task: openbookqa Metric: score
|
| 54 |
+
|
| 55 |
+
Task: winogrande Metric: score
|
| 56 |
+
|
| 57 |
+
Task: ARC-Challenge Metric: score
|
| 58 |
+
|
| 59 |
+
Task: truthful_qa Metric: score
|
| 60 |
+
|
| 61 |
+
Task: bigbench Metric: score
|
| 62 |
+
|
| 63 |
+
Task: MMLU Metric: score
|
| 64 |
+
|
| 65 |
+
Task: agieval Metric: score
|
| 66 |
+
|
| 67 |
+
|
| 68 |
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
| 69 |
should probably proofread and complete it, then remove this comment. -->
|
| 70 |
|