Nekochu
/

Llama-3.1-8B-German-ORPO

Text Generation

Model card Files Files and versions Community

Nekochu commited on Sep 22

Commit

51ce627

•

1 Parent(s): 8c0fde7

Update README.md

Files changed (1) hide show

README.md +5 -5

README.md CHANGED Viewed

@@ -180,8 +180,8 @@ Note: Output from inference [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Fac
 </details>
-Eval [MMLU-Pro](https://github.com/chigkim/Ollama-MMLU-Pro) (en).
-| Model                           | overall | biology | business | chemistry | computer science | economics | engineering | health | history | law  | math  | philosophy | physics | psychology | other |
-|----------------------------------|---------|---------|----------|-----------|------------------|-----------|-------------|--------|---------|------|-------|------------|---------|------------|-------|
-| Llama-3.1-8B-Instruct-exl2-8bpw-h8 | 37.61   | 60.81   | 37.26    | 32.86     | 38.78            | 46.33     | 23.32       | 45.48  | 39.90   | 21.62 | 38.86 | 34.67      | 28.79   | 50.63      | 44.26 |
-| **Llama-3.1-8B-German-ORPO-8.0bpw-h8-exl2** | 44.96   | 63.74   | 49.68    | 36.93     | 48.29            | 55.81     | 28.59       | 52.81  | 45.67   | 30.79 | 45.08 | 40.48      | 39.03   | 60.90      | 48.38 |

 </details>
+Eval [MMLU-Pro](https://github.com/chigkim/Ollama-MMLU-Pro):[logs](https://pastebin.com/WsTUCduN) (en):
+| Model                           | Overall Accuracy | biology | business | chemistry | computer science | economics | engineering | health | history | law  | math  | philosophy | physics | psychology | other |
+|----------------------------------|----------------------|---------|----------|-----------|------------------|-----------|-------------|--------|---------|------|-------|------------|---------|------------|-------|
+| Llama-3.1-8B-Instruct-exl2-8bpw-h8 | 38.83                | 60.81   | 37.26    | 32.86     | 38.78            | 46.33     | 23.32       | 45.48  | 39.90   | 21.62 | 38.86 | 34.67      | 28.79   | 50.63      | 44.26 |
+| **Llama-3.1-8B-German-ORPO-8.0bpw-h8-exl2** | 46.16                | 63.74   | 49.68    | 36.93     | 48.29            | 55.81     | 28.59       | 52.81  | 45.67   | 30.79 | 45.08 | 40.48      | 39.03   | 60.90      | 48.38 |