Update README.md
Browse files
README.md
CHANGED
@@ -16,13 +16,25 @@ pipeline_tag: text-generation
|
|
16 |
|
17 |
DPO fine-tuned of [microsoft/Phi-3-medium-4k-instruct](https://huggingface.co/microsoft/Phi-3-medium-4k-instruct) (14B params)
|
18 |
using the [jpacifico/french-orca-dpo-pairs-revised](https://huggingface.co/datasets/jpacifico/french-orca-dpo-pairs-revised) rlhf dataset.
|
19 |
-
Training in French also improves the model in English, surpassing the performances of its base model.
|
20 |
Window context = 4k tokens
|
21 |
|
22 |
### Benchmarks
|
23 |
|
24 |
-
|
25 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
26 |
|
27 |
### MT-Bench-French
|
28 |
|
|
|
16 |
|
17 |
DPO fine-tuned of [microsoft/Phi-3-medium-4k-instruct](https://huggingface.co/microsoft/Phi-3-medium-4k-instruct) (14B params)
|
18 |
using the [jpacifico/french-orca-dpo-pairs-revised](https://huggingface.co/datasets/jpacifico/french-orca-dpo-pairs-revised) rlhf dataset.
|
19 |
+
Training in French also improves the model in English, surpassing the performances of its base model (MMLU).
|
20 |
Window context = 4k tokens
|
21 |
|
22 |
### Benchmarks
|
23 |
|
24 |
+
Chocolatine-14B is the best-performing < 50B model in terms of MMLU-PRO on the [OpenLLM Leaderboard](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) (august 2024)
|
25 |
+
|
26 |
+
![image/png](https://github.com/jpacifico/Chocolatine-LLM/blob/main/Assets/benchmark_14B_V1.png?raw=false)
|
27 |
+
|
28 |
+
|
29 |
+
| Metric |Value|
|
30 |
+
|-------------------|----:|
|
31 |
+
|Avg. |29.83|
|
32 |
+
|IFEval (0-Shot) |46.89|
|
33 |
+
|BBH (3-Shot) |48.02|
|
34 |
+
|MATH Lvl 5 (4-Shot)|14.88|
|
35 |
+
|GPQA (0-shot) |12.19|
|
36 |
+
|MuSR (0-shot) |15.15|
|
37 |
+
|**MMLU-PRO (5-shot)** |**41.82**|
|
38 |
|
39 |
### MT-Bench-French
|
40 |
|