jpacifico commited on
Commit
b2d8214
1 Parent(s): 30677e5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -3
README.md CHANGED
@@ -16,13 +16,25 @@ pipeline_tag: text-generation
16
 
17
  DPO fine-tuned of [microsoft/Phi-3-medium-4k-instruct](https://huggingface.co/microsoft/Phi-3-medium-4k-instruct) (14B params)
18
  using the [jpacifico/french-orca-dpo-pairs-revised](https://huggingface.co/datasets/jpacifico/french-orca-dpo-pairs-revised) rlhf dataset.
19
- Training in French also improves the model in English, surpassing the performances of its base model.
20
  Window context = 4k tokens
21
 
22
  ### Benchmarks
23
 
24
- Submitted on the [OpenLLM Leaderboard](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) (aug 2024)
25
- Results coming soon.
 
 
 
 
 
 
 
 
 
 
 
 
26
 
27
  ### MT-Bench-French
28
 
 
16
 
17
  DPO fine-tuned of [microsoft/Phi-3-medium-4k-instruct](https://huggingface.co/microsoft/Phi-3-medium-4k-instruct) (14B params)
18
  using the [jpacifico/french-orca-dpo-pairs-revised](https://huggingface.co/datasets/jpacifico/french-orca-dpo-pairs-revised) rlhf dataset.
19
+ Training in French also improves the model in English, surpassing the performances of its base model (MMLU).
20
  Window context = 4k tokens
21
 
22
  ### Benchmarks
23
 
24
+ Chocolatine-14B is the best-performing < 50B model in terms of MMLU-PRO on the [OpenLLM Leaderboard](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) (august 2024)
25
+
26
+ ![image/png](https://github.com/jpacifico/Chocolatine-LLM/blob/main/Assets/benchmark_14B_V1.png?raw=false)
27
+
28
+
29
+ | Metric |Value|
30
+ |-------------------|----:|
31
+ |Avg. |29.83|
32
+ |IFEval (0-Shot) |46.89|
33
+ |BBH (3-Shot) |48.02|
34
+ |MATH Lvl 5 (4-Shot)|14.88|
35
+ |GPQA (0-shot) |12.19|
36
+ |MuSR (0-shot) |15.15|
37
+ |**MMLU-PRO (5-shot)** |**41.82**|
38
 
39
  ### MT-Bench-French
40