Text Generation
Transformers
Safetensors
mistral
alignment-handbook
Generated from Trainer
text-generation-inference
Inference Endpoints
fblgit commited on
Commit
e7573d3
1 Parent(s): 91322ed

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -0
README.md CHANGED
@@ -25,6 +25,14 @@ It achieves the following results on the evaluation set:
25
  - Logits/rejected: -2.5535
26
  - Logits/chosen: -2.7973
27
 
 
 
 
 
 
 
 
 
28
  ## Model description
29
 
30
  **It seems to outperforms the original Zephyr in most of the tasks.**
@@ -46,6 +54,15 @@ Research purposes.
46
  alignment-handbook DPO with UNA on top of the SFT lora.
47
 
48
  ### Evaluation lm-evaluation-harness
 
 
 
 
 
 
 
 
 
49
  #### 0-Shot
50
  ```
51
  hf (pretrained=fblgit/juanako-7b-v1,load_in_4bit=False,dtype=float16), limit: None, num_fewshot: 0, batch_size: 8
 
25
  - Logits/rejected: -2.5535
26
  - Logits/chosen: -2.7973
27
 
28
+
29
+ ```
30
+ hf (pretrained=fblgit/juanako-7b-v1,load_in_4bit=False,dtype=float16), limit: None, num_fewshot: 3, batch_size: 4
31
+ ```
32
+ |Tasks|Version| Filter | Metric |Value | |Stderr|
33
+ |-----|-------|----------|-----------|-----:|---|-----:|
34
+ |gsm8k|Yaml |get-answer|exact_match|0.4556|± |0.0137|
35
+
36
  ## Model description
37
 
38
  **It seems to outperforms the original Zephyr in most of the tasks.**
 
54
  alignment-handbook DPO with UNA on top of the SFT lora.
55
 
56
  ### Evaluation lm-evaluation-harness
57
+
58
+ #### GSM8K
59
+ ```
60
+ hf (pretrained=/root/juanako-7b-v1-beta,load_in_4bit=False,dtype=float16), limit: None, num_fewshot: 3, batch_size: 4
61
+ ```
62
+ |Tasks|Version| Filter | Metric |Value | |Stderr|
63
+ |-----|-------|----------|-----------|-----:|---|-----:|
64
+ |gsm8k|Yaml |get-answer|exact_match|0.4556|± |0.0137|
65
+
66
  #### 0-Shot
67
  ```
68
  hf (pretrained=fblgit/juanako-7b-v1,load_in_4bit=False,dtype=float16), limit: None, num_fewshot: 0, batch_size: 8