mtasic85 commited on
Commit
3d7c53e
1 Parent(s): 3a866cc
Files changed (1) hide show
  1. README.md +0 -1
README.md CHANGED
@@ -298,7 +298,6 @@ litgpt evaluate --tasks 'mmlu,mmlu_pro' --out_dir 'evaluate-mmlu/' --batch_size
298
  | - stem | 2|none | |acc |↑ |0.2157|± |0.0073|
299
  |mmlu_pro | 2|custom-extract| |exact_match|↑ |0.0000|± |0.0000|
300
 
301
-
302
  ```bash
303
  litgpt evaluate --tasks 'arc_challenge,boolq,gpqa,hellaswag,openbookqa,piqa,truthfulqa_mc2,winogrande' --out_dir 'evaluate-reasoning/' --batch_size 4 --dtype 'bfloat16' out/pretrain/final/
304
  ```
 
298
  | - stem | 2|none | |acc |↑ |0.2157|± |0.0073|
299
  |mmlu_pro | 2|custom-extract| |exact_match|↑ |0.0000|± |0.0000|
300
 
 
301
  ```bash
302
  litgpt evaluate --tasks 'arc_challenge,boolq,gpqa,hellaswag,openbookqa,piqa,truthfulqa_mc2,winogrande' --out_dir 'evaluate-reasoning/' --batch_size 4 --dtype 'bfloat16' out/pretrain/final/
303
  ```