Hugging Face's logo

Intel
/
Llama-2-7b-chat-hf-int4-inc

English
Model card Files Community
yintongl commited on
Commit
272c787
•
1 Parent(s): b349150

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -12
README.md CHANGED
@@ -50,19 +50,19 @@ Install [lm-eval-harness](https://github.com/EleutherAI/lm-evaluation-harness.gi
50
  lm_eval --model hf --model_args pretrained="Intel/Llama-2-7b-chat-hf-int4-inc",autogptq=True,gptq_use_triton=True --device cuda:0 --tasks lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,rte,arc_easy,arc_challenge,mmlu --batch_size 32
51
  ```
52
 
53
- | Metric | FP16 | int4 |
54
  | -------------- | ------ | -------- |
55
- | Avg. | 0.5794 | 0.5787 |
56
- | mmlu | 0.4640 | 0.4620 |
57
- | lambada_openai | 0.7105 | 0.7017 |
58
- | hellaswag | 0.5780 | 0.5699 |
59
- | winogrande | 0.6638 | 0.6592 |
60
- | piqa | 0.7639 | 0.7661 |
61
- | truthfulqa_mc1 | 0.3023 | 0.3146 |
62
- | openbookqa | 0.3340 | 0.3340 |
63
- | boolq | 0.7976 | 0.8064 |
64
- | arc_easy | 0.7382 | 0.7336 |
65
- | arc_challenge | 0.4420 | 0.4403 |
66
 
67
 
68
 
 
50
  lm_eval --model hf --model_args pretrained="Intel/Llama-2-7b-chat-hf-int4-inc",autogptq=True,gptq_use_triton=True --device cuda:0 --tasks lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,rte,arc_easy,arc_challenge,mmlu --batch_size 32
51
  ```
52
 
53
+ | Metric | BF16 | int4 |
54
  | -------------- | ------ | -------- |
55
+ | Avg. | 0.5797 | 0.5787 |
56
+ | mmlu | 0.4641 | 0.4620 |
57
+ | lambada_openai | 0.7101 | 0.7017 |
58
+ | hellaswag | 0.5779 | 0.5699 |
59
+ | winogrande | 0.6630 | 0.6592 |
60
+ | piqa | 0.7644 | 0.7661 |
61
+ | truthfulqa_mc1 | 0.3035 | 0.3146 |
62
+ | openbookqa | 0.3360 | 0.3340 |
63
+ | boolq | 0.7979 | 0.8064 |
64
+ | arc_easy | 0.7391 | 0.7336 |
65
+ | arc_challenge | 0.4411 | 0.4403 |
66
 
67
 
68