Hugging Face's logo

Intel
/
Llama-2-7b-chat-hf-int4-inc

English
Model card Files Community
yintongl commited on
Commit
b349150
•
1 Parent(s): 61b64f4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -50,7 +50,7 @@ Install [lm-eval-harness](https://github.com/EleutherAI/lm-evaluation-harness.gi
50
  lm_eval --model hf --model_args pretrained="Intel/Llama-2-7b-chat-hf-int4-inc",autogptq=True,gptq_use_triton=True --device cuda:0 --tasks lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,rte,arc_easy,arc_challenge,mmlu --batch_size 32
51
  ```
52
 
53
- | Metric | FP16 | int4 qdq |
54
  | -------------- | ------ | -------- |
55
  | Avg. | 0.5794 | 0.5787 |
56
  | mmlu | 0.4640 | 0.4620 |
 
50
  lm_eval --model hf --model_args pretrained="Intel/Llama-2-7b-chat-hf-int4-inc",autogptq=True,gptq_use_triton=True --device cuda:0 --tasks lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,rte,arc_easy,arc_challenge,mmlu --batch_size 32
51
  ```
52
 
53
+ | Metric | FP16 | int4 |
54
  | -------------- | ------ | -------- |
55
  | Avg. | 0.5794 | 0.5787 |
56
  | mmlu | 0.4640 | 0.4620 |