Text Generation
Transformers
Safetensors
starcoder2
code
Eval Results
text-generation-inference
Inference Endpoints
compressed-tensors
alexmarques commited on
Commit
a3fd17b
1 Parent(s): dadaac7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -43,7 +43,7 @@ model-index:
43
  - **Model Developers:** Neural Magic
44
 
45
  Quantized version of [starcoder2-3b](https://huggingface.co/bigcode/starcoder2-3b).
46
- It achieves an average score of 68.69 on the [OpenLLM](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) benchmark (version 1), whereas the unquantized model achieves 68.54.
47
 
48
  ### Model Optimizations
49
 
 
43
  - **Model Developers:** Neural Magic
44
 
45
  Quantized version of [starcoder2-3b](https://huggingface.co/bigcode/starcoder2-3b).
46
+ It achieves a HumanEval pass@1 of 31.0, whereas the unquantized model achieves 30.7 when evaluated under the same conditions.
47
 
48
  ### Model Optimizations
49