Correct HumanEval scores
#79
by
Muennighoff
- opened
Previous scores did not strip end of sequence tokens. The updated scores ignore the end of sequence tokens (</s>).
This is equivalent to evaluating on code generations decoded with tokenizer.decode(code_tokens, skip_special_tokens=True)
Muennighoff
changed pull request title from
Update README.md
to Correct HumanEval scores
Muennighoff
changed pull request status to
merged