Spaces:
Running
on
CPU Upgrade
Update src/submission/check_validity.py
Hello!
This is a small fix for correct model parameter size.
Because of the regex r"(\d\.)?\d+(b|m)", sometimes it returns wrong model size if it tries to extract model size from its name.
For example, Sao10K/Frostwind-10.7B-v1 currently has 0.7 param size in the leaderboard, which is wrong.
I hope this improves leaderboard accuracy a bit!
Cheers
Following the creation of this PR, an ephemeral Space HuggingFaceH4/open_llm_leaderboard-ci-pr-509 has been started. Any changes pushed to this PR will be synced with the test Space.
Since this PR has not been created by a trusted author, the ephemeral Space has not been configured with the correct hardware, storage, and secrets. An admin must configure it manually.
(This is an automated message.)
Hi ! Thanks for the fix, do you think you can also fix the typo where we define size_pattern two times ?
I'll do it, merging this so it gets integrated now :)
PR is now merged/closed. The ephemeral Space has been deleted.
(This is an automated message.)