pinned
Running
on
CPU Upgrade
4
🥇
None defined yet.
This repository is a demo leaderboard template. You can copy the leaderboard space and the two datasets (results and requests) to your org to get started with your own leaderboard!
The space does 3 things:
requests
datasetmain_backend.py
file, using the Eleuther AI Harness. Results of running evaluations are then sent to results
You can also move the backend to its own space if you need, by grabbing main_backend and putting it in its own space, with a app.py which runs it every few minutes - it is probably the best solution.
To get started on your own leaderboard, you will need to edit 2 files:
src/envs.py
to define your own environment variable (like the org name in which this has been copied)src/about.py
with the tasks and number of few_shots you want for your tasksOnce this is done, you need to edit the "fake results" file to fit the format of your tasks: in the sub dictionary results
, replace task_name1 and metric_name by the correct values you defined in Tasks above.
"results": {
"task_name1": {
"metric_name": 0
}
}