Leaderboard isn't updating its model list.

#809
by nlpguy - opened

As mentioned in another thread, the leaderboard doesn't update, but unlike what I assumed, a restart didn't fix it. Here is an example model that isn't shown on the leaderboard despite being evaluated:

https://huggingface.co/datasets/open-llm-leaderboard/openchat__openchat-3.6-8b-20240522-details

And on an unrelated note, commit 9b8c53c broke model hiding, as it hides everything but merges when enabling the hide merges option.

Open LLM Leaderboard org
edited Jun 28

The commit is completely unrelated ^^

The restart is different from the update, as we have a specific parser space which goes from all the result files to the leaderboard contents - we are therefore doing the time consuming step elsewhere, which allows to have a restart in a couple minutes. It's in the parser space that we're having an issue with the model hiding btw, we were discussing it

The Model I mentioned is now displayed, closing for now.

nlpguy changed discussion status to closed

Sign up or log in to comment