Machine readable Leaderboard?

#85
by adirendu - opened

Hi, I am trying to write a script to parse the leaderboard. I am currently looking at this file (https://huggingface.co/spaces/mteb/leaderboard/blob/main/EXTERNAL_MODEL_RESULTS.json) and assuming all the numbers in the table should be in it.

However I am not able to locate certain models in the JSON file. For example, GIST models don't seem to be in the JSON file. (a grep for grep 'GIST' EXTERNAL_MODEL_RESULTS.json returns nothing.

Is there something I'm doing wrong?

If this JSON does not have all the models, is there a source file that does?

Massive Text Embedding Benchmark org

That json only contains models that are added via https://huggingface.co/datasets/mteb/results but not models that added the mteb scores to their readme and are added that way.

It's very easy to create a json for all models by just saving the dataframe created in app.py before displaying it.

Sign up or log in to comment