Spaces:
Running
on
CPU Upgrade
GREAT work, but please add DATES / at least last update date
First - well done, it's a very serious and useful leaderboard!
Please add dates to each model (row) - OpenAI probably keeps updating its embedding models, maybe other models do too. If you evaluated all the table together and don't plan to update in the future - please add last update date.
Many thanks!
For most models, the evaluation is done by the model owners themselves, so if their model changes they can just update the scores that they report and the leaderboard would reflect those changes (e.g. https://huggingface.co/hkunlp/instructor-xl).
Generally, it's more common anyways for people to create a new v2 version of their model if sth changes, e.g. https://huggingface.co/intfloat/e5-large-v2
Similarly, OpenAI always versions their models i.e. text-embedding-ada-002 will stay the same. If something major changes, it'd be ..003 and would warrant a new model row.
Hence I'm not worried about any of the scores being outdated.
Thanks for the super quick response!
Your assumption that models in the same version don't change sound solid for open-source models, but for closed models it might not be the case, e.g. recently OpenAI announced on gpt-3.5/4 0613 versions that will replace the main versions in 3 months...
I think that adding an evaluation date column can only help :-)