Spaces:
Running
on
CPU Upgrade
New Embedding Models | Apply for refershing the results
I have added the results of STS Benchmarks for New Arabic Embedding Models and they are listed below:
- Omartificial-Intelligence-Space/Arabic-mpnet-base-all-nli-triplet
- Omartificial-Intelligence-Space/Arabert-all-nli-triplet-Matryoshka
- Omartificial-Intelligence-Space/Arabic-labse-Matryoshka
- Omartificial-Intelligence-Space/Marbert-all-nli-triplet-Matryoshka
- Omartificial-Intelligence-Space/MiniLM-L12-v2-all-nli-triplet
I have benchmarked using STS TASK and follow all the steps for the leaderboard and refreshed the page but I could not see it !!
Could you please refresh the space?
Hello!
I've refreshed the leaderboard, and I do see your models now (at least, 4 out of the 5). However, it appears that the scores in your metadata range from 0 to 1, whereas the scores in everyone else's metadata ranges from 0 to 100. Did you follow the "Adding a Model to the MTEB Leaderboard" steps? (Note: it's possible that rather than your work, some changes in MTEB caused this)
- Tom Aarsen
Hello, thank you
Actually, I was investigating the cause of this issue. I have followed the steps, but it appears the problem is with MTEB itself. Should I edit this manually, or is there a quicker way to resolve this issue?
I would open an issue on the MTEB repository, and then rerun your STS experiments with an older version of MTEB. That seems safer than trying to manually edit your scores.
- Tom Aarsen
Ok I will check that
Omar
I have successfully modified the evaluation code to return scores out of 100 instead of the previous 0-1 range. Additionally, I have updated the metadata in the README files for each model to reflect this change. If the results need to be refreshed to reflect these changes, Could you please refresh the space?
Sometimes the leaderboard working well showing the score of 100 but suddenly it goes back to 0-1 ranges .. I don’t know why