fix metrics for LongEmbed

#124
Massive Text Embedding Benchmark org
No description provided.
Massive Text Embedding Benchmark org

This is a PR from https://huggingface.co/datasets/mteb/results/discussions/49#66679ee70ec7ed4422cfd4be . I'm trying to fix the metric used for LongEmbed in the leaderboard. For now, the leaderboard uses ndcg_at_10 for all the retrieval tasks, but we would like to use ndcg_at_1 instead for the LEMBPasskeyRetrieval and LEMBNeedleRetrieval test. I have not managed to run the demo myself due to some networking issues, but I guess the modifications will work.

Muennighoff changed pull request status to open
Muennighoff changed pull request status to merged

Sign up or log in to comment