Hugging Face's logo

Spaces:
mteb
/
leaderboard
Restarting on CPU Upgrade

App Files Community
182

fix metrics for LongEmbed

#124
by dwzhu - opened
base: refs/heads/main
←
from: refs/pr/124
Discussion Files changed
+16
-2
dwzhu
 Massive Text Embedding Benchmark org
No description provided.
init branchb12b1dc2
dwzhu
 Massive Text Embedding Benchmark org

This is a PR from https://huggingface.co/datasets/mteb/results/discussions/49#66679ee70ec7ed4422cfd4be . I'm trying to fix the metric used for LongEmbed in the leaderboard. For now, the leaderboard uses ndcg_at_10 for all the retrieval tasks, but we would like to use ndcg_at_1 instead for the LEMBPasskeyRetrieval and LEMBNeedleRetrieval test. I have not managed to run the demo myself due to some networking issues, but I guess the modifications will work.

small fix877acad4
Muennighoff changed pull request status to open
Muennighoff changed pull request status to merged

Sign up or log in to comment