SpartQA score error?
Love to see people training larger embedding models and even more excited to see that you shared the training data and code.
I noticed is that your SpartQA score (on the MTEB leaderboard) of 84.83 is 54.59 higher than the second highest score (which is already an outlier). Would you mind double checking it and helping me establish if it's correct or not?
Evaluation: Our model was evaluated directly using the MTEB framework, and we encourage independent reproduction of these results.
Training: We have confirmed that the SpartQA evaluation specifically utilizes the test split . Our training process exclusively employed the train split; a comprehensive list of our training datasets is available at mteb model implementations
However, we are currently conducting a secondary check to determine if any potential data leakage or significant overlap exists between the original SpartQA train and test splits.