Hugging Face's logo

Spaces:
open-llm-leaderboard
/
open_llm_leaderboard
Running on CPU Upgrade

App Files Community
1152

[BUG] Gemma2-9b-it evaluation

#849
by DeepMount00 - opened
Discussion
DeepMount00

In my opinion, there is a bug in the evaluation of gemma-9b-it. The reported scores are very low, whereas the scores of gemma-9b-it should be much higher.
danielus

Yeah, i use Gemma 2 9b every day, for tasks like summarization, also in italian, a really good model

alozowski
 Open LLM Leaderboard org

Hi everyone!

Yes, we are looking into this issue right now, and this problem is being discussed here – please, feel free to write there in case of any questions

alozowski changed discussion status to closed

Sign up or log in to comment