Spaces:
Running on CPU Upgrade
Running on CPU Upgrade
Tired of waiting in queue. How to eval locally?
#1103
by ccocks-deca - opened
How to run same eval scripts locallly?? I don't see anything as "open and reproducible" without there being any documentation. All three README's are useless. I do see the Standalone Installation (without Docker) section but I can't seem to find out if it's just the voting/submitting/viewing app or it's the entire evaluator.
@icefog72 TY!
Thank you @icefog72 !
Yes, to evaluate a model locally you can use either lm-evaluation-harness or to reproduce the Leaderboard scores as we do it, please check "Reproducibility" section in our FAQ
Feel free to ping me in case of any questions here!
alozowski changed discussion status to closed