Evaluation
#3
by Michalea - opened
Hello, thank you for contribution to open-source.
Nevertheless, according to your paper; REAP is able to significantly decrease the quality for small models for the world-knowledge/QA tasks.
You present the accuracy preservation on code tasks which are quite ok with your method.
Adding some QA benchmarks like MMLU-Pro/GPQA would be beneficial. 😎