2x7B AWQ
Collection
Mixture of experts 2 x 7B. • 20 items • Updated
R136a1/InfinityKumon-2x7B AWQ
- Model creator: R136a1
- Original model: InfinityKumon-2x7B
Model Summary
Another MoE merge from Endevor/InfinityRP-v1-7B and grimjim/kukulemon-7B.
The reason? Because I like InfinityRP-v1-7B so much and wondering if I can improve it even more by merging 2 great models into MoE.
Prompt format:
Alpaca or ChatML
- Downloads last month
- 4
Model tree for solidrust/InfinityKumon-2x7B-AWQ
Collection including solidrust/InfinityKumon-2x7B-AWQ
Evaluation results
- normalized accuracy on AI2 Reasoning Challenge (25-Shot)test set Open LLM Leaderboard69.620
- normalized accuracy on HellaSwag (10-Shot)validation set Open LLM Leaderboard87.090
- accuracy on MMLU (5-Shot)test set Open LLM Leaderboard64.970
- mc2 on TruthfulQA (0-shot)validation set Open LLM Leaderboard61.990
- accuracy on Winogrande (5-shot)validation set Open LLM Leaderboard81.930
- accuracy on GSM8k (5-shot)test set Open LLM Leaderboard63.530