math-adaptroute
LoRA adapter for the math domain in AdaptRoute.
Mounted onto a frozen 4-bit NF4 quantised Qwen/Qwen2.5-1.5B at inference time via
peft.add_weighted_adapter() — weights provided by the gating network.
LoRA Config
- r = 16, alpha = 32, dropout = 0.05
- Target modules: ['q_proj', 'k_proj', 'v_proj', 'o_proj']
- Training: 2 epochs on 20000 samples, lr=0.0002
Training Data
- Source:
DigitalLearningGmbH/MATH-lighteval
- Downloads last month
- -
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for kunjcr2/math-adaptroute-v2
Base model
Qwen/Qwen2.5-1.5B