metadata
base_model: Qwen/Qwen2.5-1.5B
library_name: peft
license: apache-2.0
tags:
- lora
- peft
- adaptroute
- code
code-adaptroute
LoRA adapter for the code domain in AdaptRoute.
Mounted onto a frozen 4-bit NF4 quantised Qwen/Qwen2.5-1.5B at inference time via
peft.add_weighted_adapter() — weights provided by the gating network.
LoRA Config
- r = 16, alpha = 32, dropout = 0.05
- Target modules: ['q_proj', 'k_proj', 'v_proj', 'o_proj']
- Training: 2 epochs on 20000 samples, lr=0.0002
Training Data
- Source:
iamtarun/python_code_instructions_18k_alpaca