victor/qwen3_5_2b-japanese-lora

Qwen3.5-2B Japanese LoRA

LoRA adapter trained on a curated Japanese instruction mixture:

• kunishou/databricks-dolly-15k-ja
• kunishou/oasst1-89k-ja
• kunishou/hh-rlhf-49k-ja

Training setup

• Framework: mlx-lm
• Base model: Qwen/Qwen3.5-2B (mlx-community/Qwen3.5-2B-bf16 locally)
• LoRA depth: 16 layers
• Learning rate: 2e-5
• Batch size: 2
• Gradient accumulation: 8
• Iterations: 400
• Max sequence length: 1024
• Prompt masking: enabled

Data curation

• Removed known ng_translation rows where available
• Prompt length: 4–1200 chars
• Completion length: 20–1200 chars
• Japanese character ratio >= 0.15
• Deduped to one canonical response per prompt hash

Result

• Base model: test_loss=2.988
• This adapter: test_loss=2.495
• Test perplexity: 12.127

MLX usage

mlx_lm.generate \
  --model ./model \
  --adapter-path ./adapters \
  --prompt "日本語で自己紹介してください。"