younesAbaaoud/qwen3-4b_15k_v0

Darija Qwen2.5-1.5B — LoRA Adapter

Modele fine-tune sur Qwen/Qwen3-4B-Instruct-2507 pour le dialecte marocain (Darija).

Dataset

• MBZUAI-Paris/Darija-SFT-Mixture
• 500 samples selectionnes et nettoyes (MAX_TOKENS=512, MAX_LATIN_RATIO=0.6)

Configuration LoRA

• lora_rank: 8
• lora_alpha: 16
• lora_dropout: 0.05
• lora_target: all

Utilisation

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base  = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507", torch_dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(base, "darija_qwen_lora_adapter")
tokenizer = AutoTokenizer.from_pretrained("darija_qwen_lora_adapter")