younesAbaaoud/darija-gemma3-1-multigpu-lora_v1

Darija Gemma3-1B — LoRA (LLaMA-Factory + Unsloth + 2x T4)

Fine-tune sur google/gemma-3-1b-it pour le dialecte marocain (Darija).

Stack

• LLaMA-Factory (orchestration multi-GPU)
• Unsloth (backend rapide)
• 2x GPU T4 — bf16 pur (pas de quantization)

Dataset

• MBZUAI-Paris/Darija-SFT-Mixture — 5 000 samples (direction=None)

Config LoRA

• lora_rank: 16 | lora_alpha: 32 | target: all

Utilisation

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
base = AutoModelForCausalLM.from_pretrained(
    "google/gemma-3-1b-it", torch_dtype=torch.bfloat16, device_map="auto"
)
model     = PeftModel.from_pretrained(base, "REPO_ID")
tokenizer = AutoTokenizer.from_pretrained("REPO_ID")