qwen3-4b-structured-output-lora-v5.4

LoRA adapter for structured output generation (v5.4 Strategy A).

Strategy A: v5.2 Base + Low LR Targeted Learning

This adapter was created by:

1. Loading v5.2 trained adapter (kmd2525/qwen3-4b-structured-output-lora-v5.2)
2. Merging into base model
3. Additional SFT with 17 targeted samples (TOML 14, XML 3)
4. Very low learning rate (1e-6) to preserve v5.2 learning

Training Details

• Base: v5.2 merged model (94.7% accuracy)
• Additional data: 17 targeted samples
• Learning rate: 1e-6 (very low)
• Epochs: 1
• LoRA r: 64
• LoRA alpha: 128

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = "Qwen/Qwen3-4B-Instruct-2507"
adapter = "your_id/qwen3-4b-structured-output-lora-v5.4"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(
    base,
    torch_dtype=torch.float16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter)

Note

This model builds upon v5.2 with targeted fixes for:

• TOML inline table syntax
• TOML array table syntax
• XML special character escaping