tdelard/Qwen3-4B-DBT-Instruct-LoRA

Qwen3-4B-DBT-Instruct — LoRA Adapter

QLoRA adapter fine-tuned on top of unsloth/Qwen3-4B-Instruct-2507-unsloth-bnb-4bit (the pre-quantized 4-bit Unsloth variant of Qwen3-4B-Instruct) to convert natural language business questions into complete, multi-file dbt DAGs.

Looking for the ready-to-run GGUF (LM Studio / Ollama)? → tdelard/Qwen3-4b-DBT-Instruct-GGUF

---

What does this model do?

Given a business question and a SQL schema, the model generates a full dbt project structure:

• Staging layer — stg_*.sql files that clean and rename raw source data
• YAML sources — _sources.yml / _stg_*.yml schema files with column definitions
• Intermediate models — int_*.sql files that join and enrich staging data
• Marts layer — fct_*.sql or dim_*.sql final business models

---

Usage

This repo contains only the LoRA adapter weights (264 MB). You need to load the base model alongside the adapter.

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model_id = "unsloth/Qwen3-4B-Instruct-2507-unsloth-bnb-4bit"
adapter_id    = "tdelard/Qwen3-4B-DBT-Instruct-LoRA"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
model     = AutoModelForCausalLM.from_pretrained(base_model_id, torch_dtype="auto", device_map="auto", load_in_4bit=True)
model     = PeftModel.from_pretrained(model, adapter_id)

messages = [
    {
        "role": "system",
        "content": (
            "You are a dbt expert. Given a business question and a SQL schema, "
            "generate a complete, production-ready dbt DAG including staging SQL files, "
            "YAML schema files, intermediate models, and mart models. "
            "Use proper dbt conventions: ref(), source(), naming prefixes (stg_, int_, fct_, dim_)."
        ),
    },
    {
        "role": "user",
        "content": (
            "Business question: Show the total revenue per product category.\n"
            "SQL context: CREATE TABLE orders (order_id INT, product_id INT, amount DECIMAL); "
            "CREATE TABLE products (product_id INT, category VARCHAR, name VARCHAR);"
        ),
    },
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=1024, temperature=0.7, top_p=0.8, top_k=20)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

---

Training details

Parameter	Value
Base model	unsloth/Qwen3-4B-Instruct-2507-unsloth-bnb-4bit
Training framework	Unsloth + TRL SFTTrainer
Method	QLoRA
LoRA rank	32
LoRA alpha	32
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Learning rate	2e-4
Max sequence length	2048 tokens
Training loss masking	Responses only (train_on_responses_only)
Hardware	Google Colab T4 GPU (15 GB VRAM)
Training dataset	tdelard/text_to_dbt — 900 train / 100 eval

---

Source code

Full training pipeline: thomasdlr/qwen3-dbt-instruct

---

License

Apache 2.0 — same as the base Qwen3-4B / Unsloth model.

---

Fine-tuned 2x faster using Unsloth.