jeanbaptdzd/wagmi-qwen2.5-1.5b-sft

Wagmi -- Qwen 2.5 1.5B SFT (LoRA adapter)

Version: 0.2.0 | Changelog

LoRA adapter for Qwen/Qwen2.5-1.5B-Instruct, fine-tuned to serve as Wagmi, the AI assistant for Deal ex Machina -- a Paris-based tech consulting firm.

Intended use

Wagmi answers questions about Deal ex Machina's services, blog content, and founder in French and English. It is designed for a single-domain customer-facing chatbot embedded in the company website.

This model should not be used for:

• General-purpose chat or instruction following outside the Deal ex Machina domain.
• Medical, legal, financial, or safety-critical advice.
• Any decision-making with legal or significant effects on individuals.

Training

Parameter	Value
Method	LoRA (rank 32, alpha 64)
Target modules	q/k/v/o + gate/up/down proj
Max seq length	2048 tokens
Learning rate	2e-4 (cosine decay)
Epochs	2
Effective batch	16 (4 per device, grad accum 4)
Precision	bf16
Framework	Unsloth + TRL
Hardware	Hugging Face L40 (48 GB VRAM)

Dataset

~900 examples (v0.2.0), bilingual FR/EN:

• Company content (blog, site pages, Obsidian notes)
• Synthetic guardrails (identity, refusal, uncertainty)
• 121 adversarial security hardening entries (v0.2.0)
• No user conversations are used for training.

Full provenance tracked in data/metadata.json with per-entry id, source, locale, and tags.

Evaluation

Security smoke test (v0.2.0, 28 attack vectors)

Verdict	Count
PASS	23
SOFT	3
WARN	1
FAIL	1

Known weakness: refuses plain English code requests (e.g. "sort a list in Python"). Targeted for v0.3.0.

Autotune

GPT-4o judge scores on 6 criteria (factual accuracy, language match, tone/persona, guardrail compliance, conciseness, hallucination-free). Target: mean > 2.5/3.0.

Limitations

• Small model (1.5B parameters) has limited reasoning capacity.
• Domain-specific: will not perform well outside Deal ex Machina topics.
• Adversarial robustness is tested but not exhaustive (28 vectors).
• No continuous production monitoring; testing is per-release.

EU AI Act

This model powers a limited-risk AI system (customer-facing chatbot, Article 50 transparency tier). It is not used for high-risk decisions. The chat widget identifies itself as AI. See the full compliance documentation.

Related repos

Repo	Role
sft-wagmi	Training pipeline and dataset
wagmi-qwen2.5-1.5b-sft-merged	Merged BF16 model
wagmi-qwen2.5-1.5b-sft-gguf	Quantized GGUF (Q4_K_M)
dexm-one-page	Production site with RAG inference

Attribution

LoRA adapter and SFT dataset are original work by Deal ex Machina. Base model weights are the property of the Qwen team (Alibaba Cloud), licensed Apache 2.0.