adpena/Vertigo-Qwen3.5-4B-v0.5-4bit

Vertigo-Qwen3.5-4B-v0.5 (Archived Experiment)

⚠️ ARCHIVED — This model is an early experiment and should NOT be used in production. It is preserved here for research reproducibility only.

What This Is

An early LoRA fine-tune of Qwen3.5-4B for Roblox Luau code generation, trained as part of the Vertigo game engine project.

Why It's Archived

Rigorous execution-backed evaluation revealed that this adapter provides zero improvement in functional code generation over the base model:

Metric	Base Qwen3.5-4B	This Adapter
Pattern-match score	43.8%	48.0% (+4.2pp)
Execution pass@1	9.5%	9.7% (not significant)

The pattern-match improvement (+4.2pp) measures keyword presence, not working code. When evaluated on whether generated code actually compiles and passes test assertions, the adapter shows no improvement.

Key Findings

1. LoRA SFT on pattern-match-scored data does not improve execution pass rate — the training signal was disconnected from functional correctness
2. The Qwen3.5-4B hybrid architecture (24 GatedDeltaNet + 8 standard attention layers) requires targeting both self_attn and linear_attn LoRA keys
3. An agentic repair loop on the larger 35B-A3B model achieves 43.3% execution pass rate on calibrated tasks — without any training

Full technical report: vertigo-ml/docs/technical-report-lora-v1.md

Training Details

• Base: mlx-community/Qwen3.5-4B-4bit
• Method: LoRA rank-32, scale 20.0 (later found to be 10x too high)
• Data: ~2200 examples, pattern-match quality filtered
• Hardware: 128GB Apple Silicon, MLX

Don't Use This Model

Use the base Qwen3.5-4B model directly, or better yet, use a larger model (35B-A3B) with an agentic repair loop. See the vertigo-ml repository for the benchmark and evaluation infrastructure.