Harmonic-2B-GGUF

GGUF quantizations of Harmonic-2B - a reasoning-focused draft model fine-tuned from Qwen 3.5 2B, designed for speculative decoding with the larger Harmonic models.

Support This Work

I'm a PhD student in visual neuroscience at the University of Toronto who also happens to spend way too much time fine-tuning, merging, and quantizing open-weight models on rented H100s and a local DGX Spark. All training compute is self-funded — balancing GPU costs against a student budget. If my uploads have been useful to you, consider buying a PhD student a coffee. It goes a long way toward keeping these experiments running.

Support on Ko-fi

---

Available Quantizations

Quantization	Size	Use Case
F16	3.6 GB	Full precision, no quality loss
Q8_0	1.9 GB	Near-lossless, recommended for most users
Q4_K_M	1.2 GB	Smallest, great for edge/mobile

Recommended Quantization

Q8_0 for most users - near-lossless quality, runs on basically anything.

Q4_K_M for maximum speed as a speculative decoding draft model.

Usage

Ollama

echo 'FROM ./Harmonic-2B-Q8_0.gguf' > Modelfile
ollama create harmonic-2b -f Modelfile
ollama run harmonic-2b

llama.cpp

./llama-cli -m Harmonic-2B-Q8_0.gguf -p "Solve this step by step:" -n 512

LM Studio

Download the GGUF file and load it directly in LM Studio.

Speculative Decoding

This model is built as a draft model for Harmonic-27B. Both share the same training data and reasoning patterns for high acceptance rates.

Model Details

• Base: Qwen 3.5 2B (2.27B parameters)
• Training: 799 curated reasoning rows, LoRA fine-tuned
• Context: 8192 tokens
• Reasoning format: Uses think blocks for structured reasoning

See the full model card for training details and data quality metrics.

License

Apache 2.0

Links

• Source model: DJLougen/Harmonic-2B
• 27B target: DJLougen/Harmonic-27B
• 27B GGUF: DJLougen/Harmonic-27B-GGUF
• 9B variant: DJLougen/Harmonic-9B
• 9B GGUF: DJLougen/Harmonic-9B-GGUF
• Agentic 9B: DJLougen/Harmonic-Hermes-9B