Qwen 2.5 3B IRS Tax Code — GGUF (q8_0)

A fine-tuned version of Qwen/Qwen2.5-3B-Instruct trained on the IRS tax code using a 3-stage training pipeline. This repository contains the quantized GGUF model for local inference via llama.cpp, Ollama, or compatible runtimes.

Model Details

Property	Value
Base Model	Qwen/Qwen2.5-3B-Instruct
Parameters	~3 Billion
Quantization	q8_0 (8-bit, high quality)
Format	GGUF
License	Apache 2.0
Training Domain	IRS Tax Code / US Federal Tax Law

Training Pipeline

This model was produced by a 3-stage fine-tuning pipeline applied to Qwen 2.5 3B Instruct:

Stage 1 — Supervised Fine-Tuning (SFT)

The base instruct model was fine-tuned with LoRA adapters on a curated dataset of IRS tax code passages, definitions, and question-answer pairs. This stage establishes domain knowledge and instills the correct response format for tax-related queries.

Stage 2 — Direct Preference Optimization (DPO)

Using pairs of preferred and rejected responses to tax questions, DPO training aligned the model to produce accurate, well-structured answers that reflect correct tax law interpretation over hallucinated or misleading responses.

Stage 3 — Group Relative Policy Optimization (GRPO)

GRPO reinforcement learning was applied to further optimize response quality using a reward model that evaluated accuracy, completeness, and citation of relevant tax code sections. This stage improves the model's ability to reason through multi-step tax scenarios.

Files

File	Description
`qwen25-tax-3b-q8_0.gguf`	q8_0 quantized GGUF for local inference

LoRA Adapters

The raw LoRA adapter weights for all three training stages are available in the companion repository: dennisonb/qwen25-tax-3b-adapters

Usage

With Ollama

ollama create qwen25-tax-3b -f Modelfile
ollama run qwen25-tax-3b "What is the standard deduction for a single filer in 2024?"

With llama.cpp

./llama-cli -m qwen25-tax-3b-q8_0.gguf -p "What are the income limits for contributing to a Roth IRA?" -n 512

Intended Use

This model is intended as a research and educational tool for exploring IRS tax code information. It is not a substitute for professional tax advice. Always consult a qualified tax professional for your specific situation.