rchow93/crewai-qwen3-8b-gguf

CrewAI Code Generation - Qwen3 8B (GGUF)

Fine-tuned Qwen3-8B models for generating CrewAI multi-agent code from natural language descriptions.

Models Available

V2 Thinking Models (Recommended)

File	Size	Description
crewai-qwen3-8b-thinking-v2-q4_k_m.gguf	4.7 GB	4-bit quantized, best balance
crewai-qwen3-8b-thinking-v2-q8_0.gguf	8.2 GB	8-bit quantized, higher quality

V2 Improvements:

• Native Qwen3 chat template (ChatML format)
• Higher LoRA rank (r=32 vs r=16)
• Thinking mode with reasoning tags
• Better structured outputs

V1 Models (Legacy)

File	Size	Description
qwen3-8b.Q4_K_M.gguf	~4.7 GB	4-bit quantized
qwen3-8b.Q5_K_M.gguf	~5.5 GB	5-bit quantized
qwen3-8b.Q8_0.gguf	~8.2 GB	8-bit quantized

Usage with Ollama

1. Download the GGUF file
2. Create a Modelfile:

FROM ./crewai-qwen3-8b-thinking-v2-q4_k_m.gguf

TEMPLATE """{{- if .System }}
<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

SYSTEM """You are a CrewAI code generation expert. When given a task description and required inputs, generate complete, working CrewAI Python code that includes:
- All necessary imports (crewai, crewai_tools)
- Agent definitions with roles, goals, backstories, and tools
- Task definitions with proper context dependencies
- Crew instantiation with appropriate process type
- Kickoff code with the provided inputs

Think through the problem step by step before generating code."""

PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER num_ctx 8192
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|im_start|>"

3. Create and run:

ollama create crewai-qwen3-8b -f Modelfile
ollama run crewai-qwen3-8b

Example Prompt

Create a CrewAI crew for analyzing competitor websites.
Required inputs:
- competitor_urls: list of URLs to analyze
- analysis_focus: what aspects to focus on

Training Details

• Base Model: Qwen/Qwen3-8B
• Fine-tuning: Unsloth + QLoRA (r=32, alpha=32)
• Dataset: 2,500 CrewAI code examples
• Training: 3 epochs on RTX 4090

License

Apache 2.0 (same as base Qwen3 model)