HauhauCS

/

Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive

like 1.36k

🧠 Running Qwen3.5 (Uncensored) GGUF in Ollama — Fix for “unable to load model”

If you’re trying to run models like:

hf.co/HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive:Q4_K_M

and getting:

Error: 500 Internal Server Error: unable to load model

— you’re not alone. This happens even when the model downloads successfully.

🚨 Problem

Ollama currently does not fully support some external GGUF models (e.g. Qwen3.5) when loading directly via:

ollama run hf.co/...

The model downloads fine, but fails during the load stage.

⸻

✅ Solution (Works)

Instead of running directly from Hugging Face, you need to:
1. Extract the GGUF file
2. Create a local model using a Modelfile

⸻

🔍 Step 1 — Find the downloaded GGUF

Ollama stores models here:

~/.ollama/models/blobs/

Find the large blob (usually several GB). Then verify it’s actually a GGUF:

xxd -l 32 ~/.ollama/models/blobs/sha256-XXXX

You should see:

GGUF

⸻

📦 Step 2 — Copy it as a .gguf file

cp ~/.ollama/models/blobs/sha256-XXXX
~/qwen35.gguf

⸻

⚙️ Step 3 — Create a Modelfile

cat > ~/Modelfile <<'EOF'
FROM /Users/YOUR_USERNAME/qwen35.gguf

TEMPLATE {{ .Prompt }}
RENDERER qwen3.5
PARSER qwen3.5

PARAMETER top_p 0.95
PARAMETER temperature 1
PARAMETER top_k 20
PARAMETER presence_penalty 1.5
EOF

⸻

🚀 Step 4 — Create and run the model

ollama create qwen35-unc -f ~/Modelfile
ollama run qwen35-unc

You should now see:

Send a message (/? for help)

⸻

💡 Notes
• This workaround fixes loading issues for Qwen3.5 GGUF models
• Works for text-only usage
• Vision/multimodal features are not supported with this method
• The issue is related to Ollama’s current GGUF/architecture support

⸻

🧩 Summary

Method Status
ollama run hf.co/... ❌ Fails
Local .gguf + Modelfile ✅ Works

⸻

If you’re stuck with unable to load model — this is the cleanest workaround right now.