Error: 500 Internal Server Error: unable to load model
Encountering an issue where models fail to run on Ollama v0.12.3 with OpenWebUI v0.6.32. The system consistently returns a 500 Internal Server Error and is unable to load the requested model.
CMD Logs:
ollama pull hf.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF:Q4_K_M
pulling manifest
pulling be8452823145: 100% ▕██████████████████████████████████████████████████████████▏ 6.5 GB
pulling e50819c9c50e: 100% ▕██████████████████████████████████████████████████████████▏ 165 B
pulling 72fd1b50d4da: 100% ▕██████████████████████████████████████████████████████████▏ 76 B
pulling e6bfe11f489a: 100% ▕██████████████████████████████████████████████████████████▏ 567 B
verifying sha256 digest
writing manifest
success
ollama run hf.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF:Q4_K_M
Error: 500 Internal Server Error: unable to load model: C:\Users\notyu1337.ollama\models\blobs\sha256-be84528231457f766a35612386e2638d9af10beb10a9d8d23f0cc73c20818da7
I'd have to look but it's possible ollama hasn't merged support for this architecture yet?
Sadly no, there doesn't seem to even be an issue open for this.
Edit: oh never mind:
https://github.com/ollama/ollama/issues/12126
But still not merged.