Multimodality not working
{"error":{"code":500,"message":"image input is not supported - hint: if this is unexpected, you may need to provide the mmproj","type":"server_error"}}
Im getting this error in llama.cpp (using the ghcr.io/ggml-org/llama.cpp:server-cuda llama.cpp build)
do i need to compile it from source to get it to work or do i need to change something ?
/app/llama-server --port 9999 -m /root/.cache/llama.cpp/Qwen3.5-122B-A10B-UD-Q4_K_XL-00001-of-00003.gguf --tensor-split 35,16 --n-cpu-moe 20 --ctx-size 40000 --batch-size 4096 --jinja --flash-attn on
this is my run command.
it starts properly and outputs text, but when adding an image it throws the above error.
anyone had these problems?
Hi,
You need add --mmproj, like this
/app/llama-server --port 9999 -m /root/.cache/llama.cpp/Qwen3.5-122B-A10B-UD-Q4_K_XL-00001-of-00003.gguf --mmproj /root/.cache/llama.cpp/unsloth_Qwen3.5-122B-A10B-GGUF_mmproj-BF16.gguf --tensor-split 35,16 --n-cpu-moe 20 --ctx-size 40000 --batch-size 4096 --jinja --flash-attn on
Thank you , i found this out now and its working !<3