Qwen 3.5 vision support
Decided to ask here as Kimi K2.5 had vision support from AesSedai - couldnt find any references on how (if possible) to run vision on Qwen3.5 in llampa.cpp - it seems it has it built in vision and doesnt use separate mmporj file
Idk about that (you still need separate vision encoder it seems, but it takes at most ~2 Gb) but adding to cmdline
--mmproj mmproj-BF16.gguf
from https://huggingface.co/unsloth/Qwen3.5-397B-A17B-GGUF/blob/main/mmproj-BF16.gguf
worked for unsloth-quantized llama.cpp (llama-server for example starts to allow passing images in), maybe it will work for this quant too because GPT says the mmproj is not dependent on quantization but only on architecture and layers/input/output.
Thanks - I've totally missed that!
@agentsmit The unsloth mmproj should work fine, I'll get them uploaded to my repo later today but they'll likely end up being basically identical I'd imagine.