Endless looping while analyzing image and thinking.
I deployed the Qwen3.5-35B-A3B-UD-Q4_K_M.gguf model in llama.cpp, when analyze image, I facing the issues as the image shown, it did not stop looping and endless thinking, and did not return the anwer to me. did anyone else has the same ? how to fix that?
I confirm, it's a very bad model. It doesn't exit the thought process, it gets stuck in a loop regularly. I tried running it without mmproj, but the result is the same. I downloaded a new version of gguf today, no improvements, it's inoperable.
Same thing for me... this model is unusable in a production env. Hope it will be fixed someday 😄
I tried using this model and it works quite stable.
Well you are lucky!
It might come from my setup...
Try this question in instruct mode:
Which word comes next: Stone, Often, Canine, _: A. Helpful, B. Freight, C. Glow, D. Grape
Here's my llama-swap configuration with the model:
"05_q3.5-35b-a3b-ist-q4_k_m":
cmd: |
--port ${PORT}
--offline
--jinja
--flash-attn on
-b 4096
-ub 2048
--parallel 1
--fit off
--reasoning-budget 0
--chat-template-kwargs "{"enable_thinking":false}"
-c 131072
-ngl 41
-dev ROCm3,ROCm4
--temp 0.7
--top-p 0.8
--top-k 20
--min-p 0.0
--presence-penalty 1.5
-m /mnt/bigfoot/models/huihui-Qwen3.5-35B-A3B-abliterated/Huihui-Qwen3.5-35B-A3B-abliterated-MXFP4_MOE.gguf
--mmproj /mnt/bigfoot/models/huihui-Qwen3.5-35B-A3B-abliterated/mmproj-F16.gguf
"Let's consider the Alphabetical Order of the first letters as the primary pattern: Sequence: S, O, C. If we sort them: C, O, S"
And a few moment later... the same pattern is back!
Yes, after more careful work with this quantization, I also encountered the same problems. Although it took longer to detect the glitch, I still find this quantization (Huihui-Qwen3.5-35B-A3B-abliterated-MXFP4_MOE.gguf) more stable and can be used for quite a long time without glitches, but the problems still persist.
I think we should expect version 3.5.1