gemma-4-31B-it-UD-IQ2_XXS.gguf is busted
Helping a friend troubleshoot this, they have limited vram. This quant is busted.
Ways to reproduce:
Shove 60k context at it and watch the reply be gibberish.
Helping a friend troubleshoot this, they have limited vram. This quant is busted.
Ways to reproduce:
Shove 60k context at it and watch the reply be gibberish.
What CUDA version are you guys using? Works fine for me
Cuda 12.9 here with a 3060 RTX I get similar gibberish with this quant.
UD IQ3XXS seems to be fine though so not sure why this one is broken.
Helping a friend troubleshoot this, they have limited vram. This quant is busted.
Ways to reproduce:
Shove 60k context at it and watch the reply be gibberish.
What CUDA version are you guys using? Works fine for me
He's on 12.9, I'm on 13.1. If I take your regular Q8 quant (the regular one) it works fine.
I asked another friend with a 96GB monster card to try it out, and they have the same issue.
Sometimes it will work for one message then next message it starts doing weird words like sticking Own into sentences that make no sense or repeating the word Same.