Gibberish
#1
by gtkunit - opened
Hi, thanks for the quant. I've tried to convert it myself but got gibberish. The same thing happens for me with your quant (exl3 works fine).
I'm on the latest exl2 dev branch. TabbyAPI or examples/chat.py give the same result.
Did you get it to work?
Hi, im very sorry, i didn't try the model before, only the exl3. The output is for me gibberish as well, there is now an issue for that.
Looks like exl3 is the way to go: https://github.com/turboderp-org/exllamav2/issues/772#issuecomment-2825374088