Please consider re-quantization, it could improve the support and overall quality of the model.
#2
by MrDevolver - opened
This model seems to be quantized before the first but very important fix was merged into Llama.cpp. Check Unsloth version for more info as they already updated their own quants and readme file.
PS: Your REAP version seems to be more recent, so that one should be fine.
Yes, was cooking some quants for it with an experimental imatrix also, so will update in a bit!
noctrex changed discussion status to closed