Q6 & Q8
#1
by RuneXX - opened
Any chance for higher quants?
And thanks for those already there, works great ;-)
thanks a ton ;-) I know Q5_K is usually plenty ... But nice to have the Q6 and Q8 as options
Any chance for higher quants?
And thanks for those already there, works great ;-)
? the full model is only 20gb anyways so why?
not sure why, but my end the GGUF models seems to work better memory wise.
Often prefer the gguf ones
Arunk25 changed discussion status to closed