GPTQ int4-int8 mixed
#4
by darkstar3537 - opened
Any plans for a GPTQ Int8 mixed int4? Your GLM 4.7 quant of that was much better than the AWQ.
This repo already mixed little bit of 16bit, though not as much as we did in GPTQ Int4-int8-mixed.
Consider Qwen3.5 series already having some "QAT" baked in, a specialized GPTQ mixed version is not currently planned, unless they are really needed, I guess.
This repo already mixed little bit of 16bit, though not as much as we did in GPTQ Int4-int8-mixed.
Consider Qwen3.5 series already having some "QAT" baked in, a specialized GPTQ mixed version is not currently planned, unless they are really needed, I guess.
Makes sense, thanks for another great quant!