GPTQ vs Q4 GGUF
#2
by ciprianv - opened
Thank you for providing these GPTQ version. How should it compare as accuracy vs standard Q4 GGUF. Is there any post quantization training done? (QAT). In vllm/sglang does this has advantages over AWQ?
I also wonder about this, but from testing this model does very well. I imagine having the original training set to use as the dataset would provide a good model.