quant_method Question?
#2
by glrra30 - opened
You list this model as AWQ, but config.json shows "quant_method": "compressed-tensors"
Was this quantized with vllm-project/llm-compressor ??
Just trying to understand why it is not outputting "quant_method": "awq". ??
Current vllm capability with rocm looks for awq tag with ['w_bit', 'bits'] in the model's quantization config