AWQ 4/5/6-bit request for Qwopus3.5-27B-v3
#2
by celikburak - opened
Hi QuantTrio,
Jackrong just released Qwopus3.5-27B-v3.
Could you please make AWQ 4-bit, 5-bit and 6-bit quants when you have time?
We’re running on 5090 with vLLM + FlashInfer and your previous quants worked great.
Thanks in advance!
noted
Hi, even waiting for the awq versions for gemma 4 - 26B and 31B variants, thanks !
https://huggingface.co/QuantTrio/Qwopus3.5-27B-v3-AWQ
https://huggingface.co/QuantTrio/Qwopus3.5-27B-v3-AWQ-6Bit
working on gemma4 series, stay tuned