mxfp4 QAT versions?
#4
by Dampfinchen - opened
Hi Qwen team,
Quantization aware trained models are pretty rare in the current open source landscape. Only OpenAI did with gpt-oss and Google with Gemma 3 QAT. I think it would be amazing if you could make QAT trained models of your Qwen 3.5 series, perhaps in mxfp4 format like gpt-oss. This would deliver much higher quality quants at the very commonly used 4 bit quantization than currently where BF16 models are quantized with a noticeable quality loss to 4 bit.
Please think about it. Best regards!