Any chance for same 4bit quants but in AWQ format?
#1
by MDSExpro - opened
vLLM version would be amazing!
vLLM version would be amazing!
Thanks for the tip! I didn't know MLX-LM also supports AWQ. I will give it a shot if they support MiniMax
vLLM version would be amazing!
I can't seem to dequant using Transformers lib on mps unfortunately, and mlx doesnt support M2 yet 😭
Will try again tomorrow.