spicyneuron/GLM-5.1-MLX-2.9bit

GLM 5.1 optimized for MLX. A mixed-precision quant that balances speed, memory, and accuracy.

Usage

# Start server at http://localhost:8080/chat/completions
uvx --from mlx-lm mlx_lm.server \
  --host 127.0.0.1 \
  --port 8080 \
  --model spicyneuron/GLM-5.1-MLX-2.9bit

Methodology

Quantized with a mlx-lm fork, drawing inspiration from Unsloth/AesSedai/ubergarm style mixed-precision GGUFs. MLX quantization options differ than llama.cpp, but the principles are the same:

• Sensitive layers like MoE routing, attention, and output embeddings get higher precision
• More tolerant layers like MoE experts get lower precision

Benchmarks

metric	baa-ai/GLM-5.1-RAM-270GB-MLX	2.9bit (this model)
bpw	3.1096	2.9064
peak memory (1024/512)	291.257	272.358
prompt tok/s (1024)	194.958 ± 0.075	194.216 ± 0.167
gen tok/s (512)	21.381 ± 0.050	19.527 ± 0.035
perplexity	4.780 ± 0.020	4.118 ± 0.016
hellaswag	0.546 ± 0.011	0.59 ± 0.011
piqa	0.776 ± 0.01	0.794 ± 0.009
winogrande	0.668 ± 0.013	0.695 ± 0.013

Tested on a Mac Studio M3 Ultra with:

mlx_lm.perplexity --sequence-length 2048 --seed 123
mlx_lm.benchmark --prompt-tokens 1024 --generation-tokens 512 --num-trials 5
mlx_lm.evaluate --tasks hellaswag --seed 123 --num-shots 0 --limit 2000
mlx_lm.evaluate --tasks piqa --seed 123 --num-shots 0 --limit 2000
mlx_lm.evaluate --tasks winogrande --seed 123 --num-shots 0 --limit 2000