gemma-4-31b-it-oQ3

oQ3 mixed-precision MLX quantization produced via oMLX.

• Quantization: oQ3 (sensitivity-driven, group_size=64)
• Format: MLX safetensors, loadable with mlx-vlm and mlx-lm

Usage

pip install mlx-vlm
python3 -m mlx_vlm generate --model bearzi/gemma-4-31b-it-oQ3 --prompt "Your prompt here" --max-tokens 512

About oQ

oQ measures per-layer quantization sensitivity through calibration inference and allocates bits where they matter most — critical layers stay at higher precision, tolerant layers compress aggressively. See oMLX docs.