metadata
license: apache-2.0
base_model: MiniMaxAI/MiniMax-M2.7
base_model_relation: quantized
tags:
- auto-round
- int4
- w4a16
- quantization
- moe
library_name: transformers
MiniMax-M2.7 INT4 AutoRound
4-bit quantized version of MiniMaxAI/MiniMax-M2.7 using Intel AutoRound.
Quantization Config
| Setting | Value |
|---|---|
| Scheme | W4A16 (INT4 weights, FP16 activations) |
| Group size | 128 |
| Ignored layers | MoE gate layers (kept at full precision) |
| Method | RTN (iters=0) |
Usage
vLLM
vllm serve Lasimeri/MiniMax-M2.7-int4-AutoRound \
--trust-remote-code \
--tensor-parallel-size 8 \
--enable-auto-tool-choice \
--tool-call-parser minimax_m2 \
--reasoning-parser minimax_m2_append_think
SGLang
python -m sglang.launch_server \
--model-path Lasimeri/MiniMax-M2.7-int4-AutoRound \
--trust-remote-code \
--tp 8 \
--reasoning-parser minimax-append-think \
--tool-call-parser minimax-m2
Quantization Hardware
Quantized on a single-node rig:
| Component | Spec |
|---|---|
| CPU | AMD EPYC 7742 (64C / 128T) |
| RAM | 251 GB DDR4 |
| GPUs | 8× RTX 3080 (20 GB modded) |
Peak resource usage during quantization: ~25.6 GB RAM, ~5 GB VRAM on GPU 0, ~1.3 GB on each remaining GPU.