auto-round-best --model /mnt/storage/models/gemma-4-31B-it/ --batch_size 1 --iters 500 --act_dtype float16 --output_dir /mnt/storage/models/ --format "auto_gptq" --bits 4 --group_size 32 --device 0,1 --fp_layers ".5.self_attn,.11.self_attn,.17.self_attn,.23.self_attn,.29.self_attn,.35.self_attn,.41.self_attn,.47.self_attn,.53.self_attn,.59.self_attn"

Downloads last month: 32

Safetensors

Model size

8B params

Tensor type

I32

BF16

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for hjc4869/gemma-4-31B-it-w4g32

Base model

google/gemma-4-31B-it

Quantized

(164)

this model