AWQ Quant of Tarek07/Legion-V2.1-LLaMa-70B

AWQ quant of Tarek07/Legion-V2.1-LLaMa-70B using AutoAWQ for quantization.

Model was quantized down to INT4 using GEMM kernels, with zero-point quantization and a group size of 128.

Safetensors

Model size

71B params

Tensor type

I32

BF16

Model tree for ArtusDev/Tarek07_Legion-V2.1-LLaMa-70B-AWQ

Base model

Quantized

(14)

this model