quantization tools and the code for the quantization configuration used
#1
by zanepoe - opened
Could you disclose the quantization tools and the code for the quantization configuration used?
Hi @zanepoe ,
My apologies for the late response.
The quantization tool I used is llm-compressor , with the quantization configuration described in this repo recipe.yaml.
The quantization code is kinda messy, I might open-source it in the future after I have cleaned it.
Thanks,
Ton