Code
#2
by ehartford - opened
Can you please share the code you used to quantize it?
I would love to be able to build my own!
You can use the recipe with llmcompressor: https://huggingface.co/RedHatAI/Qwen3.5-397B-A17B-FP8-dynamic/resolve/main/recipe.yaml
The last I checked, LLM compressor is a Python library not a CLI tool
Did that change?
If not - would you please share the python script you used to call the LLM compressor library?
ehartford changed discussion status to closed