Can you share the quantize code?
#3
by LukeJacob2023 - opened
https://huggingface.co/DevParker/VibeVoice7b-low-vram/blob/main/4bit/quantize_and_save_vibevoice.py
this code works well with 4bit, but noise with 8bit.
https://huggingface.co/DevParker/VibeVoice7b-low-vram/blob/main/4bit/quantize_and_save_vibevoice.py
this code works well with 4bit, but noise with 8bit.