How to create GGUF?
Hi befox, Thanks a lot for your work creating the ggufs.
Just curious to know how you converted the safetensors to gguf. I tried some methods but everything got errored or the size becomes doubled and not loading in comfyui.
Please provide the tool name/steps if possible.
I used tools from city96's ComfyUI-GGUF to convert safetensors to gguf. There's also a step by step guide on the repo too.
The script may have specific requirements for how the tensor keys are named, so the input safetensors file might need to be preprocessed first (usually, rename the key to meet requirement of convert.py is enough).
I Will try using this.
Thanks a lot, befox!
I found a better tool, but it only supports fp16 format, and further quantization fails thereafter.
https://github.com/calcuis/gguf