Hugging Face's logo

01-ai
/
Yi-9B

Text Generation
Transformers
Safetensors
llama
text-generation-inference
Model card Files
xet
 Community
15

How to convert safetensor model into gguf?

#2
by HildaM - opened
Discussion
HildaM

I only have one 4090 graphic card, I wonder if it can convert Yi-9B safetensor model type into gguf?
deleted

i do conversions on a CPU only machine and it does not take long at all.
itsliupeng
 01-ai org

I only have one 4090 graphic card, I wonder if it can convert Yi-9B safetensor model type into gguf?

Yes, you can follow the instruction in llama.cpp, https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#prepare-and-quantize

itsliupeng changed discussion status to closed

Sign up or log in to comment