GGUF versions
#3
by maroo87 - opened
I love the idea but please can you provide a gguf versions to run on low-vram hardware?
I doubt GGUF conversion will work since it needs custom code to both convert and run it. FP8 should be doable, however.
I doubt GGUF conversion will work since it needs custom code to both convert and run it. FP8 should be doable, however.
smthem has a gguf but its too big to run on my pc, more like an fp8 if you want to call it that