GGUF Support

#1
by Void2377 - opened

Hey Dude, Great Model, can we have a GGUF version of this ? 😊

Thank You So much !!

Is this one different ?, i mean any optimisations ?

Is most... uncensored... with the ideal prompt. Test and tell me your results.

Sure man, Just some time, i will let you know, thank BTW

it is working superrrr Crazyyyyy

Haha, I thought the tweak I made would make it work better than the original. It must have been the model with the roleplay dataset. Check back in a few hours; I'll be releasing another version soon... though I can't promise it'll be the best. Thanks for the info.

Take this version; it's a smoother, more purist version. In theory, it's free of any weird stuff. Another thing... also use KoboldAI. For some reason, AI doesn't behave the same in all environments; it tends to vary according to hidden internal prompts to "optimize" the models. You might get better results with one interface than another.

https://huggingface.co/Novaciano/Gemma3-Emophilic-1B-GGUF/resolve/main/Gemma3-Emophilic-1B-Q4_K_M.gguf

Hey Dude Thanks Again i saw, it came in my feed + idk but i have made a optimised version of llama.cpp, so most of the optimizations from kb.cpp are done in my backend, still i will have a look into it, btw thanx again ⭐

Sign up or log in to comment