DarkArtsForge/huihui-ai_Huihui-GLM-4.5-Air-abliterated-Q3_K_M-GGUF

saved from huihui's repo before the Q3_K_M was deleted

https://huggingface.co/huihui-ai/Huihui-GLM-4.5-Air-abliterated-GGUF/tree/main/Q3_K_M-GGUF

useful for 64gb ram

this is the setup I use with 8gb vram

koboldcpp.exe --host 0.0.0.0 --port 5001 --model Q3_K_M-GGUF-00001-of-00006.gguf --flashattention --contextsize 57344 --gpulayers 22 --moecpu

or

koboldcpp.exe --host 0.0.0.0 --port 5001 --model Q3_K_M-GGUF-00001-of-00006.gguf --flashattention --contextsize 16384 --gpulayers 48 --moecpu

this model is really good but i recommend mlabonne's abliterated gemma 3 27B for cases when GLM struggles with prompt adherence