IQ4_KSS?
Hello, could you make IQ4_KSS version of this model? Would appreciate it a lot.
Hello, could you make IQ4_KSS version of this model? Would appreciate it a lot.
Would that work?
https://huggingface.co/llmfan46/Q3.5-BlueStar-v2-27B-ultra-uncensored-heretic-v2-GGUF
Hello, could you make IQ4_KSS version of this model? Would appreciate it a lot.
Would that work?
https://huggingface.co/llmfan46/Q3.5-BlueStar-v2-27B-ultra-uncensored-heretic-v2-GGUF
Yeah, but there is only no-imatrix quantizations. Quality is slightly worse, but I will check it. Just want you to try to create IQ4_KSS quantization like you did before.
Yeah, but there is only no-imatrix quantizations. Quality is slightly worse, but I will check it.
Imatrix quants as it turns out are not so very good, they cause "behavioral shift" due to the dataset that is required by Imatrix quantization and a good/big dataset would take 20-21 hours to compile on this model for ik_llama.cpp and this compilation needs to be done specifically for EVERY model (you can not use this dataset compiled on Q3.5-BlueStar-v2-27B for say Gemma 3 27B It, it needs to be compiled specifically for every model, which would mean 40+ hours of compiling on ik_llama.cpp if you want to create Imatrix quants of Q3.5-BlueStar-v2-27B + Gemma 3 27B It), I could use a much smaller/much lower quality dataset that will take much less time to compile, but the quality of the model would probably be decreased even more with even bigger behavioral shift introduced in the model due to it.
Just want you to try to create IQ4_KSS quantization like you did before.
It can be done, the issue is that this is a specialized format that basically nobody else uses, 95% of people uses GGUF and the ~5% left uses Safetensors, GPTQ and AWQ, IQK this is a very obscure format and it takes quite a bit of time to make and necessitate quite a bit of computing power to make, basically I would just be making all of this just for you.
Since this is a very custom request, consider supporting me on Patreon: https://www.patreon.com/c/llmfan46/membership or on Ko-Fi: https://ko-fi.com/llmfan46 and I'll definitely do it for you.