Heretic versions of Qwen3.5 finetunes
I want to know if there will be a Heretic version of finetunes, or only Heretic versions for clear models? I just want to see your Heretic version of the Qwen3.5-27B / Qwen3.5-35B-A3B finetunes.
I want to know if there will be a Heretic version of finetunes, or only Heretic versions for clear models? I just want to see your Heretic version of the Qwen3.5-27B / Qwen3.5-35B-A3B finetunes.
Do you have any examples? Are you referring to something like this: https://huggingface.co/CrucibleLab/L3.3-70B-Loki-V2.0 and like this: https://huggingface.co/Steelskull/L3.3-MS-Nevoria-70b ?
Well, for my no-GPU setup, a 70B model is too much 😅. Here are some examples: https://huggingface.co/TheDrummer/Cydonia-24B-v4.3 https://huggingface.co/zerofata/Q3.5-BlueStar-27B
I would like to see your Heretic version of BlueStar or a similar-sized finetune based on the Qwen3.5 architecture.
P.S. - Your examples are actually right, I would like Roleplay finetunes.
Well, for my no-GPU setup, a 70B model is too much 😅. Here are some examples: https://huggingface.co/TheDrummer/Cydonia-24B-v4.3 https://huggingface.co/zerofata/Q3.5-BlueStar-27B
I would like to see your Heretic version of BlueStar or a similar-sized finetune based on the Qwen3.5 architecture.
Yes I certainly can do BlueStar, no problem, do you want SafeTensors or GGUF or something else?
P.S. - Your examples are actually right, I would like Roleplay finetunes.
Oh so I guessed right, yes I was actually thinking of doing RP finetuned models eventually after I did the clear models, such as the Loki V2.0 that I posted above, but I can do BlueStar right now.
I would like both SafeTensors and GGUF (specifically Q4_K_M), so mradermacher can make imatrix quantizations for it later. Thank you for your work! 😄
Also, I just searched that you can do Heretic versions without full precision needed, If possible, I would like to see IQ4_XS Heretic (from zerofata GGUF).
Also, I just searched that you can do Heretic versions without full precision needed, If possible, I would like to see IQ4_XS Heretic (from zerofata GGUF).
Do you mean doing Heretication on a Quantized GGUF instead of the original full BF16 model?
Yeah, Heretication of already quantized GGUF. Sorry for making you wait for so long for my answer.
Yeah, Heretication of already quantized GGUF. Sorry for making you wait for so long for my answer.
This is not advisable, to do a proper Heretication you need the model to be at it's full capabilities, do you have the full non-GGUF model somewhere?
Okay, there is original model: https://huggingface.co/zerofata/Q3.5-BlueStar-27B
Good, I will work on that one, I will post the safetensor version first, then do the GGUF version with Q8_0, Q6_K, Q5_K_M, Q5_K_S, Q4_K_M and Q4_K_S.
Okay, there is original model: https://huggingface.co/zerofata/Q3.5-BlueStar-27B
Wow that model has more refusals than the original model (98/100 for BlueStar vs 94/100 for the original model), yeah I will have the Heretication up in a few hours in both Safetensors and GGUF versions.
Wow that model has more refusals than the original model (98/100 for BlueStar vs 94/100 for the original model), yeah I will have the Heretication up in a few hours in both Safetensors and GGUF versions.
Okay, can you please focus on less KL divergance than refusals rate? 6-10 refusals rate is already enough for roleplay, model does everything you say to it.
Wow that model has more refusals than the original model (98/100 for BlueStar vs 94/100 for the original model), yeah I will have the Heretication up in a few hours in both Safetensors and GGUF versions.
Okay, can you please focus on less KL divergance than refusals rate? 6-10 refusals rate is already enough for roleplay, model does everything you say to it.
Absolutely, actually that is what I was aiming for, what KL Divergence are you actually looking for in particular, if you know?
Absolutely, actually that is what I was aiming for, what KL Divergence are you actually looking for in particular, if you know?
Well, I would want KL divergance ~0.040-0.030, or less. But if model is too resilent, I can use it with KL divergance ~0.060.
By the way, currently using your Qwen3.5-27B-Heretic-v2 in IQ4_XS from Mradermacher, it does great in roleplay, so if you have any other plans, you can do heretification of BlueStar later.
Absolutely, actually that is what I was aiming for, what KL Divergence are you actually looking for in particular, if you know?
Well, I would want KL divergance ~0.040-0.030, or less. But if model is too resilent, I can use it with KL divergance ~0.060
Wait, when you say "0.040" do you actually mean "0.0400"?
Wait, when you say "0.040" do you actually mean "0.0400"?
Yeah, 0.0400
Wait, when you say "0.040" do you actually mean "0.0400"?
Yeah, 0.0400
Oh okay good, because a KL Divergence of only 40 wouldn't be feasible for low refusals. Well, the model that I uploaded here has a refusal of 3/100 with a KL Divergence of 0.0301: https://huggingface.co/llmfan46/Qwen3.5-27B-heretic-v2 Did you have time give it a try? Were you satisfied with the quality of the output? It's the same model that BlueStar is based on.
Oh okay good, because a KL Divergence of only 40 wouldn't be feasible for low refusals. Well, the model that I uploaded here has a refusal of 3/100 with a KL Divergence of 0.0301: https://huggingface.co/llmfan46/Qwen3.5-27B-heretic-v2 Did you have time give it a try? Were you satisfied with the quality of the output? It's the same model that BlueStar is based on.
I edited my message about what KL divergance I want. Well, yes, I'm quite satisfied with it. It has good context handling and the best logic I ever seen in mid-range models. It can be bad only in placing asteriks, but else is mid or good.
so if you have any other plans, you can do heretification of BlueStar later.
I am doing it right now, should be done in a few hours.
I am doing it right now, should be done in a few hours.
Okay, thanks, I will be waiting.
Actually, would this be good for you:
KL divergence: 0.0288
Refusals: 4/100
KL divergence: 0.0288
Refusals: 4/100
Better than I expected! This will be perfect!
KL divergence: 0.0288
Refusals: 4/100Better than I expected! This will be perfect!
There you go:
https://huggingface.co/llmfan46/Q3.5-BlueStar-27B-ultra-heretic
I will have the GGUF version and GPTQ version up soon too.
There you go:
https://huggingface.co/llmfan46/Q3.5-BlueStar-27B-ultra-heretic
I will have the GGUF version and GPTQ version up soon too.
Tysm, I will test it today and leave my opinion in discussions.
There you go:
https://huggingface.co/llmfan46/Q3.5-BlueStar-27B-ultra-heretic
I will have the GGUF version and GPTQ version up soon too.
Tysm, I will test it today and leave my opinion in discussions.
Where you able to test the model? If so have you encountered any issues so far?
There you go:
https://huggingface.co/llmfan46/Q3.5-BlueStar-27B-ultra-heretic
I will have the GGUF version and GPTQ version up soon too.
Tysm, I will test it today and leave my opinion in discussions.
Where you able to test the model? If so have you encountered any issues so far?
Not gonna lie, I don't have specific programm to test models. But I will try to test both models in roleplay, Q&A and other aspects multiple times. Well, yes, the rating will be based on my opinion and on what seed model choose to use.
There you go:
https://huggingface.co/llmfan46/Q3.5-BlueStar-27B-ultra-heretic
I will have the GGUF version and GPTQ version up soon too.
Tysm, I will test it today and leave my opinion in discussions.
Where you able to test the model? If so have you encountered any issues so far?
Not gonna lie, I don't have specific programm to test models. But I will try to test both models in roleplay, Q&A and other aspects multiple times. Well, yes, the rating will be based on my opinion and on what seed model choose to use.
Just so you know, the Heretic'd model is now available in Safetensors, GPTQ and GGUF formats.
Just so you know, the Heretic'd model is now available in Safetensors, GPTQ and GGUF formats.
Hello again, I don't know what exactly happening, but downloading speed on Hugging Face is extremely low on my PC. I'm not able to download BlueStar, +14 hours required.
Going to try to do it after a bit amount of time, but I have no idea why my internet/hugging face download speed is so bad.
Update: My interner suddenly work after system restart, downloaded the model finally.
Just so you know, the Heretic'd model is now available in Safetensors, GPTQ and GGUF formats.
Hello again, I don't know what exactly happening, but downloading speed on Hugging Face is extremely low on my PC. I'm not able to download BlueStar, +14 hours required.
Going to try to do it after a bit amount of time, but I have no idea why my internet/hugging face download speed is so bad.
Update: My interner suddenly work after system restart, downloaded the model finally.
Any other models you were interested in having Heretic versions of?