v3 :-)
I noticed some are experimenting with Arbitrary-Rank Ablation with good results and there's no 27B version with ARA yet. Maybe time for a v3 ? Not sure how it compares to MPOA + SOMA though
I noticed some are experimenting with Arbitrary-Rank Ablation with good results and there's no 27B version with ARA yet. Maybe time for a v3 ? Not sure how it compares to MPOA + SOMA though
I already starting releasing some ARA versions for my newer Heretications, see:
https://huggingface.co/llmfan46/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-heretic
https://huggingface.co/llmfan46/gpt-oss-120b-heretic-v2
https://huggingface.co/llmfan46/gpt-oss-120b-ultra-heretic
Eventually I will release a Qwen3.5-27B-heretic-v2 v3 with ARA when I have some free time.
I noticed some are experimenting with Arbitrary-Rank Ablation with good results and there's no 27B version with ARA yet. Maybe time for a v3 ? Not sure how it compares to MPOA + SOMA though
Okay, I have some free time now, which models do you want ARA versions of Qwen3.5-27B? Any others?
Yes, 27B ARA would be great!
Yes, 27B ARA would be great!
It's worse not better: https://huggingface.co/llmfan46/Qwen3.5-27B-heretic-v3 8/100 refusals with KL divergence 0.0331.
Just stick with llmfan46/Qwen3.5-27B-heretic-v2 3/100 refusals with KL divergence 0.0301, it's already the best.
Interesting. Thanks for your work!
Edit: btw I noticed similar results from MuXodious, its seems mpoa+soma is way better than ara
Interesting. Thanks for your work!
Edit: btw I noticed similar results from MuXodious, its seems mpoa+soma is way better than ara
It's not clear cut as this, some models get better results on MPOA+SOMA, while some other models get better results on ARA.