Model differences?
Is this functionally any different from https://huggingface.co/coder3101/gemma-4-31B-it-heretic ? The methodology looks the same, "made using Heretic v1.2.0 with the Arbitrary-Rank Ablation (ARA) method", but you report 10 refusals and they show 15? Is this random variance, or is there an actual difference?
I ask because their version appears, based on the UGI, to be a very good model, and with yours showing fewer refusals still it would 'seem' a strict upgrade, but if it's effectively the same model, it'd save downloading it twice.
Is this functionally any different from https://huggingface.co/coder3101/gemma-4-31B-it-heretic ? The methodology looks the same, "made using Heretic v1.2.0 with the Arbitrary-Rank Ablation (ARA) method", but you report 10 refusals and they show 15? Is this random variance, or is there an actual difference?
I ask because their version appears, based on the UGI, to be a very good model, and with yours showing fewer refusals still it would 'seem' a strict upgrade, but if it's effectively the same model, it'd save downloading it twice.
It's not the same model, my version offer 5 less refusals, meaning it should be more uncensored, however I do not know how it compares to coder3101 UGI rating since his was tested on UGI and mine wasn't, I made a request for it to be tested on UGI here though:
https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard/discussions/641
Also if you would scroll down I provide MMLU test scores for both the original base model and the Heretic model.