anongecko

/

gemma-3-12b-it-heretic-ltx

like 6

I tested out Grimjim's ORBA ablit on LTX23 and it worked great for prompt generation. Threw up an FP4 quant over here.

Just thought that might be interesting :3

how many have you tested? now I see 3 fp4 versions on HF (this, yours, and sikaworld)(when I search ltx+gemma)(and gitmylo which is not uncensored), which is the best in your opinion?

In general, Grim knows his stuff. I'm in the AI roleplay scene and his MPOA method sparked a new wave of interest in Gemma 3.

mlabonne's Gemma 3 ablits are famously pretty bad (through no fault of labonne's own!!), they're actually what inspired further iteration on refusal ablation to begin with. The v2 was better, but also made it clear that a change in kind rather than degree was needed.

Heretic was one of those iterations, based partly on Grim's work, but at this point encapsulates a couple different methods, including MPOA.
That said if this Heretic has broken instruction following, I wouldn't expect one based on mlabonne's work to be better. I haven't tested it but that's what my intuition says.

ORBA and MPOA are much gentler methods; I recall Grim actually said he was worried that ORBA would have broken instruction following but it seems to work. I might run it back on another one now that I have a toolkit for them (which I also published; but they were written by Claude to be called as tools, so there's a lot of hardcoded paths. I might have it clean them up into something more usable. It basically just wraps safetensors and comfy-kitchen

The difficult but most correct answer I can give is "try all of them" 😅

In general, Grim knows his stuff. I'm in the AI roleplay scene and his MPOA method sparked a new wave of interest in Gemma 3.

mlabonne's Gemma 3 ablits are famously pretty bad (through no fault of labonne's own!!), they're actually what inspired further iteration on refusal ablation to begin with. The v2 was better, but also made it clear that a change in kind rather than degree was needed.

Heretic was one of those iterations, based partly on Grim's work, but at this point encapsulates a couple different methods, including MPOA.
That said if this Heretic has broken instruction following, I wouldn't expect one based on mlabonne's work to be better. I haven't tested it but that's what my intuition says.

ORBA and MPOA are much gentler methods; I recall Grim actually said he was worried that ORBA would have broken instruction following but it seems to work. I might run it back on another one now that I have a toolkit for them (which I also published; but they were written by Claude to be called as tools, so there's a lot of hardcoded paths. I might have it clean them up into something more usable. It basically just wraps safetensors and comfy-kitchen

The difficult but most correct answer I can give is "try all of them" 😅

thank you for your opinion

I personally haven't tried your suggestion, but have pinned this in case others are interested and have the issue with prompt generation (another user reported not having the issue at all, which is strange. But I get better results writing my own prompts anyway).

Thanks for contributing!