Huihui-gemma-4-E2B-it-abliterated-MNN

Pre-converted huihui-ai/Huihui-gemma-4-E2B-it-abliterated in MNN format for TokForge on-device inference.

Requires TokForge 3.4.9 or later.

These Gemma 4 bundles depend on the updated TokForge 3.4.9 runtime and patched libMNN.so with Gemma 4 attention-scale support. Older TokForge builds such as 3.4.7 and pre-patched 3.4.9 builds are not compatible.

What This Repo Contains

This repo ships the working TokForge bundle layout:

File	Purpose
llm.mnn	MNN graph
llm.mnn.weight	Quantized weights
llm.mnn.json	Graph metadata
llm_config.json	Runtime config and chat template
config.json	TokForge backend defaults
embeddings_int4.bin	Token embeddings
per_layer_embeddings_int4.bin	Per-layer embeddings
tokenizer.txt / tokenizer.mtok	Tokenizer assets

Validation

Validated in TokForge 3.4.9 benchmark and chat runs:

• Coherent at 128 tokens on RedMagic 24GB.
• Clean on-device bundle re-push confirmed this variant works with the canonical TokForge layout.
• RedMagic 128-token decode observed at roughly 23 tok/s.

CPU is the recommended backend for Gemma 4 in TokForge. For E2B, keep MNN precision at normal or high; low precision can produce degenerate prompt-parrot output.

Usage

1. Update TokForge to 3.4.9 or later.
2. Add model repo darkmaniac7/Huihui-gemma-4-E2B-it-abliterated-MNN.
3. Use the CPU backend.
4. Set MNN precision to normal or high.
5. Chat normally. Avoid low precision on E2B.

Attribution

• Base model: huihui-ai/Huihui-gemma-4-E2B-it-abliterated
• MNN conversion and TokForge packaging: darkmaniac7
• Runtime target: TokForge + MNN mobile inference