blascotobasco/Gemma-4-64E-A4B-Heretic-GGUF

Gemma 4 26B A4B pruned down to 64 experts from 128. This model was made as an experiment to test if a severe prune can be repaired with a custom fine tuning method. As is, the model is great at following instructions and roleplaying but it is horrible at math to the point it gets basic math wrong. However, the model was restored from mind fog to coherence through full parameter tuning but there may be unexpected errors since the prune was quite aggressive.

Recommended Settings:

Sampler: Top-P 0.9 / Temp 0.7 - 0.8

Repeat Penalty: 1.15 (Crucial for preventing loops)

System Prompt: Works best with a strong persona. The model relies on its roleplay training to maintain coherence.

Vision: Pair with the SigLIP mmproj for multimodal capabilities.