base_model: google/gemma-4-31B-it
library_name: peft
license: apache-2.0
tags:
- activation-oracles
- interpretability
- lora
- self-introspection
- sae
- deprecated
- legacy
Legacy Activation Oracle: gemma-4-31B-it
Deprecated / legacy checkpoint This activation oracle was trained with an older Gemma 4 activation-injection recipe. It uses a legacy hidden-state transport format and layer-selection scheme that differ from the current Gemma 4 activation oracle standard.
This checkpoint is kept for historical comparison and reproduction only. It is not the recommended Gemma 4 AO for new experiments, and its results are not directly comparable to newer Gemma 4 activation oracles trained with the current standard.
This is a legacy LoRA adapter for gemma-4-31B-it. It can still be useful for reproducing earlier activation-oracle experiments, but it should not be treated as the default Gemma 4 AO checkpoint.
Why This Checkpoint Is Legacy
This model was trained before the current Gemma 4 AO injection convention was adopted. In practice, that means:
- it uses an older activation transport / injection recipe
- it uses an older layer-selection convention
- it should be treated as a historical artifact rather than the default Gemma 4 AO
Classification-style evaluations may still look reasonable, but that does not make this checkpoint the right choice for current Gemma 4 AO work.
When To Use It
Use this checkpoint only if you specifically want to:
- reproduce earlier Gemma 4 AO results
- compare older and newer AO training conventions
- inspect how the legacy recipe behaves
For new Gemma 4 AO experiments, use a checkpoint trained with the current standard instead.
Quick Start
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
base_model = AutoModelForCausalLM.from_pretrained(
"google/gemma-4-31B-it",
torch_dtype=torch.bfloat16,
device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("google/gemma-4-31B-it")
model = PeftModel.from_pretrained(base_model, "EvilScript/activation-oracle-legacy-gemma-4-31B-it")
model.eval()
Legacy Training Details
| Parameter | Value |
|---|---|
| Base model | google/gemma-4-31B-it |
| Adapter | LoRA |
| Training tasks | LatentQA, classification, PastLens (next-token), SAE features |
| Checkpoint status | Legacy / deprecated |
| Activation injection | Older Gemma 4 AO recipe |
| Recommended use | Historical comparison and reproduction only |
Related Resources
- Paper: Activation Oracles (arXiv:2512.15674)
- Code: activation_oracles