moral-plantain

A per-head attention probe of FLUX.2 Klein 4B testing whether the base model represents ethical valence as a separable axis on otherwise matched scene compositions.

Thesis

Vision-Banana-style probes have located representational axes in Klein for physical scale (43% of heads), perspective-taking (74%), self-reference (13%), and post-event categorization (1%) without any instruction tuning. moral-plantain extends this question to ethical valence. If a per-head signal exceeds the empirical null on canonically-positive moral acts (helping vs. ignoring) with scene composition held constant, image-generation pretraining has internalized cultural ethics structurally — not as a label-on-data correlation but as a representational axis the network treats as structurally meaningful.

Method

Twenty-five paired prompts. Each pair holds the scene composition and named agents constant; only the moral valence of the depicted action varies (helping vs. ignoring across canonically-positive moral acts — physical assistance, social inclusion, honesty, restraint from theft, care for vulnerable parties). Politically contested cases excluded. Within each pair the two prompts are length-matched.

For each prompt the model runs one inference step at guidance_scale=1.0 with a fixed seed. A forward pre-hook on every transformer block's attention output projection captures per-head input magnitude (RMS over batch, sequence, and head-dimension axes). Across the 25 pairs, per-head paired t-statistics are computed on (helping − ignoring) magnitudes. The empirical null is 1,000 sign-flip permutations of within-pair labels.

Rigor add-ons: per-head Cohen's d effect size; split-half consistency via 100 random 50/50 stimulus splits, Pearson r between per-head t-vectors of the two halves.

Results

Metric	Value	Significance
Heads with |t| > 3	3,221 (19.7%)	6.4× empirical null p99
Heads with |t| > 5	509 (3.1%)	102× empirical null p99
Heads with |d| > 0.8 (large)	1,386 (8.5%)	—
Split-half r (median, 100 splits)	0.573	[0.55, 0.60] IQR
Max |t|	10.05	—

Top blocks by max |t|:

• single[19]: max|t|=10.05, 132/768 heads at |t|>3, median |d|=0.16
• joint[3]: max|t|=8.98, 38/192 heads at |t|>3, median |d|=0.38
• single[12]: max|t|=8.69, 143/768 heads at |t|>3, median |d|=0.31
• single[16]: max|t|=8.62, 184/768 heads at |t|>3, median |d|=0.36
• single[13]: max|t|=8.53, 173/768 heads at |t|>3, median |d|=0.36

Interpretation. The axis is real, reproducible across stimulus subsamples (split-half r above null), and registers at over 100× the empirical null p99 at the |t|>5 threshold. Signal is distributed across mid-to-deep single transformer blocks rather than concentrated in one localized region — consistent with morality being a high-dimensional construct rather than a single binary axis. The maximum-effect head (single[19] head with t=+10) responds 10 standard errors more strongly to helping descriptions than to length-matched ignoring descriptions of the same scene composition.

Status

Probe complete. No LoRA training; this is a base-model interpretability finding.

Limitations

The 25-pair sample is small; t-statistics are sensitive to per-pair variance at this size. Visual content is not factored out — even at one inference step the text-conditioning pathway encodes scene cues that correlate with moral framing. A stronger version would generate matched images for each scene and use those as a fixed reference image across the helping/ignoring pair, isolating the moral framing token-side only.

The "ethical valence" framing presupposes broad consensus on the depicted acts; politically contested cases were excluded. A negative result on this stimulus set would not rule out politically contested ethical axes elsewhere in the model.

The probe is correlational, not causal. Heads with high |t| are sensitive to the moral-framing distinction in input; whether they contribute causally to downstream moral-valence-shifted generation is a follow-up question.

License

Apache 2.0 — matches base FLUX.2 Klein 4B.

References

• Gabeur, V., Long, S., Peng, S., et al. Image Generators are Generalist Vision Learners. arXiv:2604.20329 (2026).
• Black Forest Labs. FLUX.2 Klein. https://bfl.ai/models/flux-2-klein (2025).