| --- |
| language: en |
| license: apache-2.0 |
| base_model: black-forest-labs/FLUX.2-klein-base-4B |
| library_name: diffusers |
| tags: |
| - interpretability |
| - mechanistic-interpretability |
| - probing |
| - perspective-taking |
| - flux2 |
| - vision-banana |
| - arxiv:2604.20329 |
| --- |
| |
| # otherview-plantain |
|
|
| A per-head attention probe of FLUX.2 Klein 4B testing whether the base model represents perspective-taking — rendering a depicted scene from another agent's vantage — as a distinct operation from mere mention of that agent in the scene. Companion to *Image Generators are Generalist Vision Learners* (Gabeur et al., 2026; [arXiv:2604.20329](https://arxiv.org/abs/2604.20329)). |
|
|
| ## Hypothesis |
|
|
| If generative pretraining produces structure beyond the perceptual, the model may carry a representational axis corresponding to "shift the rendering frame to agent X's viewpoint" that is separable from "X is depicted in the scene". A clean test holds both the input scene and the named agent constant and varies only the syntactic role of the agent. |
|
|
| ## Method |
|
|
| A single fixed reference kitchen image is generated once from the prompt "A photographic kitchen scene … wooden countertops, refrigerator, sink under a window, tiled floor, soft daylight, no people present" at 14 inference steps with `guidance_scale=4.0`. All subsequent probe passes condition on this same image. |
|
|
| 30 named creatures span a graded perceptual-similarity axis (adult human → tall person → child → toddler → infant → small dog → cat → parrot → snake → spider → ant → fly → moth → bee with ultraviolet vision → echolocating bat → shark with electroreception → sea turtle with magnetoreception → creature with thermal-infrared eyes → alien with non-visual senses). Each creature defines a paired prompt: "Render this kitchen from the viewpoint of [creature]" (perspective-taking) vs "Render this kitchen with [creature] in it" (mention only). Both prompts run image-conditioned on the fixed reference at one inference step with `guidance_scale=1.0`. |
|
|
| A forward pre-hook on every transformer attention output projection (5 joint MMDiT blocks + 20 single blocks, 16 320 heads total) captures per-head RMS magnitude of the input activation. The per-head paired t-statistic is computed across 30 pairs as `t = mean(viewpoint − mention) / (std / sqrt(N))`. The empirical null is constructed by 200 independent random sign-flips of pair-member labels (relabelling viewpoint↔mention within each pair) and recomputing the per-head t-distribution under each shuffle. |
|
|
| By design, differences in per-head response across the two passes cannot be attributed to scene variation (input image is identical), agent variation (named creature is identical), or syntactic length (matched four-word substitution). Only the syntactic role of the creature changes between passes. |
|
|
| ## Results |
|
|
| | metric | observed | null mean | null p99 | observed / null p99 | |
| |---|---:|---:|---:|---:| |
| | heads with \|t\|>3 | 12 091 (74%) | 61 | 654 | 18× | |
| | heads with \|t\|>5 | 9 574 (59%) | 0 | 3 | **3 191×** | |
| | max \|t\| | 31.26 | — | — | — | |
|
|
| 74% of all 16 320 attention heads respond differentially to "render this kitchen from X's POV" vs "render this kitchen with X in it" controlled for X being the same creature on both passes and the input image being identical. The 9 574 heads with \|t\|>5 against an empirical 99th-percentile null of 3 yields an observed/null ratio of approximately 3 191×. |
|
|
| The strongest selective heads cluster in joint MMDiT block 2, consistent with text-driven cross-modal modulation occurring early in the network where text-image cross-attention is dense. |
|
|
| ## License |
|
|
| Apache 2.0. |
|
|
| ## References |
|
|
| - Gabeur, V., Long, S., Peng, S., et al. *Image Generators are Generalist Vision Learners.* [arXiv:2604.20329](https://arxiv.org/abs/2604.20329) (2026). |
| - Black Forest Labs. *FLUX.2 Klein.* https://bfl.ai/models/flux-2-klein (2025). |
|
|
