README.md

metadata

license: mit
tags:
  - sparse-autoencoder
  - temporal-crosscoder
  - gemma-2-2b

txcdr-base — Temporal Crosscoder checkpoints (Phase 7, Gemma-2-2b base)

This repo contains SAE / TXC / MLC / TFA checkpoints trained on google/gemma-2-2b (base, NOT IT) as part of Phase 7 of the Temporal Crosscoders project.

All ckpts in this repo were trained at:

• Subject model: google/gemma-2-2b (base).
• Anchor layer: 0-indexed L12 (≈ 50% model depth) — matches T-SAE (Ye et al. 2025) and TFA (Lubana et al. 2025).
• Sparsity: k_win = 500 global TopK on d_sae = 18432 (with one paper-faithful exception, tsae_paper_k20).
• Probing protocol: long-tail sliding mean-pool, S = 128 tokens.

Related repositories

repo	purpose	model regime
han1823123123/txcdr	Phase 5 / 5B ckpts	gemma-2-2b-IT (historical)
han1823123123/txcdr-data	Phase 5 / 5B caches	gemma-2-2b-IT (historical)
han1823123123/txcdr-base (this repo)	Phase 7 ckpts	gemma-2-2b base
han1823123123/txcdr-base-data	Phase 7 activation + probe caches	gemma-2-2b base

DO NOT MIX ckpts from the IT and base repos. The residual stream of gemma-2-2b and gemma-2-2b-it differ; Phase 7 ckpts were trained on base activations and are not compatible with IT.

See the project's docs/han/research_logs/phase7_unification/ directory at https://github.com/chainik1125/temp_xc for the full Phase 7 plan.