README.md

---
license: mit
tags:
  - sparse-autoencoder
  - temporal-crosscoder
  - gemma-2-2b
---

# txcdr-base — Temporal Crosscoder checkpoints (Phase 7, Gemma-2-2b base)

This repo contains SAE / TXC / MLC / TFA checkpoints **trained on
`google/gemma-2-2b` (base, NOT IT)** as part of Phase 7 of the
Temporal Crosscoders project.

All ckpts in this repo were trained at:

- Subject model: `google/gemma-2-2b` (base).
- Anchor layer: 0-indexed L12 (≈ 50% model depth) — matches T-SAE
  (Ye et al. 2025) and TFA (Lubana et al. 2025).
- Sparsity: `k_win = 500` global TopK on `d_sae = 18432` (with one
  paper-faithful exception, `tsae_paper_k20`).
- Probing protocol: long-tail sliding mean-pool, S = 128 tokens.

## Related repositories

| repo | purpose | model regime |
|---|---|---|
| [`han1823123123/txcdr`](https://huggingface.co/han1823123123/txcdr) | Phase 5 / 5B ckpts | **gemma-2-2b-IT** (historical) |
| [`han1823123123/txcdr-data`](https://huggingface.co/datasets/han1823123123/txcdr-data) | Phase 5 / 5B caches | **gemma-2-2b-IT** (historical) |
| **`han1823123123/txcdr-base`** (this repo) | **Phase 7 ckpts** | **gemma-2-2b base** |
| [`han1823123123/txcdr-base-data`](https://huggingface.co/datasets/han1823123123/txcdr-base-data) | Phase 7 activation + probe caches | **gemma-2-2b base** |

> **DO NOT MIX** ckpts from the IT and base repos. The residual
> stream of `gemma-2-2b` and `gemma-2-2b-it` differ; Phase 7 ckpts
> were trained on base activations and are not compatible with IT.

See the project's `docs/han/research_logs/phase7_unification/`
directory at https://github.com/chainik1125/temp_xc for the full
Phase 7 plan.