| --- |
| license: mit |
| tags: |
| - sparse-autoencoder |
| - temporal-crosscoder |
| - gemma-2-2b |
| --- |
| |
| # txcdr-base — Temporal Crosscoder checkpoints (Phase 7, Gemma-2-2b base) |
|
|
| This repo contains SAE / TXC / MLC / TFA checkpoints **trained on |
| `google/gemma-2-2b` (base, NOT IT)** as part of Phase 7 of the |
| Temporal Crosscoders project. |
|
|
| All ckpts in this repo were trained at: |
|
|
| - Subject model: `google/gemma-2-2b` (base). |
| - Anchor layer: 0-indexed L12 (≈ 50% model depth) — matches T-SAE |
| (Ye et al. 2025) and TFA (Lubana et al. 2025). |
| - Sparsity: `k_win = 500` global TopK on `d_sae = 18432` (with one |
| paper-faithful exception, `tsae_paper_k20`). |
| - Probing protocol: long-tail sliding mean-pool, S = 128 tokens. |
|
|
| ## Related repositories |
|
|
| | repo | purpose | model regime | |
| |---|---|---| |
| | [`han1823123123/txcdr`](https://huggingface.co/han1823123123/txcdr) | Phase 5 / 5B ckpts | **gemma-2-2b-IT** (historical) | |
| | [`han1823123123/txcdr-data`](https://huggingface.co/datasets/han1823123123/txcdr-data) | Phase 5 / 5B caches | **gemma-2-2b-IT** (historical) | |
| | **`han1823123123/txcdr-base`** (this repo) | **Phase 7 ckpts** | **gemma-2-2b base** | |
| | [`han1823123123/txcdr-base-data`](https://huggingface.co/datasets/han1823123123/txcdr-base-data) | Phase 7 activation + probe caches | **gemma-2-2b base** | |
|
|
| > **DO NOT MIX** ckpts from the IT and base repos. The residual |
| > stream of `gemma-2-2b` and `gemma-2-2b-it` differ; Phase 7 ckpts |
| > were trained on base activations and are not compatible with IT. |
|
|
| See the project's `docs/han/research_logs/phase7_unification/` |
| directory at https://github.com/chainik1125/temp_xc for the full |
| Phase 7 plan. |
| |
