File size: 1,772 Bytes
fb401d4 cca9232 fb401d4 7a34036 fb401d4 cca9232 fb401d4 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 | ---
library_name: saelens
---
SAE panels and SAEBench results from the paper "[Are Sparse Autoencoder Benchmarks Reliable?](https://arxiv.org/abs/2605.18229)"
This repo is split into 2 panels, a cross-architecture panel consisting of 4 SAEs (K=50 Matryoska, k=100 Matryoshka, k=50 BatchTopK, k=100 BatchTopK),
and a Matryoshka panel consisting of 4 Matryoshka SAEs verying the number of Matryoshka prefixes from 1 to 4 (n-1, n-2, n-3, n-4). Each SAE in the Matryoshka panel
is trained 3 times with different seeds (so 12 SAEs total). The cross-architecture panel is trained for 1.5B tokens, while the Matryoshka panel is trained for 300M tokens.
Within each SAE dir, there are a number of snapshots of the SAE taken throughout training. Each of these snapshot dirs include the following:
- SAE weights (`sae_weights.safetensors`) and `cfg.json` for loading with SAELens
- SAEBench raw result JSON files for all SAEBench metrics
To load an SAE snapshot using SAELens, run the following:
```python
from sae_lens import SAE
sae = SAE.from_pretrained("decoderesearch/sae-snapshot-panels", "path/to/snapshot")
```
For instance, to load the SAE snapshot for the K=100 BatchTopK SAE after 500M tokens of training, you would run:
```python
sae = SAE.from_pretrained(
"decoderesearch/sae-snapshot-panels",
"cross-arch-panel/gemma-2-2b/batchtopk/k-100/seed-0/snapshots/step-122070-tokens-500000000",
)
```
## Citation
If you use these SAEs in your work, please cite the following:
```bibtex
@misc{chanin2026saebenchmarks,
title={Are Sparse Autoencoder Benchmarks Reliable?},
author={David Chanin},
year={2026},
eprint={2605.18229},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2605.18229},
}
``` |