YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
DINO Sparse Autoencoder Checkpoint
This model is a sparse autoencoder trained on DINO's internal representations.
Model Details
Architecture
- Layer: 11
- Layer Type: hook_resid_post
- Model: facebook/dino-vitb16
- Dictionary Size: 49152
- Input Dimension: 768
- Expansion Factor: 64
- CLS Token Only: False
Training
- Learning Rate: 0.00017054432023208037
- L1 Coefficient: -
- Batch Size: 4096
- Context Size: 197
Checkpoint details
- Checkpoint: 2/10
Sparsity
- L0 (Active Features): 1024
Additional Information
- Wandb Run: https://wandb.ai/perceptual-alignment/dino-sae-sweep/runs/5nyvhlgc
- Random Seed: 42
- Downloads last month
- 1
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support