kiji-inspector-NVIDIA-Nemotron-3-Nano-30B-A3B-FP8

JumpReLU Sparse Autoencoders (SAEs) trained on contrastive activation data for mechanistic interpretability of tool-selection decisions.

Layers

Layer	Described features	Contrast types
layer_8	411	37
layer_17	251	37
layer_20	232	37
layer_26	242	37
layer_35	452	37
layer_44	949	37

Architecture

input x (d_model=2688)
    |
    +-> W_enc @ (x - b_dec) + b_enc -> JumpReLU(-, theta) -> features (d_sae=10752)
    |
    +-> W_dec @ features + b_dec -> reconstruction (d_model=2688)

• Type: JumpReLU SAE with learnable per-feature thresholds
• Parameters: 57,826,944
• dtype: bfloat16
• d_model: 2688
• d_sae: 10752
• bandwidth: 0.001

Config

Key	Value
bandwidth	0.001
d_model	2688
d_sae	10752
dtype	bfloat16

Repo structure

├── layer_<N>/
│   ├── activations/
│   │   ├── contrastive_features.json
│   │   ├── feature_descriptions.json
│   │   ├── shard_*.npy
│   │   └── ...
│   └── sae_checkpoints/
│       ├── sae_final.pt
│       ├── config.json
│       └── ...

Usage

from sae.model import JumpReLUSAE

sae = JumpReLUSAE.from_pretrained("layer_17/sae_checkpoints/sae_final.pt", device="cuda")
features = sae.encode(activations)
reconstruction = sae.decode(features)

Generated by kiji-inspector.