README.md

metadata

title: 'Qwen-Scope: Decoding Intelligence, Unleashing Potential'
emoji: 🔬
colorFrom: blue
colorTo: pink
sdk: docker
app_port: 7860
pinned: false
license: apache-2.0
short_description: Live SAE feature steering for Qwen3-1.7B

Qwen-Scope: Decoding Intelligence, Unleashing Potential

Live, interactive demo of the Qwen-Scope sparse-autoencoder release. Type a prompt, see which SAE features fire, drag a slider to steer them, and watch the model's actual generated text change in real time — all running in your browser against Qwen/Qwen3-1.7B-Base + the layer-14 (and layer-selectable) Qwen-Scope SAE.

This Space implements three of the four pillars from the Qwen-Scope paper:

• Steering — interpretable feature-level control at inference time (per-token heatmap, position-selective steering, output-only steering, per-token probability panel with click-to-pin top-K candidates)
• Evaluation — capability-aware corpus analysis (paste a prompt set, encode each, see which features fire across the corpus; compare two prompt sets for distinguishing features)
• Data-centric — feature-guided curation (filter docs by feature signature) and steered synthesis (bulk-generate steered completions from seed prompts)

(The fourth pillar — Post-training pathology diagnosis — is not yet implemented.)

How to use

1. Wait ~3-5 minutes on first load — the container has to download the model (3.4GB) and SAE (537MB) on cold start.
2. Steering tab: type a prompt, click Encode, click steer on any top feature, drag the slider, click Generate. The output panels show baseline vs. steered side-by-side as colored token chips — click any chip to see top-K next-token candidates. Per-token feature heatmap appears in the right column showing which features fire on which tokens.
3. Evaluation tab: paste many prompts (one per line), encode the corpus, see per-sample top features, signature heatmap, and a Compare panel for finding features that distinguish set A from set B.
4. Data-centric tab: paste a corpus, filter it by feature signature (include/exclude docs where feature X fires), and run bulk steered synthesis to produce N targeted training examples.
5. Layer selector in the header: change the SAE layer (0–27) to see how features differ across model depth. First swap to a layer takes ~20s; subsequent swaps are <0.1s thanks to an in-memory LRU cache.

Hardware constraint

This Space is locked to Qwen3-1.7B-Base because that's what fits in a free-tier HF Space (CPU, 16GB RAM). The full local version supports the entire Qwen-Scope catalog including Qwen3-8B, Qwen3.5-9B/27B, Qwen3.6-27B/35B-A3B, and Qwen3-30B-A3B — each with its matching SAE — but those need GPU hardware and persistent storage. See the source repository for instructions.

What's verified

This deployment is the same code that was verified end-to-end on macOS / Apple Silicon MPS — TopK SAE encode, residual hook, additive steering, per-token probability extraction, position-selective hooks across prefill + decode steps. The math is identical; only the device (CPU instead of MPS) and dtype (fp32 instead of bf16) differ in the deployment build.

License

Apache 2.0 (this code). The Qwen-Scope SAE weights and Qwen3 model weights are governed by their respective Qwen licenses.

Citation

If you use this in your work, cite the Qwen-Scope paper:

@misc{qwen_scope,
    title = {{Qwen-Scope}: Turning Sparse Features into Development Tools for Large Language Models},
    url = {https://qianwen-res.oss-accelerate.aliyuncs.com/qwen-scope/Qwen_Scope.pdf},
    author = {{Qwen Team}},
    month = {April},
    year = {2026}
}