README.md

---
title: "Qwen-Scope: Decoding Intelligence, Unleashing Potential"
emoji: 🔬
colorFrom: blue
colorTo: pink
sdk: docker
app_port: 7860
pinned: false
license: apache-2.0
short_description: Live SAE feature steering for Qwen3-1.7B
---

# Qwen-Scope: Decoding Intelligence, Unleashing Potential

Live, interactive demo of the [Qwen-Scope](https://huggingface.co/Qwen/SAE-Res-Qwen3-1.7B-Base-W32K-L0_50) sparse-autoencoder release. Type a prompt, see which SAE features fire, drag a slider to steer them, and watch the model's actual generated text change in real time — all running in your browser against `Qwen/Qwen3-1.7B-Base` + the layer-14 (and layer-selectable) Qwen-Scope SAE.

This Space implements **three of the four pillars** from the Qwen-Scope paper:

- **Steering** — interpretable feature-level control at inference time (per-token heatmap, position-selective steering, output-only steering, per-token probability panel with click-to-pin top-K candidates)
- **Evaluation** — capability-aware corpus analysis (paste a prompt set, encode each, see which features fire across the corpus; compare two prompt sets for distinguishing features)
- **Data-centric** — feature-guided curation (filter docs by feature signature) and steered synthesis (bulk-generate steered completions from seed prompts)

(The fourth pillar — Post-training pathology diagnosis — is not yet implemented.)

## How to use

1. **Wait ~3-5 minutes** on first load — the container has to download the model (~3.4GB) and SAE (~537MB) on cold start.
2. **Steering tab**: type a prompt, click *Encode*, click *steer* on any top feature, drag the slider, click *Generate*. The output panels show baseline vs. steered side-by-side as colored token chips — click any chip to see top-K next-token candidates. Per-token feature heatmap appears in the right column showing which features fire on which tokens.
3. **Evaluation tab**: paste many prompts (one per line), encode the corpus, see per-sample top features, signature heatmap, and a Compare panel for finding features that distinguish set A from set B.
4. **Data-centric tab**: paste a corpus, filter it by feature signature (include/exclude docs where feature X fires), and run bulk steered synthesis to produce N targeted training examples.
5. **Layer selector** in the header: change the SAE layer (0–27) to see how features differ across model depth. First swap to a layer takes ~20s; subsequent swaps are <0.1s thanks to an in-memory LRU cache.

## Hardware constraint

This Space is **locked to Qwen3-1.7B-Base** because that's what fits in a free-tier HF Space (CPU, 16GB RAM). The full local version supports the entire Qwen-Scope catalog including Qwen3-8B, Qwen3.5-9B/27B, Qwen3.6-27B/35B-A3B, and Qwen3-30B-A3B — each with its matching SAE — but those need GPU hardware and persistent storage. See the source repository for instructions.

## What's verified

This deployment is the same code that was [verified end-to-end on macOS / Apple Silicon MPS](https://huggingface.co/Qwen/SAE-Res-Qwen3-1.7B-Base-W32K-L0_50) — TopK SAE encode, residual hook, additive steering, per-token probability extraction, position-selective hooks across prefill + decode steps. The math is identical; only the device (CPU instead of MPS) and dtype (fp32 instead of bf16) differ in the deployment build.

## License

Apache 2.0 (this code). The Qwen-Scope SAE weights and Qwen3 model weights are governed by their respective Qwen licenses.

## Citation

If you use this in your work, cite the Qwen-Scope paper:

```bibtex
@misc{qwen_scope,
    title = {{Qwen-Scope}: Turning Sparse Features into Development Tools for Large Language Models},
    url = {https://qianwen-res.oss-accelerate.aliyuncs.com/qwen-scope/Qwen_Scope.pdf},
    author = {{Qwen Team}},
    month = {April},
    year = {2026}
}
```