README.md

---
license: apache-2.0
tags:
  - text-to-video
  - image-to-video
  - camera-control
  - world-model
  - diffusion
---

# SANA-WM (Bidirectional)

**SANA-WM** is an efficient open-source world model trained natively for
one-minute generation. The bidirectional checkpoint released here is a
2.6B-parameter image-to-video diffusion transformer that synthesises
720p, minute-scale videos with precise 6-DoF camera control, paired with
the LTX-2 sink-bidirectional Euler refiner for high-fidelity decoding.

Four core designs drive the architecture:

1. **Hybrid Linear Attention** — frame-wise Gated DeltaNet combined with
   softmax attention every Nth block for memory-efficient long-context
   modelling.
2. **Dual-Branch Camera Control** — independent main and camera branches
   enable precise per-frame trajectory adherence.
3. **Two-Stage Generation Pipeline** — a long-video refiner stitched on
   top of Stage-1 latents improves quality and temporal consistency.
4. **Robust Annotation Pipeline** — metric-scale 6-DoF camera poses
   extracted from public video corpora yield spatiotemporally consistent
   action supervision.

Paper: <https://arxiv.org/abs/2605.15178>

```bibtex
@article{zhu2026sanawm,
  title   = {{SANA-WM}: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer},
  author  = {Zhu, Haoyi and Liu, Haozhe and Zhao, Yuyang and Ye, Tian and Chen, Junsong and Yu, Jincheng and He, Tong and Han, Song and Xie, Enze},
  journal = {arXiv preprint arXiv:2605.15178},
  year    = {2026},
}
```

## Repository layout

| Component                          | Path in repo                              | Size  |
|------------------------------------|-------------------------------------------|------:|
| Sana DiT (Stage 1)                 | `dit/sana_wm_1600m_720p.safetensors`      | 10 GB |
| LTX-2 VAE (diffusers)              | `vae/`                                    |  2 GB |
| LTX-2 refiner (Stage 2)            | `refiner/refiner.safetensors`             | 41 GB |
| Gemma text encoder for the refiner | `refiner/text_encoder/`                   | 46 GB |
| Inference config                   | `config.yaml`                             |     — |

The Sana text encoder (`gemma-2-2b-it`) is **not** bundled here — it is
fetched on demand from the public Hugging Face mirror.

## Usage

```bash
python inference_video_scripts/inference_sana_wm.py \
  --image      asset/sana_wm/demo_0.png \
  --prompt     asset/sana_wm/demo_0.txt \
  --action     "w-80,jw-40,w-40,lw-60,w-100" \
  --translation_speed 0.055 \
  --rotation_speed_deg 1.2 \
  --num_frames 321 \
  --output_dir results/demo
```

Weights are fetched from this repository on first use. Pass `--no_refiner`
to skip the LTX-2 refiner and decode Stage-1 latents with the Sana VAE
instead. To run fully offline, override any of `--config` / `--model_path` /
`--refiner_checkpoint` / `--refiner_gemma_root` with local paths.

## Inputs

| Argument            | Format                                                                                  |
|---------------------|-----------------------------------------------------------------------------------------|
| `--image`           | RGB image (any PIL-readable format) — used as the first frame.                          |
| `--prompt`          | UTF-8 text file containing the conditioning prompt.                                     |
| `--camera`          | NumPy `.npy` of shape `(F, 4, 4)` — per-frame camera-to-world matrices.                  |
| `--action`          | WASD/IJKL DSL, e.g. `"w-80,jw-40,w-40,lw-60,w-100"`. We roll it out to a `(F+1, 4, 4)` trajectory. Mutually exclusive with `--camera`. |
| `--intrinsics`      | Optional. `.npy` of shape `(3, 3)`, `(F, 3, 3)`, or `(4,)`. If omitted, we estimate intrinsics from `--image` with Pi3X and abort if the resulting FOV is outside `[25°, 120°]`. |

The output frame size is fixed at `704 x 1280`; input images are
aspect-preserving resized + center-cropped to that resolution.

## License

Released under the Apache 2.0 license. The bundled LTX-2 refiner and VAE
inherit the LTX-2 upstream license.