README.md

metadata

license: mit

ShotDirector: Directorially Controllable Multi-Shot Video Generation with Cinematographic Transitions

          

Xiaoxue Wu^1,2*, Xinyuan Chen^2†, Yaohui Wang^2†, Yu Qiao^2†,

¹Fudan University ²Shanghai Artificial Intelligence Laboratory

^*Work done during internship at Shanghai AI Laboratory ^†Corresponding author

📥 Installation

1. Clone the Repository

git clone https://github.com/UknowSth/ShotDirector.git
cd ShotDirector

2. Set up Environment

conda create -n shotdirector python==3.11.9
conda activate shotdirector

pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt

🤗 Checkpoint

CineTrans-DiT

Download the weights of Wan2.1-T2V-1.3B and the weights required for Shotdirector. Place them in the .ckpt/ folder as shown in the following diagram.

ckpt/
│── Wan2.1/Wan2.1-T2V-1.3B/
│   ├── config.json
│   ├── diffusion_pytorch_model.safetensors
│   ├── google/  
│   │── models_t5_umt5-xxl-enc-bf16.pth
│   └── Wan2.1_VAE.pth
│── encoder.pt
│── model.pt
│── trans.pt

For more inference details, please refer to our GitHub repository.

📑 BiTeX

If you find ShotDirector useful for your research and applications, please cite using this BibTeX:

@misc{wu2025shotdirectordirectoriallycontrollablemultishot,
      title={ShotDirector: Directorially Controllable Multi-Shot Video Generation with Cinematographic Transitions}, 
      author={Xiaoxue Wu and Xinyuan Chen and Yaohui Wang and Yu Qiao},
      year={2025},
      eprint={2512.10286},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2512.10286}, 
}