---
license: mit
base_model:
- stabilityai/stable-diffusion-2
---
# StableMotion: Repurposing Diffusion-Based Image Priors for Motion Estimation
This is the official repo for paper [StableMotion: Repurposing Diffusion-Based Image Priors for Motion Estimation](https://www.arxiv.org/abs/2505.06668)

## Setup
0. Clone the [code repo](https://github.com/ziyiwhat/StableMotion).
1. Create your environment from `requirements.txt`.
2. Download [DIR-D](https://drive.google.com/file/d/1KR5DtekPJin3bmQPlTGP4wbM1zFR80ak/view?usp=sharing) and [RS-Real](https://huggingface.co/datasets/Yzl-code/RS-Diffusion/tree/main). Put them into `StableMotion_SIR` and `StableMotion_RSC` respectively.

## StableMotion for Stitched Image Rectangling (SIR)
### Inference
0. Download the checkpoints of `StableMotion_SIR`
1. Run `cd StableMotion_SIR && sh sample.sh`. You may want to change this file to modify the inference configurations.
2. Run `sh metrics.sh` to evaluate the results.

### Training
0. Replicate [RecDiffusion](https://github.com/lhaippp/RecDiffusion) to get pseudo labels, i.e., the flow labels generated by [RecDiffusion](https://github.com/lhaippp/RecDiffusion) on the training set of [DIR-D](https://drive.google.com/file/d/1KR5DtekPJin3bmQPlTGP4wbM1zFR80ak/view?usp=sharing). Put and rename it into `StableMotion_SIR/MDM_Flow`.
1. Run `cd StableMotion_SIR && sh train.sh`. You may want to change this file to modify the training configurations. The default configuration requires approximately 80 GB of VRAM per card.

## StableMotion for Rolling Shutter Correction (RSC)
### Inference
0. Download the checkpoints of `StableMotion_RSC`
1. Run `cd StableMotion_RSC && sh sample.sh`. You may want to change this file to modify the inference configurations.
2. Run `sh metrics.sh` to evaluate the results.

### Training
Run `cd StableMotion_RSC && sh train.sh`. You may want to change this file to modify the training configurations. The default configuration requires approximately 40 GB of VRAM per card.