| --- |
| license: apache-2.0 |
| pipeline_tag: text-to-image |
| --- |
| |
| # MVSplit-DiT (1000 layers) |
|
|
| This repository contains the weights for the 1000-layer Diffusion Transformer (DiT) presented in the paper [Mean Mode Screaming: Mean--Variance Split Residuals for 1000-Layer Diffusion Transformers](https://huggingface.co/papers/2605.06169). |
|
|
| [Project Page](https://erwold.github.io/mv-split/) | [GitHub Repository](https://github.com/erwold/mv-split) |
|
|
| ## Introduction |
|
|
| Scaling Diffusion Transformers to extreme depths (hundreds or thousands of layers) introduces a structural vulnerability known as **Mean Mode Screaming (MMS)**. In this state, token representations homogenize, and centered variation is suppressed, leading to model collapse. |
|
|
| MVSplit-DiT addresses this by using **Mean-Variance Split (MV-Split) Residuals**, which combine a separately gained centered residual update with a leaky trunk-mean replacement. This architecture enables the stable training of DiTs at boundary scales, such as the 1000-layer model provided here. |
|
|
| ## Usage |
|
|
| To use this model for image generation, please refer to the official [GitHub repository](https://github.com/erwold/mv-split) for installation instructions and requirements. |
|
|
| ### Sampling |
|
|
| You can generate images using the `sample.py` script. The model requires the DiT checkpoint from this repo, a FLUX.2 VAE, and a Qwen3 text encoder. |
|
|
| ```bash |
| # Custom prompt example |
| python sample.py \ |
| --checkpoint_path /path/to/model.pt \ |
| --flux_vae_path /path/to/flux2_ae.safetensors \ |
| --qwen_model_path Qwen/Qwen3-0.6B \ |
| --prompt "a red panda climbing a bamboo stalk" \ |
| --output_dir ./samples |
| ``` |
|
|
| ### Key sampling flags |
|
|
| | Flag | Default | Meaning | |
| |---|---|---| |
| | `--image_size` | 256 | Square output side in pixels. | |
| | `--num_inference_steps` | 35 | Euler steps for the flow-matching ODE. | |
| | `--cfg_scale` | 2.0 | Classifier-free guidance. | |
| | `--time_shift_alpha` | 4.0 | Time-shift in the flow schedule (must match training). | |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{lu2026mms, |
| title = {Mean Mode Screaming: Mean--Variance Split Residuals for 1000-Layer Diffusion Transformers}, |
| author = {Lu, Pengqi}, |
| journal = {arXiv preprint arXiv:2605.06169}, |
| year = {2026}, |
| } |
| ``` |
