---
license: mit
library_name: diffusers
pipeline_tag: image-to-image
tags:
  - image-inpainting
  - object-removal
  - object-insertion
  - flux
  - lora
  - iccv2025
base_model:
  - black-forest-labs/FLUX.1-dev
---

# OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting

<p align="center">
  <b>ICCV 2025</b>
</p>

<p align="center">
  <a href="https://arxiv.org/pdf/2503.08677"><img src="https://img.shields.io/badge/Paper-arXiv-b31b1b.svg"></a>
  <a href="https://www.yongshengyu.com/OmniPaint-Page/"><img src="https://img.shields.io/badge/Website-Project-6535a0"></a>
  <a href="https://github.com/yeates/OmniPaint"><img src="https://img.shields.io/badge/GitHub-Code-181717?logo=github"></a>
  <a href="https://huggingface.co/datasets/yeates/omnipaint-bench"><img src="https://img.shields.io/badge/HuggingFace-Dataset-FFD21E?logo=huggingface"></a>
</p>

## Model Description

OmniPaint is a unified framework for **object-oriented image editing** that disentangles object insertion and removal into two specialized inpainting tasks. Built on top of [FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev), OmniPaint uses **LoRA adapters** to enable:

- **Object Removal** — Remove foreground objects and their visual effects (shadows, reflections) using only object masks
- **Object Insertion** — Seamlessly insert objects into existing scenes with harmonious lighting and perspective

The model also introduces a novel **CFD (Content-aware Frechet Distance)** metric for reference-free evaluation of object removal quality.

## Model Files

| File | Description |
|------|-------------|
| `weights/omnipaint_remove.safetensors` | LoRA weights for object removal |
| `weights/omnipaint_insert.safetensors` | LoRA weights for object insertion |
| `embeddings/remove.npz` | Pre-computed text embeddings for removal |
| `embeddings/insert.npz` | Pre-computed text embeddings for insertion |

## Usage

### Installation

```bash
git clone https://github.com/yeates/OmniPaint.git
cd OmniPaint
bash scripts/setup.sh
```

This will install dependencies and automatically download model weights from this repository.

### Object Removal

```bash
python scripts/omnipaint_remove.py \
  --input ./demo_assets/removal_samples/images/5.jpg \
  --mask ./demo_assets/removal_samples/masks/5.png \
  --output_dir ./outputs \
  --seed 42 --steps 28 --device cuda:0
```

### Object Insertion

```bash
python scripts/omnipaint_insert.py \
  --background ./demo_assets/insertion_samples/backgrounds/background-2.png \
  --mask ./demo_assets/insertion_samples/masks/mask-2.png \
  --subject ./demo_assets/insertion_samples/subjects/subject-2.png \
  --output_dir ./outputs \
  --seed 42 --steps 28 --device cuda:0
```

### Gradio Demo

```bash
bash scripts/app_setup.sh  # one-time setup (~15 min)
python app.py
```

## Technical Details

- **Base model**: [FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) (FluxTransformer2DModel)
- **Training method**: LoRA fine-tuning with disentangled adapters for insertion and removal
- **Framework**: Diffusers + PEFT
- **Inference steps**: 28 (default)
- **Guidance scale**: 3.5 (default)

## Citation

```bibtex
@inproceedings{yu2025omnipaint,
  title={Omnipaint: Mastering object-oriented editing via disentangled insertion-removal inpainting},
  author={Yu, Yongsheng and Zeng, Ziyun and Zheng, Haitian and Luo, Jiebo},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={17324--17334},
  year={2025}
}
```

## License

This model is released under the [MIT License](https://opensource.org/licenses/MIT).