metadata
license: mit
library_name: diffusers
pipeline_tag: image-to-image
tags:
- image-inpainting
- object-removal
- object-insertion
- flux
- lora
- iccv2025
base_model:
- black-forest-labs/FLUX.1-dev
OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting
ICCV 2025
Model Description
OmniPaint is a unified framework for object-oriented image editing that disentangles object insertion and removal into two specialized inpainting tasks. Built on top of FLUX.1-dev, OmniPaint uses LoRA adapters to enable:
- Object Removal — Remove foreground objects and their visual effects (shadows, reflections) using only object masks
- Object Insertion — Seamlessly insert objects into existing scenes with harmonious lighting and perspective
The model also introduces a novel CFD (Content-aware Frechet Distance) metric for reference-free evaluation of object removal quality.
Model Files
| File | Description |
|---|---|
weights/omnipaint_remove.safetensors |
LoRA weights for object removal |
weights/omnipaint_insert.safetensors |
LoRA weights for object insertion |
embeddings/remove.npz |
Pre-computed text embeddings for removal |
embeddings/insert.npz |
Pre-computed text embeddings for insertion |
Usage
Installation
git clone https://github.com/yeates/OmniPaint.git
cd OmniPaint
bash scripts/setup.sh
This will install dependencies and automatically download model weights from this repository.
Object Removal
python scripts/omnipaint_remove.py \
--input ./demo_assets/removal_samples/images/5.jpg \
--mask ./demo_assets/removal_samples/masks/5.png \
--output_dir ./outputs \
--seed 42 --steps 28 --device cuda:0
Object Insertion
python scripts/omnipaint_insert.py \
--background ./demo_assets/insertion_samples/backgrounds/background-2.png \
--mask ./demo_assets/insertion_samples/masks/mask-2.png \
--subject ./demo_assets/insertion_samples/subjects/subject-2.png \
--output_dir ./outputs \
--seed 42 --steps 28 --device cuda:0
Gradio Demo
bash scripts/app_setup.sh # one-time setup (~15 min)
python app.py
Technical Details
- Base model: FLUX.1-dev (FluxTransformer2DModel)
- Training method: LoRA fine-tuning with disentangled adapters for insertion and removal
- Framework: Diffusers + PEFT
- Inference steps: 28 (default)
- Guidance scale: 3.5 (default)
Citation
@inproceedings{yu2025omnipaint,
title={Omnipaint: Mastering object-oriented editing via disentangled insertion-removal inpainting},
author={Yu, Yongsheng and Zeng, Ziyun and Zheng, Haitian and Luo, Jiebo},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
pages={17324--17334},
year={2025}
}
License
This model is released under the MIT License.