README.md

---
license: cc-by-nc-4.0
tags:
  - relighting
  - inverse-rendering
  - pbr
  - image-to-image
  - pytorch
pipeline_tag: image-to-image
library_name: pytorch
---

# PIXLRelight

PIXLRelight is a feed-forward model for physically controllable single-image relighting. Given a source photograph and a target lighting condition — specified either as an RGB image or as a path-traced Blender Cycles render — it produces a relit version of the source under the new illumination in under a tenth of a second.

This repository hosts the trained model checkpoint and matching configuration. Full code, training details, and additional results are on the [project page](https://mlfarinha.github.io/pixl-relight/).

## Abstract

We present PIXLRelight, a feed-forward approach for physically controllable single-image relighting. Existing methods either provide limited lighting control (e.g. through text or environment maps), accumulate errors when chaining inverse and forward rendering, or require costly per-image optimization. Our key idea is to bridge physically based rendering (PBR) and learned image synthesis through a shared intrinsic conditioning that can be obtained from either real photographs or PBR renders. At training time, paired multi-illumination photographs are decomposed into albedo, diffuse shading, and non-diffuse residuals, which condition the model. At inference time, the same conditioning is computed from a path-traced render of a coarse 3D reconstruction of the input under user-specified PBR lights. A transformer-based neural renderer then applies the target illumination to the source photograph, preserving fine image detail through a per-pixel affine modulation. PIXLRelight enables arbitrary PBR-style lighting control, achieves state-of-the-art relighting quality, and runs in under a tenth of a second per image.

## Files

- `model.safetensors` — model weights
- `config.yaml` — OmegaConf config defining the model architecture (`net`, `head`, `rgbx` submodules)

## Quick start

### Command line

After cloning the [GitHub repository](https://github.com/mlfarinha/pixlrelight) and installing its requirements, place each sample in its own directory:

    samples/
    ├── room00/
    │   ├── source.jpg
    │   └── target.jpg                  # target lighting as an RGB image
    ├── room01/
    │   ├── source.jpg
    │   └── target.jpg                  # target lighting as an RGB image
    ├── room02/
    │   ├── source.jpg
    │   ├── diffuse_color.exr           # ...or as Blender Cycles passes
    │   ├── diffuse_direct.exr
    │   ├── diffuse_indirect.exr
    │   ├── glossy_color.exr
    │   ├── glossy_direct.exr
    │   ├── glossy_indirect.exr
    │   ├── transmission_color.exr
    │   ├── transmission_direct.exr
    │   ├── transmission_indirect.exr
    │   ├── volume_direct.exr
    │   ├── volume_indirect.exr
    │   ├── environment.exr
    │   └── emission.exr
    ├── room03/
    │   └── ...
    └── ...

Then run:

    python infer.py --input_dir samples/ --output_dir outputs/

The model and config are downloaded from this repository on first run and cached locally.

### Python

```python
from huggingface_hub import hf_hub_download
from omegaconf import OmegaConf
from safetensors.torch import load_file

from src.utils.cfg import create_object

REPO_ID = "mlfarinha/pixlrelight"

config_path = hf_hub_download(repo_id=REPO_ID, filename="config.yaml")
ckpt_path = hf_hub_download(repo_id=REPO_ID, filename="model.safetensors")

cfg = OmegaConf.load(config_path)
model = create_object(cfg.model)
model.load_state_dict(load_file(ckpt_path), strict=False)
model.cuda().eval()

# Target lighting as an RGB image (decomposed on the fly).
relit = model.inference(
    source_images="source.jpg",
    target_images="target.jpg",
    output_type="pil",
)
relit[0].save("relit.png")

# Or as a directory of Blender Cycles render passes.
relit = model.inference(
    source_images="source.jpg",
    cycles="path/to/cycles_passes/",
    output_type="pil",
)
relit[0].save("relit.png")
```

## Limitations

PIXLRelight is trained on indoor scenes and relights at the source's native resolution by applying a per-pixel affine modulation. The model does not synthesise content not present in the source — strong cast shadows or specular highlights that change the visible surface geometry may not transfer faithfully.

## Citation

If you use PIXLRelight in your work, please cite:

```bibtex
@article{farinha2026pixlrelight,
  title     = {PIXLRelight: Controllable Relighting via Intrinsic Conditioning},
  author    = {Farinha, Miguel and Clark, Ronald},
  journal   = {arXiv preprint arXiv:2605.18735},
  year      = {2026}
}
```

## License

This model is released under the [Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)](https://creativecommons.org/licenses/by-nc/4.0/) license. The model weights and configuration may be used for research and personal purposes with attribution, but not for commercial purposes.