README.md

metadata

license: cc-by-nc-4.0
tags:
  - relighting
  - inverse-rendering
  - pbr
  - image-to-image
  - pytorch
pipeline_tag: image-to-image
library_name: pytorch

PIXLRelight

PIXLRelight is a feed-forward model for physically controllable single-image relighting. Given a source photograph and a target lighting condition — specified either as an RGB image or as a path-traced Blender Cycles render — it produces a relit version of the source under the new illumination in under a tenth of a second.

This repository hosts the trained model checkpoint and matching configuration. Full code, training details, and additional results are on the project page.

Abstract

We present PIXLRelight, a feed-forward approach for physically controllable single-image relighting. Existing methods either provide limited lighting control (e.g. through text or environment maps), accumulate errors when chaining inverse and forward rendering, or require costly per-image optimization. Our key idea is to bridge physically based rendering (PBR) and learned image synthesis through a shared intrinsic conditioning that can be obtained from either real photographs or PBR renders. At training time, paired multi-illumination photographs are decomposed into albedo, diffuse shading, and non-diffuse residuals, which condition the model. At inference time, the same conditioning is computed from a path-traced render of a coarse 3D reconstruction of the input under user-specified PBR lights. A transformer-based neural renderer then applies the target illumination to the source photograph, preserving fine image detail through a per-pixel affine modulation. PIXLRelight enables arbitrary PBR-style lighting control, achieves state-of-the-art relighting quality, and runs in under a tenth of a second per image.

Files

• model.safetensors — model weights
• config.yaml — OmegaConf config defining the model architecture (net, head, rgbx submodules)

Quick start

Command line

After cloning the GitHub repository and installing its requirements, place each sample in its own directory:

samples/
├── room00/
│   ├── source.jpg
│   └── target.jpg                  # target lighting as an RGB image
├── room01/
│   ├── source.jpg
│   └── target.jpg                  # target lighting as an RGB image
├── room02/
│   ├── source.jpg
│   ├── diffuse_color.exr           # ...or as Blender Cycles passes
│   ├── diffuse_direct.exr
│   ├── diffuse_indirect.exr
│   ├── glossy_color.exr
│   ├── glossy_direct.exr
│   ├── glossy_indirect.exr
│   ├── transmission_color.exr
│   ├── transmission_direct.exr
│   ├── transmission_indirect.exr
│   ├── volume_direct.exr
│   ├── volume_indirect.exr
│   ├── environment.exr
│   └── emission.exr
├── room03/
│   └── ...
└── ...

Then run:

python infer.py --input_dir samples/ --output_dir outputs/

The model and config are downloaded from this repository on first run and cached locally.

Python

from huggingface_hub import hf_hub_download
from omegaconf import OmegaConf
from safetensors.torch import load_file

from src.utils.cfg import create_object

REPO_ID = "mlfarinha/pixlrelight"

config_path = hf_hub_download(repo_id=REPO_ID, filename="config.yaml")
ckpt_path = hf_hub_download(repo_id=REPO_ID, filename="model.safetensors")

cfg = OmegaConf.load(config_path)
model = create_object(cfg.model)
model.load_state_dict(load_file(ckpt_path), strict=False)
model.cuda().eval()

# Target lighting as an RGB image (decomposed on the fly).
relit = model.inference(
    source_images="source.jpg",
    target_images="target.jpg",
    output_type="pil",
)
relit[0].save("relit.png")

# Or as a directory of Blender Cycles render passes.
relit = model.inference(
    source_images="source.jpg",
    cycles="path/to/cycles_passes/",
    output_type="pil",
)
relit[0].save("relit.png")

Limitations

PIXLRelight is trained on indoor scenes and relights at the source's native resolution by applying a per-pixel affine modulation. The model does not synthesise content not present in the source — strong cast shadows or specular highlights that change the visible surface geometry may not transfer faithfully.

Citation

If you use PIXLRelight in your work, please cite:

@article{farinha2026pixlrelight,
  title     = {PIXLRelight: Controllable Relighting via Intrinsic Conditioning},
  author    = {Farinha, Miguel and Clark, Ronald},
  journal   = {arXiv preprint arXiv:2605.18735},
  year      = {2026}
}

License

This model is released under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license. The model weights and configuration may be used for research and personal purposes with attribution, but not for commercial purposes.