| --- |
| license: apache-2.0 |
| pipeline_tag: image-to-image |
| tags: |
| - relighting |
| - multi-view |
| - lighting-control |
| - latent-bridge-matching |
| - sdxl |
| datasets: |
| - davidserra9/synclight |
| --- |
| |
| # SyncLight |
|
|
| Pretrained weights of **[SyncLight: Single-Edit Multi-View Relighting](https://arxiv.org/abs/2601.16981)** (NeurIPS 2026). |
|
|
| [Paper](https://arxiv.org/abs/2601.16981) · [Code](https://github.com/CVC-Color/synclight) · [Project page](https://color.cvc.uab.cat/synclight) · [Dataset](https://huggingface.co/datasets/davidserra9/synclight) |
|
|
| SyncLight relights a set of multi-view images consistently from a single light edit. It takes a reference image, any number of additional views, and a lightmap describing the edit in the reference view (which lights to switch on or off, and their target intensity and colour). It then generates every view under the new lighting in a single step. The model is a multi-view diffusion model built on SDXL and trained with Latent Bridge Matching, with MVDream-style 3D attention across views. |
|
|
| ## Files |
|
|
| | File | Content | |
| |---|---| |
| | `synclight_sdxl.ckpt` | model weights: denoiser and VAE, 2.5 B parameters, bfloat16, 5.05 GB | |
| | `config.yaml` | model configuration read by the loader | |
|
|
| The checkpoint contains only the model weights; the training state is not included. It loads with `torch.load(..., weights_only=True)`. |
|
|
| ## Usage |
|
|
| Install the code from [GitHub](https://github.com/CVC-Color/synclight): |
|
|
| ```bash |
| git clone https://github.com/CVC-Color/synclight |
| cd synclight |
| pip install -e . |
| ``` |
|
|
| The weights are downloaded from this repository automatically: |
|
|
| ```bash |
| python test.py \ |
| --image_paths input_images/example_1_0_ref.png input_images/example_1_1.png input_images/example_1_2.png \ |
| --lightmap_path input_images/example_1_0_lightmap.png \ |
| --output_dir outputs/ |
| ``` |
|
|
| ```python |
| from PIL import Image |
| from src.synclight.inference import evaluate, get_model |
| from src.synclight.lightmap_io import load_lightmap |
| |
| model = get_model("davidserra9/synclight", device="cuda") |
| images = [Image.open(p).convert("RGB") for p in image_paths] # reference view first |
| lightmap = load_lightmap("lightmap.png") # (H, W, 4) float32 |
| outputs = evaluate(model, images, lightmap_image=lightmap, num_sampling_steps=1) |
| ``` |
|
|
| To keep a local copy, run `hf download davidserra9/synclight --local-dir ckpt` and pass `--model_weights ckpt/`. |
|
|
| The lightmap is a `(H, W, 4)` map of the reference view: activation (−1 turn off, 0 no change, 1 set), target intensity, and target colour as CIELAB a/b. See the [code repository](https://github.com/CVC-Color/synclight#lightmap-format) for the exact format and an interactive lightmap editor. |
|
|
| ## Training |
|
|
| - **Data:** the [SyncLight dataset](https://huggingface.co/datasets/davidserra9/synclight), which combines procedurally generated Infinigen rooms, artist-made BlenderKit scenes and real multi-view captures. All are relit by recombining images captured or rendered one light at a time. |
| - **Model:** the SDXL backbone is fine-tuned as a two-view denoiser, and the VAE is kept frozen. |
| - **Checkpoint:** taken at step 65,250. |
| - **Views:** although trained on image pairs, the model generalises zero-shot to any number of views. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @inproceedings{serrano2026synclight, |
| title = {SyncLight: Single-Edit Multi-View Relighting}, |
| author = {Serrano-Lozano, David and Bhattad, Anand and Herranz, Luis and Lalonde, Jean-Fran{\c{c}}ois and Vazquez-Corral, Javier}, |
| booktitle = {Advances in Neural Information Processing Systems (NeurIPS)}, |
| year = {2026} |
| } |
| ``` |
|
|
