metadata
library_name: diffusers
pipeline_tag: image-to-image
license: other
license_name: waymo-non-commercial-and-stability-ai-community
license_link: LICENSE.md
base_model:
- stabilityai/sd-turbo
- madebyollin/taesd
tags:
- r3d2
- autonomous-driving
- image-to-image
- custom-code
- arxiv:2506.07826
R3D2
One-step diffusion for realistic lighting and shadows around inserted 3D assets. This is the tiny VAE version of R3D2.
Paper 路 Code 路 Project 路 R3D2-big
Non-commercial use only. This model is released under the Waymo Dataset License Agreement. All downstream use or modification is subject to its terms, including Section 4. Full agreements and attribution: LICENSE.md, NOTICE.
Use with Diffusers
Tested with PyTorch 2.6.0, Diffusers 0.40.0 and Transformers 5.17.0. A CUDA GPU is required for the example below:
pip install torch==2.6.0 diffusers==0.40.0 transformers==5.17.0 peft
import torch
from diffusers import DiffusionPipeline
from PIL import Image
pipe = DiffusionPipeline.from_pretrained(
"bertaveira/R3D2",
trust_remote_code=True,
torch_dtype=torch.float16,
).to("cuda")
pipe(Image.open("input.png").convert("RGB")).images[0].save("output.png")
Use with the original repository
Inside zenseact/R3D2, after uv sync:
uv run huggingface-cli download bertaveira/R3D2 --local-dir checkpoints/R3D2
uv run r3d2/scripts/eval.py --model-path checkpoints/R3D2 \
--image-path input.png --output-path output.png
The repository CLI uses FP32. For lower GPU memory use, use the FP16 example above.
Credits
Built on SD-Turbo and TAESD, using the Waymo Open Dataset. Powered by Stability AI. For citation, see the paper's repository.