Update DiffHDR model card and teaser
Browse files- .gitattributes +1 -0
- README.md +41 -22
- assets/teaser.png +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
assets/teaser.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -18,19 +18,36 @@ tags:
|
|
| 18 |
- vace
|
| 19 |
---
|
| 20 |
|
| 21 |
-
# DiffHDR
|
| 22 |
|
| 23 |
-
|
| 24 |
-
(
|
| 25 |
-
[
|
| 26 |
-
|
| 27 |
-
supports text- and reference-image-guided reconstruction.
|
| 28 |
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
- **Base model:** [Wan-AI/Wan2.1-VACE-14B](https://huggingface.co/Wan-AI/Wan2.1-VACE-14B)
|
| 32 |
|
| 33 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
|
| 35 |
This repository contains LoRA adapters, not a standalone model. Download the
|
| 36 |
base model separately before running inference.
|
|
@@ -40,7 +57,7 @@ base model separately before running inference.
|
|
| 40 |
| `DiffHDR.safetensors` | `infer_video.py`, `infer_image.py`, and `infer_long_video.py` |
|
| 41 |
| `DiffHDR_Pano.safetensors` | `infer_hdri.py` for 360-degree HDR panoramas |
|
| 42 |
|
| 43 |
-
## Setup
|
| 44 |
|
| 45 |
Clone and install the inference code:
|
| 46 |
|
|
@@ -70,7 +87,7 @@ The Wan2.1-VACE-14B download is approximately 75 GB and is not included in this
|
|
| 70 |
repository. Set `MODEL_BASE` if you store the base model somewhere other than
|
| 71 |
`models/`.
|
| 72 |
|
| 73 |
-
## Inference
|
| 74 |
|
| 75 |
Minimal video example:
|
| 76 |
|
|
@@ -117,7 +134,7 @@ one EXR file per frame; panorama inference writes `predicted.exr`. See the
|
|
| 117 |
text conditioning, reference-image conditioning, arguments, and additional
|
| 118 |
examples.
|
| 119 |
|
| 120 |
-
## Intended use
|
| 121 |
|
| 122 |
DiffHDR is intended for research and creative LDR-to-HDR reconstruction,
|
| 123 |
including:
|
|
@@ -128,7 +145,7 @@ including:
|
|
| 128 |
- text- or reference-image-guided HDR reconstruction; and
|
| 129 |
- reconstructing HDR environment panoramas.
|
| 130 |
|
| 131 |
-
## Limitations and responsible use
|
| 132 |
|
| 133 |
- Detail in clipped or quantized regions is generated by the model. It is a
|
| 134 |
plausible reconstruction and is not guaranteed to reproduce the original
|
|
@@ -145,15 +162,17 @@ including:
|
|
| 145 |
- Users are responsible for ensuring that their input media and intended use
|
| 146 |
comply with applicable rights, licenses, and laws.
|
| 147 |
|
| 148 |
-
## Training and evaluation
|
| 149 |
|
| 150 |
-
DiffHDR is trained as a LoRA adapter on top of Wan2.1-VACE-14B
|
| 151 |
-
|
| 152 |
-
|
| 153 |
-
|
|
|
|
|
|
|
| 154 |
[paper](https://arxiv.org/abs/2604.06161).
|
| 155 |
|
| 156 |
-
## License
|
| 157 |
|
| 158 |
The DiffHDR LoRA adapter weights in this repository are released under the
|
| 159 |
[Apache License 2.0](LICENSE).
|
|
@@ -162,7 +181,7 @@ The Wan2.1-VACE-14B base weights are distributed separately and remain subject
|
|
| 162 |
to their own license and model-card guidance. This license does not grant rights
|
| 163 |
to third-party code, datasets, or user-supplied input content.
|
| 164 |
|
| 165 |
-
## Citation
|
| 166 |
|
| 167 |
```bibtex
|
| 168 |
@article{yu2026diffhdr,
|
|
@@ -173,7 +192,7 @@ to third-party code, datasets, or user-supplied input content.
|
|
| 173 |
}
|
| 174 |
```
|
| 175 |
|
| 176 |
-
## Acknowledgements
|
| 177 |
|
| 178 |
DiffHDR builds on
|
| 179 |
[Wan2.1-VACE-14B](https://huggingface.co/Wan-AI/Wan2.1-VACE-14B) and
|
|
|
|
| 18 |
- vace
|
| 19 |
---
|
| 20 |
|
| 21 |
+
# DiffHDR: Re-Exposing LDR Videos with Video Diffusion Models (ECCV 2026)
|
| 22 |
|
| 23 |
+
[](https://eyeline-labs.github.io/DiffHDR/)
|
| 24 |
+
[](https://arxiv.org/abs/2604.06161)
|
| 25 |
+
[](https://github.com/Eyeline-Labs/DiffHDR)
|
| 26 |
+
[](https://youtu.be/kq8qZfwBRs0)
|
|
|
|
| 27 |
|
| 28 |
+
[Zhengming Yu](https://yzmblog.github.io/)<sup>1,2</sup>, [Li Ma](https://limacv.github.io/homepage/)<sup>2</sup>, [Mingming He](https://mingminghe.com/)<sup>2</sup>, [Leo Isikdogan](https://isikdogan.com/)<sup>3</sup>, [Yuancheng Xu](https://yuancheng-xu.github.io/)<sup>2,3</sup>, [Dmitriy Smirnov](https://dsmirnov.com/)<sup>3</sup>, [Pablo Salamanca](https://pablosalaman.ca/)<sup>2,3</sup>, [Dao Mi](#)<sup>3</sup>, [Pablo Delgado](#)<sup>3</sup>, [Ning Yu](https://ningyu1991.github.io/)<sup>2,3</sup>, [Julien Philip](https://julienphilip.com/)<sup>2</sup>, [Xin Li](https://people.tamu.edu/~xinli/)<sup>1</sup>, [Wenping Wang](https://engineering.tamu.edu/cse/profiles/Wang-Wenping.html)<sup>1</sup>, [Paul Debevec](https://www.debevec.org/)<sup>2,3</sup><br/>
|
| 29 |
+
<sup>1</sup>Texas A&M University, <sup>2</sup>Eyeline Labs, <sup>3</sup>Netflix
|
|
|
|
| 30 |
|
| 31 |
+
<p align="center">
|
| 32 |
+
<img src="./assets/teaser.png" alt="DiffHDR teaser figure">
|
| 33 |
+
</p>
|
| 34 |
+
|
| 35 |
+
## π Abstract
|
| 36 |
+
|
| 37 |
+
> Most digital videos are stored in 8-bit low dynamic range (LDR) formats, where much of the original high dynamic range (HDR) scene radiance is lost due to saturation and quantization. This loss of highlight and shadow detail precludes mapping accurate luminance to HDR displays and limits meaningful re-exposure in post-production workflows. Although techniques have been proposed to convert LDR images to HDR through dynamic range expansion, they struggle to restore realistic detail in the over- and underexposed regions. To address this, we present **DiffHDR**, a framework that formulates LDR-to-HDR conversion as a generative radiance inpainting task within the latent space of a video diffusion model. By operating in Log-Gamma color space, DiffHDR leverages spatio-temporal generative priors from a pretrained video diffusion model to synthesize plausible HDR radiance in over- and underexposed regions while recovering the continuous scene radiance of the quantized pixels. Our framework further enables controllable LDR-to-HDR video conversion guided by text prompts or reference images. To address the scarcity of paired HDR video data, we develop a pipeline that synthesizes high-quality HDR video training data from static HDRI maps. Extensive experiments demonstrate that DiffHDR significantly outperforms state-of-the-art approaches in radiance fidelity and temporal stability, producing realistic HDR videos with considerable latitude for re-exposure.
|
| 38 |
+
|
| 39 |
+
## π¬ Model overview
|
| 40 |
+
|
| 41 |
+
DiffHDR uses a proposed **Log-Gamma Color Mapping** to compress linear HDR
|
| 42 |
+
radiance into a bounded representation aligned with natural LDR statistics.
|
| 43 |
+
This allows the pretrained LDR video VAE to encode and decode HDR content
|
| 44 |
+
without VAE finetuning or shifting its learned latent space. A LoRA-adapted
|
| 45 |
+
[Wan2.1-VACE-14B](https://huggingface.co/Wan-AI/Wan2.1-VACE-14B) backbone then
|
| 46 |
+
performs generative radiance inpainting conditioned on the input LDR content
|
| 47 |
+
and exposure masks, with optional text or reference-image control. The decoded
|
| 48 |
+
result is inverse Log-Gamma mapped to linear HDR radiance.
|
| 49 |
+
|
| 50 |
+
## π¦ Model files
|
| 51 |
|
| 52 |
This repository contains LoRA adapters, not a standalone model. Download the
|
| 53 |
base model separately before running inference.
|
|
|
|
| 57 |
| `DiffHDR.safetensors` | `infer_video.py`, `infer_image.py`, and `infer_long_video.py` |
|
| 58 |
| `DiffHDR_Pano.safetensors` | `infer_hdri.py` for 360-degree HDR panoramas |
|
| 59 |
|
| 60 |
+
## π οΈ Setup
|
| 61 |
|
| 62 |
Clone and install the inference code:
|
| 63 |
|
|
|
|
| 87 |
repository. Set `MODEL_BASE` if you store the base model somewhere other than
|
| 88 |
`models/`.
|
| 89 |
|
| 90 |
+
## π₯ Inference
|
| 91 |
|
| 92 |
Minimal video example:
|
| 93 |
|
|
|
|
| 134 |
text conditioning, reference-image conditioning, arguments, and additional
|
| 135 |
examples.
|
| 136 |
|
| 137 |
+
## π― Intended use
|
| 138 |
|
| 139 |
DiffHDR is intended for research and creative LDR-to-HDR reconstruction,
|
| 140 |
including:
|
|
|
|
| 145 |
- text- or reference-image-guided HDR reconstruction; and
|
| 146 |
- reconstructing HDR environment panoramas.
|
| 147 |
|
| 148 |
+
## β οΈ Limitations and responsible use
|
| 149 |
|
| 150 |
- Detail in clipped or quantized regions is generated by the model. It is a
|
| 151 |
plausible reconstruction and is not guaranteed to reproduce the original
|
|
|
|
| 162 |
- Users are responsible for ensuring that their input media and intended use
|
| 163 |
comply with applicable rights, licenses, and laws.
|
| 164 |
|
| 165 |
+
## ποΈ Training and evaluation
|
| 166 |
|
| 167 |
+
DiffHDR is trained as a LoRA adapter on top of Wan2.1-VACE-14B while keeping
|
| 168 |
+
the video VAE frozen. Its synthetic training set is rendered from approximately
|
| 169 |
+
800 high-resolution HDRIs and contains about 5,400 temporally consistent
|
| 170 |
+
81-frame HDR sequences. Exposure shifts, temporally correlated camera noise,
|
| 171 |
+
clipping, and 8-bit quantization are simulated to produce paired LDR inputs.
|
| 172 |
+
Method details, experimental settings, and comparisons are reported in the
|
| 173 |
[paper](https://arxiv.org/abs/2604.06161).
|
| 174 |
|
| 175 |
+
## π License
|
| 176 |
|
| 177 |
The DiffHDR LoRA adapter weights in this repository are released under the
|
| 178 |
[Apache License 2.0](LICENSE).
|
|
|
|
| 181 |
to their own license and model-card guidance. This license does not grant rights
|
| 182 |
to third-party code, datasets, or user-supplied input content.
|
| 183 |
|
| 184 |
+
## π Citation
|
| 185 |
|
| 186 |
```bibtex
|
| 187 |
@article{yu2026diffhdr,
|
|
|
|
| 192 |
}
|
| 193 |
```
|
| 194 |
|
| 195 |
+
## π Acknowledgements
|
| 196 |
|
| 197 |
DiffHDR builds on
|
| 198 |
[Wan2.1-VACE-14B](https://huggingface.co/Wan-AI/Wan2.1-VACE-14B) and
|
assets/teaser.png
ADDED
|
Git LFS Details
|