ZhengmingYu commited on
Commit
2d48846
Β·
verified Β·
1 Parent(s): 0f673c6

Update DiffHDR model card and teaser

Browse files
Files changed (3) hide show
  1. .gitattributes +1 -0
  2. README.md +41 -22
  3. assets/teaser.png +3 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ assets/teaser.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -18,19 +18,36 @@ tags:
18
  - vace
19
  ---
20
 
21
- # DiffHDR
22
 
23
- DiffHDR reconstructs high-dynamic-range (HDR) radiance from low-dynamic-range
24
- (LDR) videos and images using LoRA-finetuned
25
- [Wan2.1-VACE-14B](https://huggingface.co/Wan-AI/Wan2.1-VACE-14B). It formulates
26
- LDR-to-HDR conversion as generative radiance inpainting in Log-Gamma space and
27
- supports text- and reference-image-guided reconstruction.
28
 
29
- - **Paper:** [DiffHDR: Re-Exposing LDR Videos with Video Diffusion Models](https://arxiv.org/abs/2604.06161)
30
- - **Code and full inference instructions:** [Eyeline-Labs/DiffHDR](https://github.com/Eyeline-Labs/DiffHDR)
31
- - **Base model:** [Wan-AI/Wan2.1-VACE-14B](https://huggingface.co/Wan-AI/Wan2.1-VACE-14B)
32
 
33
- ## Model files
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
34
 
35
  This repository contains LoRA adapters, not a standalone model. Download the
36
  base model separately before running inference.
@@ -40,7 +57,7 @@ base model separately before running inference.
40
  | `DiffHDR.safetensors` | `infer_video.py`, `infer_image.py`, and `infer_long_video.py` |
41
  | `DiffHDR_Pano.safetensors` | `infer_hdri.py` for 360-degree HDR panoramas |
42
 
43
- ## Setup
44
 
45
  Clone and install the inference code:
46
 
@@ -70,7 +87,7 @@ The Wan2.1-VACE-14B download is approximately 75 GB and is not included in this
70
  repository. Set `MODEL_BASE` if you store the base model somewhere other than
71
  `models/`.
72
 
73
- ## Inference
74
 
75
  Minimal video example:
76
 
@@ -117,7 +134,7 @@ one EXR file per frame; panorama inference writes `predicted.exr`. See the
117
  text conditioning, reference-image conditioning, arguments, and additional
118
  examples.
119
 
120
- ## Intended use
121
 
122
  DiffHDR is intended for research and creative LDR-to-HDR reconstruction,
123
  including:
@@ -128,7 +145,7 @@ including:
128
  - text- or reference-image-guided HDR reconstruction; and
129
  - reconstructing HDR environment panoramas.
130
 
131
- ## Limitations and responsible use
132
 
133
  - Detail in clipped or quantized regions is generated by the model. It is a
134
  plausible reconstruction and is not guaranteed to reproduce the original
@@ -145,15 +162,17 @@ including:
145
  - Users are responsible for ensuring that their input media and intended use
146
  comply with applicable rights, licenses, and laws.
147
 
148
- ## Training and evaluation
149
 
150
- DiffHDR is trained as a LoRA adapter on top of Wan2.1-VACE-14B. The training
151
- approach uses synthetic HDR video data derived from static HDR environment maps
152
- to address the scarcity of paired HDR video data. Method details, experimental
153
- settings, and comparisons are reported in the
 
 
154
  [paper](https://arxiv.org/abs/2604.06161).
155
 
156
- ## License
157
 
158
  The DiffHDR LoRA adapter weights in this repository are released under the
159
  [Apache License 2.0](LICENSE).
@@ -162,7 +181,7 @@ The Wan2.1-VACE-14B base weights are distributed separately and remain subject
162
  to their own license and model-card guidance. This license does not grant rights
163
  to third-party code, datasets, or user-supplied input content.
164
 
165
- ## Citation
166
 
167
  ```bibtex
168
  @article{yu2026diffhdr,
@@ -173,7 +192,7 @@ to third-party code, datasets, or user-supplied input content.
173
  }
174
  ```
175
 
176
- ## Acknowledgements
177
 
178
  DiffHDR builds on
179
  [Wan2.1-VACE-14B](https://huggingface.co/Wan-AI/Wan2.1-VACE-14B) and
 
18
  - vace
19
  ---
20
 
21
+ # DiffHDR: Re-Exposing LDR Videos with Video Diffusion Models (ECCV 2026)
22
 
23
+ [![Project Page](https://img.shields.io/badge/Project-Page-yellow?logo=googlechrome&logoColor=yellow)](https://eyeline-labs.github.io/DiffHDR/)
24
+ [![Paper](https://img.shields.io/badge/Paper-arXiv-b31b1b?logo=arxiv&logoColor=red)](https://arxiv.org/abs/2604.06161)
25
+ [![Code](https://img.shields.io/badge/Code-GitHub-black?logo=github)](https://github.com/Eyeline-Labs/DiffHDR)
26
+ [![Demo Video](https://img.shields.io/badge/Demo-Video-red?logo=youtube&logoColor=white)](https://youtu.be/kq8qZfwBRs0)
 
27
 
28
+ [Zhengming Yu](https://yzmblog.github.io/)<sup>1,2</sup>, [Li Ma](https://limacv.github.io/homepage/)<sup>2</sup>, [Mingming He](https://mingminghe.com/)<sup>2</sup>, [Leo Isikdogan](https://isikdogan.com/)<sup>3</sup>, [Yuancheng Xu](https://yuancheng-xu.github.io/)<sup>2,3</sup>, [Dmitriy Smirnov](https://dsmirnov.com/)<sup>3</sup>, [Pablo Salamanca](https://pablosalaman.ca/)<sup>2,3</sup>, [Dao Mi](#)<sup>3</sup>, [Pablo Delgado](#)<sup>3</sup>, [Ning Yu](https://ningyu1991.github.io/)<sup>2,3</sup>, [Julien Philip](https://julienphilip.com/)<sup>2</sup>, [Xin Li](https://people.tamu.edu/~xinli/)<sup>1</sup>, [Wenping Wang](https://engineering.tamu.edu/cse/profiles/Wang-Wenping.html)<sup>1</sup>, [Paul Debevec](https://www.debevec.org/)<sup>2,3</sup><br/>
29
+ <sup>1</sup>Texas A&amp;M University, <sup>2</sup>Eyeline Labs, <sup>3</sup>Netflix
 
30
 
31
+ <p align="center">
32
+ <img src="./assets/teaser.png" alt="DiffHDR teaser figure">
33
+ </p>
34
+
35
+ ## πŸ“ Abstract
36
+
37
+ > Most digital videos are stored in 8-bit low dynamic range (LDR) formats, where much of the original high dynamic range (HDR) scene radiance is lost due to saturation and quantization. This loss of highlight and shadow detail precludes mapping accurate luminance to HDR displays and limits meaningful re-exposure in post-production workflows. Although techniques have been proposed to convert LDR images to HDR through dynamic range expansion, they struggle to restore realistic detail in the over- and underexposed regions. To address this, we present **DiffHDR**, a framework that formulates LDR-to-HDR conversion as a generative radiance inpainting task within the latent space of a video diffusion model. By operating in Log-Gamma color space, DiffHDR leverages spatio-temporal generative priors from a pretrained video diffusion model to synthesize plausible HDR radiance in over- and underexposed regions while recovering the continuous scene radiance of the quantized pixels. Our framework further enables controllable LDR-to-HDR video conversion guided by text prompts or reference images. To address the scarcity of paired HDR video data, we develop a pipeline that synthesizes high-quality HDR video training data from static HDRI maps. Extensive experiments demonstrate that DiffHDR significantly outperforms state-of-the-art approaches in radiance fidelity and temporal stability, producing realistic HDR videos with considerable latitude for re-exposure.
38
+
39
+ ## πŸ”¬ Model overview
40
+
41
+ DiffHDR uses a proposed **Log-Gamma Color Mapping** to compress linear HDR
42
+ radiance into a bounded representation aligned with natural LDR statistics.
43
+ This allows the pretrained LDR video VAE to encode and decode HDR content
44
+ without VAE finetuning or shifting its learned latent space. A LoRA-adapted
45
+ [Wan2.1-VACE-14B](https://huggingface.co/Wan-AI/Wan2.1-VACE-14B) backbone then
46
+ performs generative radiance inpainting conditioned on the input LDR content
47
+ and exposure masks, with optional text or reference-image control. The decoded
48
+ result is inverse Log-Gamma mapped to linear HDR radiance.
49
+
50
+ ## πŸ“¦ Model files
51
 
52
  This repository contains LoRA adapters, not a standalone model. Download the
53
  base model separately before running inference.
 
57
  | `DiffHDR.safetensors` | `infer_video.py`, `infer_image.py`, and `infer_long_video.py` |
58
  | `DiffHDR_Pano.safetensors` | `infer_hdri.py` for 360-degree HDR panoramas |
59
 
60
+ ## πŸ› οΈ Setup
61
 
62
  Clone and install the inference code:
63
 
 
87
  repository. Set `MODEL_BASE` if you store the base model somewhere other than
88
  `models/`.
89
 
90
+ ## πŸŽ₯ Inference
91
 
92
  Minimal video example:
93
 
 
134
  text conditioning, reference-image conditioning, arguments, and additional
135
  examples.
136
 
137
+ ## 🎯 Intended use
138
 
139
  DiffHDR is intended for research and creative LDR-to-HDR reconstruction,
140
  including:
 
145
  - text- or reference-image-guided HDR reconstruction; and
146
  - reconstructing HDR environment panoramas.
147
 
148
+ ## ⚠️ Limitations and responsible use
149
 
150
  - Detail in clipped or quantized regions is generated by the model. It is a
151
  plausible reconstruction and is not guaranteed to reproduce the original
 
162
  - Users are responsible for ensuring that their input media and intended use
163
  comply with applicable rights, licenses, and laws.
164
 
165
+ ## πŸ‹οΈ Training and evaluation
166
 
167
+ DiffHDR is trained as a LoRA adapter on top of Wan2.1-VACE-14B while keeping
168
+ the video VAE frozen. Its synthetic training set is rendered from approximately
169
+ 800 high-resolution HDRIs and contains about 5,400 temporally consistent
170
+ 81-frame HDR sequences. Exposure shifts, temporally correlated camera noise,
171
+ clipping, and 8-bit quantization are simulated to produce paired LDR inputs.
172
+ Method details, experimental settings, and comparisons are reported in the
173
  [paper](https://arxiv.org/abs/2604.06161).
174
 
175
+ ## πŸ“„ License
176
 
177
  The DiffHDR LoRA adapter weights in this repository are released under the
178
  [Apache License 2.0](LICENSE).
 
181
  to their own license and model-card guidance. This license does not grant rights
182
  to third-party code, datasets, or user-supplied input content.
183
 
184
+ ## πŸ“š Citation
185
 
186
  ```bibtex
187
  @article{yu2026diffhdr,
 
192
  }
193
  ```
194
 
195
+ ## πŸ™ Acknowledgements
196
 
197
  DiffHDR builds on
198
  [Wan2.1-VACE-14B](https://huggingface.co/Wan-AI/Wan2.1-VACE-14B) and
assets/teaser.png ADDED

Git LFS Details

  • SHA256: 0ab15ab1dd15a775ee908be05be3221a1cdfba25ce68d60fa8f737db9e91acdd
  • Pointer size: 132 Bytes
  • Size of remote file: 1.78 MB