vLAR commited on
Commit
e13fbb6
·
verified ·
1 Parent(s): 78321c7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +37 -34
README.md CHANGED
@@ -1,4 +1,4 @@
1
- ---
2
  license: cc-by-nc-sa-4.0
3
  language:
4
  - en
@@ -10,65 +10,64 @@ tags:
10
  - siggraph-asia-2026
11
  - wan2.1
12
  pipeline_tag: image-to-image
13
- ---
14
 
15
- <p align="center">
16
- <svg width="100%" height="100" viewBox="0 0 1000 100" xmlns="http://w3.org">
17
- <defs>
18
- <!-- 垂直多点渐变:模拟金属表面对光源(Relighting)的反射和暗面折射 -->
19
- <linearGradient id="relightingMetal" x1="0%" y1="0%" x2="0%" y2="100%">
20
- <!-- 顶部:金属边缘高光,在深色背景下勾勒轮廓 -->
21
- <stop offset="0%" style="stop-color:#ffffff; stop-opacity:1" />
22
- <!-- 上中部:光照直射区,高亮铝合金质感(浅色模式下依然清晰) -->
23
- <stop offset="25%" style="stop-color:#d1d5db; stop-opacity:1" />
24
- <!-- 中下部:金属核心暗面,呈现深钛合金色,提供深色对比度 -->
25
- <stop offset="70%" style="stop-color:#1f2937; stop-opacity:1" />
26
- <!-- 底部:地面或环境光的反光(Glow/Reflection Effect) -->
27
- <stop offset="100%" style="stop-color:#9ca3af; stop-opacity:1" />
28
- </linearGradient>
29
- </defs>
30
- <!-- 使用加粗现代科技字体,字间距拉开至 5,突出三维立体的分量感 -->
31
- <text x="50%" y="55%" font-family="'Montserrat', 'Helvetica Neue', 'Segoe UI', sans-serif" font-size="75" font-weight="900" fill="url(#relightingMetal)" text-anchor="middle" dominant-baseline="middle" letter-spacing="5">RelightFormer</text>
32
- </svg>
33
- </p>
34
 
 
35
 
 
 
 
 
36
 
37
  <p align="center">
38
- <img src="https://raw.githubusercontent.com/vLAR-group/RelightFormer/main/demo/teaser.jpg" alt="Teaser" width="80%">
 
 
 
 
39
  </p>
40
 
41
-
42
  <p align="center">
43
  <strong>vLAR Group</strong> | <em>SIGGRAPH Asia 2026</em>
44
  </p>
45
 
46
  <p align="center">
47
- <a href="https://github.com/vLAR-group/RelightFormer">
48
- <img src="https://img.shields.io/badge/Code-GitHub-black" alt="Code">
49
  </a>
50
  <a href="https://huggingface.co/datasets/vLAR/LavalObjaverseDataset">
51
- <img src="https://img.shields.io/badge/Dataset-Hugging%20Face-yellow" alt="Dataset">
 
 
 
 
 
 
52
  </a>
53
  </p>
54
 
55
 
56
-
57
  ## 🌟 Overview
58
 
59
- **RelightFormer** revolutionizes image relighting by replacing traditional, computationally expensive inverse rendering with a feed-forward generative Transformer. By seamlessly injecting target lighting into spatial features and processing multiple views symmetrically, it delivers highly photorealistic results. Trained on the newly introduced, large-scale open-source **Laval-Objaverse Dataset (LOD)**, RelightFormer achieves state-of-the-art quality and remarkable generalization across diverse scenes.
60
 
61
  ### ✨ Key Features
 
62
  - 🏹 **Feed-Forward Architecture**: No iterative optimization required, enabling rapid generation.
 
63
  - 🌟 **Multi-View Consistency**: Coherent and physically plausible relighting across all viewpoints.
 
64
  - ⚡ **Performant Inference**: Highly optimized and expeditious execution on modern GPUs.
 
65
  - 🎨 **Competitive Quality**: State-of-the-art, photorealistic relighting results.
66
 
67
  ---
68
 
69
  ## 🚀 Quick Start
70
 
71
- You can easily load and run the model using the `diffsynth` library in our [GitHub Repository](https://github.com/vLAR-group/RelightFormer).
72
  We provide two revisions: `main` (RelightFormer) and `post` (RelightFormer-Post, fine-tuned for enhanced quality).
73
 
74
  ```python
@@ -84,14 +83,16 @@ pipe = RelightFormerPipeline.from_pretrained(
84
  # output = pipe(image=..., lighting=..., ...)
85
  ```
86
 
87
- > 💡 **For full inference scripts, multi-GPU evaluation, and training code, please visit the [Official GitHub Repository](https://github.com/vLAR-group/RelightFormer).**
88
 
89
  ---
90
 
91
  ## 📦 Dataset
92
 
93
- This model is trained on the **Laval-Objaverse Dataset (LOD)**, comprising **90,545 high-quality 3D assets** and **39,008 diverse illumination conditions**.
 
94
  - 🤗 **Browse/Download the Dataset**: [vLAR/LavalObjaverseDataset](https://huggingface.co/datasets/vLAR/LavalObjaverseDataset)
 
95
  - 📖 **Rendering Instructions**: See the [`RENDERING_INSTRUCTION.md`](https://github.com/vLAR-group/RelightFormer/blob/main/laval-objaverse-dataset/RENDERING_INSTRUCTION.md) in the GitHub repo.
96
 
97
  ---
@@ -99,8 +100,10 @@ This model is trained on the **Laval-Objaverse Dataset (LOD)**, comprising **90,
99
  ## 🏋️ Training Details
100
 
101
  RelightFormer is fine-tuned from the **Wan 2.1** base model. The training pipeline consists of two stages:
 
102
  1. **Main Training**: Trained on the full LOD dataset to learn multi-view relighting priors.
103
- 2. **Post-Training**: A secondary fine-tuning stage to further enhance photorealism and consistency.
 
104
 
105
  Detailed training configurations, hardware requirements (e.g., 4× H200 GPUs), and scripts are available in the [GitHub Repository](https://github.com/vLAR-group/RelightFormer).
106
 
@@ -108,10 +111,10 @@ Detailed training configurations, hardware requirements (e.g., 4× H200 GPUs), a
108
 
109
  ## 📜 License
110
 
111
- This model, its code, and associated datasets are licensed under the [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License](https://creativecommons.org/licenses/by-nc-sa/4.0/) (CC BY-NC-SA 4.0).
112
 
113
  ---
114
 
115
  ## 🙏 Acknowledgements
116
 
117
- This work was supported in part by the National Natural Science Foundation of China, the Research Grants Council of Hong Kong, the Otto Poon Charitable Foundation Smart Cities Research Institute, the Research Center for Unmanned Autonomous Systems, and the PolyU Kunpeng & Ascend Technology Innovation Incubation Center, The Hong Kong Polytechnic University.
 
1
+ ```yaml
2
  license: cc-by-nc-sa-4.0
3
  language:
4
  - en
 
10
  - siggraph-asia-2026
11
  - wan2.1
12
  pipeline_tag: image-to-image
13
+ ```
14
 
15
+ <div align="center">
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
 
17
+ <h1>✨ RelightFormer ✨</h1>
18
 
19
+ <h2>Feed-Forward Multi-View Relighting
20
+ with Generative Transformers</h2>
21
+
22
+ </div>
23
 
24
  <p align="center">
25
+ <img
26
+ src="https://raw.githubusercontent.com/vLAR-group/RelightFormer/main/demo/teaser.jpg"
27
+ alt="RelightFormer teaser showing photorealistic multi-view relighting results"
28
+ width="80%"
29
+ >
30
  </p>
31
 
 
32
  <p align="center">
33
  <strong>vLAR Group</strong> | <em>SIGGRAPH Asia 2026</em>
34
  </p>
35
 
36
  <p align="center">
37
+ <a href="https://arxiv.org/abs/2609.07414">
38
+ <img src="https://img.shields.io/badge/arXiv-2609.07414-b31b1b.svg" alt="arXiv">
39
  </a>
40
  <a href="https://huggingface.co/datasets/vLAR/LavalObjaverseDataset">
41
+ <img src="https://img.shields.io/badge/🤗-Dataset-yellow" alt="Dataset">
42
+ </a>
43
+ <a href="https://huggingface.co/vLAR/RelightFormer">
44
+ <img src="https://img.shields.io/badge/🤗-Model-yellow" alt="Model">
45
+ </a>
46
+ <a href="#license">
47
+ <img src="https://img.shields.io/badge/License-CC%20BY--NC--SA%204.0-lightgrey.svg" alt="License">
48
  </a>
49
  </p>
50
 
51
 
 
52
  ## 🌟 Overview
53
 
54
+ **RelightFormer** revolutionizes image relighting by replacing traditional, computationally expensive inverse rendering with a feed-forward generative Transformer. By seamlessly injecting target lighting into spatial features and processing multiple views symmetrically, it delivers highly photorealistic results. Trained on the newly introduced, large-scale open-source **Laval-Objaverse Dataset (LOD )**, RelightFormer achieves state-of-the-art quality and remarkable generalization across diverse scenes.
55
 
56
  ### ✨ Key Features
57
+
58
  - 🏹 **Feed-Forward Architecture**: No iterative optimization required, enabling rapid generation.
59
+
60
  - 🌟 **Multi-View Consistency**: Coherent and physically plausible relighting across all viewpoints.
61
+
62
  - ⚡ **Performant Inference**: Highly optimized and expeditious execution on modern GPUs.
63
+
64
  - 🎨 **Competitive Quality**: State-of-the-art, photorealistic relighting results.
65
 
66
  ---
67
 
68
  ## 🚀 Quick Start
69
 
70
+ You can easily load and run the model using the `diffsynth` library in our [GitHub Repository](https://github.com/vLAR-group/RelightFormer).
71
  We provide two revisions: `main` (RelightFormer) and `post` (RelightFormer-Post, fine-tuned for enhanced quality).
72
 
73
  ```python
 
83
  # output = pipe(image=..., lighting=..., ...)
84
  ```
85
 
86
+ > 💡 **For full inference scripts, multi-GPU evaluation, and training code, please visit the **[**Official GitHub Repository**](https://github.com/vLAR-group/RelightFormer)**.**
87
 
88
  ---
89
 
90
  ## 📦 Dataset
91
 
92
+ This model is trained on the **Laval-Objaverse Dataset (LOD)**, comprising **90,545 high-quality 3D assets** and **39,008 diverse illumination conditions**.
93
+
94
  - 🤗 **Browse/Download the Dataset**: [vLAR/LavalObjaverseDataset](https://huggingface.co/datasets/vLAR/LavalObjaverseDataset)
95
+
96
  - 📖 **Rendering Instructions**: See the [`RENDERING_INSTRUCTION.md`](https://github.com/vLAR-group/RelightFormer/blob/main/laval-objaverse-dataset/RENDERING_INSTRUCTION.md) in the GitHub repo.
97
 
98
  ---
 
100
  ## 🏋️ Training Details
101
 
102
  RelightFormer is fine-tuned from the **Wan 2.1** base model. The training pipeline consists of two stages:
103
+
104
  1. **Main Training**: Trained on the full LOD dataset to learn multi-view relighting priors.
105
+
106
+ 1. **Post-Training**: A secondary fine-tuning stage to further enhance photorealism and consistency.
107
 
108
  Detailed training configurations, hardware requirements (e.g., 4× H200 GPUs), and scripts are available in the [GitHub Repository](https://github.com/vLAR-group/RelightFormer).
109
 
 
111
 
112
  ## 📜 License
113
 
114
+ This model, its code, and associated datasets are licensed under the [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License](https://creativecommons.org/licenses/by-nc-sa/4.0/) (CC BY-NC-SA 4.0).
115
 
116
  ---
117
 
118
  ## 🙏 Acknowledgements
119
 
120
+ This work was supported in part by the National Natural Science Foundation of China, the Research Grants Council of Hong Kong, the Otto Poon Charitable Foundation Smart Cities Research Institute, the Research Center for Unmanned Autonomous Systems, and the PolyU Kunpeng & Ascend Technology Innovation Incubation Center, The Hong Kong Polytechnic University.