ChenDY commited on
Commit
2647be6
·
0 Parent(s):

Release NitroFusion2-Controlnet-Union

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +37 -0
  2. LICENSE +204 -0
  3. NitroFusion2-Controlnet-Union-lite.safetensors +3 -0
  4. NitroFusion2-Controlnet-Union.safetensors +3 -0
  5. README.md +110 -0
  6. asset/canny.jpg +3 -0
  7. asset/canny2.jpg +3 -0
  8. asset/depth.jpg +0 -0
  9. asset/depth2.jpg +0 -0
  10. asset/hed.jpg +3 -0
  11. asset/hed2.jpg +3 -0
  12. asset/pose.jpg +0 -0
  13. asset/pose2.jpg +0 -0
  14. asset/pose3.jpg +0 -0
  15. asset/pose_inpaint.jpg +0 -0
  16. asset/pose_inpaint_mask.png +0 -0
  17. asset/pose_inpaint_pose.jpg +0 -0
  18. asset/pose_inpaint_source.png +3 -0
  19. results/2step/canny2_original.jpg +3 -0
  20. results/2step/canny2_ours.jpg +3 -0
  21. results/2step/canny_original.jpg +3 -0
  22. results/2step/canny_ours.jpg +3 -0
  23. results/2step/depth2_original.jpg +3 -0
  24. results/2step/depth2_ours.jpg +3 -0
  25. results/2step/depth_original.jpg +3 -0
  26. results/2step/depth_ours.jpg +3 -0
  27. results/2step/hed2_original.jpg +3 -0
  28. results/2step/hed2_ours.jpg +3 -0
  29. results/2step/hed_original.jpg +3 -0
  30. results/2step/hed_ours.jpg +3 -0
  31. results/2step/pose2_original.jpg +3 -0
  32. results/2step/pose2_ours.jpg +3 -0
  33. results/2step/pose3_original.jpg +3 -0
  34. results/2step/pose3_ours.jpg +3 -0
  35. results/2step/pose_inpaint_original.jpg +3 -0
  36. results/2step/pose_inpaint_ours.jpg +3 -0
  37. results/canny2_original.jpg +3 -0
  38. results/canny2_ours.jpg +3 -0
  39. results/canny_original.jpg +3 -0
  40. results/canny_ours.jpg +3 -0
  41. results/depth2_original.jpg +3 -0
  42. results/depth2_ours.jpg +3 -0
  43. results/depth_original.jpg +3 -0
  44. results/depth_ours.jpg +3 -0
  45. results/hed2_original.jpg +3 -0
  46. results/hed2_ours.jpg +3 -0
  47. results/hed_original.jpg +3 -0
  48. results/hed_ours.jpg +3 -0
  49. results/pose2_original.jpg +3 -0
  50. results/pose2_ours.jpg +3 -0
.gitattributes ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.png filter=lfs diff=lfs merge=lfs -text
2
+ *.jpg filter=lfs diff=lfs merge=lfs -text
3
+ *.7z filter=lfs diff=lfs merge=lfs -text
4
+ *.arrow filter=lfs diff=lfs merge=lfs -text
5
+ *.bin filter=lfs diff=lfs merge=lfs -text
6
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
7
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
8
+ *.ftz filter=lfs diff=lfs merge=lfs -text
9
+ *.gz filter=lfs diff=lfs merge=lfs -text
10
+ *.h5 filter=lfs diff=lfs merge=lfs -text
11
+ *.joblib filter=lfs diff=lfs merge=lfs -text
12
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
13
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
14
+ *.model filter=lfs diff=lfs merge=lfs -text
15
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
16
+ *.npy filter=lfs diff=lfs merge=lfs -text
17
+ *.npz filter=lfs diff=lfs merge=lfs -text
18
+ *.onnx filter=lfs diff=lfs merge=lfs -text
19
+ *.ot filter=lfs diff=lfs merge=lfs -text
20
+ *.parquet filter=lfs diff=lfs merge=lfs -text
21
+ *.pb filter=lfs diff=lfs merge=lfs -text
22
+ *.pickle filter=lfs diff=lfs merge=lfs -text
23
+ *.pkl filter=lfs diff=lfs merge=lfs -text
24
+ *.pt filter=lfs diff=lfs merge=lfs -text
25
+ *.pth filter=lfs diff=lfs merge=lfs -text
26
+ *.rar filter=lfs diff=lfs merge=lfs -text
27
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
28
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
29
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
30
+ *.tar filter=lfs diff=lfs merge=lfs -text
31
+ *.tflite filter=lfs diff=lfs merge=lfs -text
32
+ *.tgz filter=lfs diff=lfs merge=lfs -text
33
+ *.wasm filter=lfs diff=lfs merge=lfs -text
34
+ *.xz filter=lfs diff=lfs merge=lfs -text
35
+ *.zip filter=lfs diff=lfs merge=lfs -text
36
+ *.zst filter=lfs diff=lfs merge=lfs -text
37
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
LICENSE ADDED
@@ -0,0 +1,204 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Copyright 2026 The NitroFusion2 Authors.
2
+
3
+ Apache License
4
+ Version 2.0, January 2004
5
+ http://www.apache.org/licenses/
6
+
7
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
8
+
9
+ 1. Definitions.
10
+
11
+ "License" shall mean the terms and conditions for use, reproduction,
12
+ and distribution as defined by Sections 1 through 9 of this document.
13
+
14
+ "Licensor" shall mean the copyright owner or entity authorized by
15
+ the copyright owner that is granting the License.
16
+
17
+ "Legal Entity" shall mean the union of the acting entity and all
18
+ other entities that control, are controlled by, or are under common
19
+ control with that entity. For the purposes of this definition,
20
+ "control" means (i) the power, direct or indirect, to cause the
21
+ direction or management of such entity, whether by contract or
22
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
23
+ outstanding shares, or (iii) beneficial ownership of such entity.
24
+
25
+ "You" (or "Your") shall mean an individual or Legal Entity
26
+ exercising permissions granted by this License.
27
+
28
+ "Source" form shall mean the preferred form for making modifications,
29
+ including but not limited to software source code, documentation
30
+ source, and configuration files.
31
+
32
+ "Object" form shall mean any form resulting from mechanical
33
+ transformation or translation of a Source form, including but
34
+ not limited to compiled object code, generated documentation,
35
+ and conversions to other media types.
36
+
37
+ "Work" shall mean the work of authorship, whether in Source or
38
+ Object form, made available under the License, as indicated by a
39
+ copyright notice that is included in or attached to the work
40
+ (an example is provided in the Appendix below).
41
+
42
+ "Derivative Works" shall mean any work, whether in Source or Object
43
+ form, that is based on (or derived from) the Work and for which the
44
+ editorial revisions, annotations, elaborations, or other modifications
45
+ represent, as a whole, an original work of authorship. For the purposes
46
+ of this License, Derivative Works shall not include works that remain
47
+ separable from, or merely link (or bind by name) to the interfaces of,
48
+ the Work and Derivative Works thereof.
49
+
50
+ "Contribution" shall mean any work of authorship, including
51
+ the original version of the Work and any modifications or additions
52
+ to that Work or Derivative Works thereof, that is intentionally
53
+ submitted to Licensor for inclusion in the Work by the copyright owner
54
+ or by an individual or Legal Entity authorized to submit on behalf of
55
+ the copyright owner. For the purposes of this definition, "submitted"
56
+ means any form of electronic, verbal, or written communication sent
57
+ to the Licensor or its representatives, including but not limited to
58
+ communication on electronic mailing lists, source code control systems,
59
+ and issue tracking systems that are managed by, or on behalf of, the
60
+ Licensor for the purpose of discussing and improving the Work, but
61
+ excluding communication that is conspicuously marked or otherwise
62
+ designated in writing by the copyright owner as "Not a Contribution."
63
+
64
+ "Contributor" shall mean Licensor and any individual or Legal Entity
65
+ on behalf of whom a Contribution has been received by Licensor and
66
+ subsequently incorporated within the Work.
67
+
68
+ 2. Grant of Copyright License. Subject to the terms and conditions of
69
+ this License, each Contributor hereby grants to You a perpetual,
70
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
71
+ copyright license to reproduce, prepare Derivative Works of,
72
+ publicly display, publicly perform, sublicense, and distribute the
73
+ Work and such Derivative Works in Source or Object form.
74
+
75
+ 3. Grant of Patent License. Subject to the terms and conditions of
76
+ this License, each Contributor hereby grants to You a perpetual,
77
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
78
+ (except as stated in this section) patent license to make, have made,
79
+ use, offer to sell, sell, import, and otherwise transfer the Work,
80
+ where such license applies only to those patent claims licensable
81
+ by such Contributor that are necessarily infringed by their
82
+ Contribution(s) alone or by combination of their Contribution(s)
83
+ with the Work to which such Contribution(s) was submitted. If You
84
+ institute patent litigation against any entity (including a
85
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
86
+ or a Contribution incorporated within the Work constitutes direct
87
+ or contributory patent infringement, then any patent licenses
88
+ granted to You under this License for that Work shall terminate
89
+ as of the date such litigation is filed.
90
+
91
+ 4. Redistribution. You may reproduce and distribute copies of the
92
+ Work or Derivative Works thereof in any medium, with or without
93
+ modifications, and in Source or Object form, provided that You
94
+ meet the following conditions:
95
+
96
+ (a) You must give any other recipients of the Work or
97
+ Derivative Works a copy of this License; and
98
+
99
+ (b) You must cause any modified files to carry prominent notices
100
+ stating that You changed the files; and
101
+
102
+ (c) You must retain, in the Source form of any Derivative Works
103
+ that You distribute, all copyright, patent, trademark, and
104
+ attribution notices from the Source form of the Work,
105
+ excluding those notices that do not pertain to any part of
106
+ the Derivative Works; and
107
+
108
+ (d) If the Work includes a "NOTICE" text file as part of its
109
+ distribution, then any Derivative Works that You distribute must
110
+ include a readable copy of the attribution notices contained
111
+ within such NOTICE file, excluding those notices that do not
112
+ pertain to any part of the Derivative Works, in at least one
113
+ of the following places: within a NOTICE text file distributed
114
+ as part of the Derivative Works; within the Source form or
115
+ documentation, if provided along with the Derivative Works; or,
116
+ within a display generated by the Derivative Works, if and
117
+ wherever such third-party notices normally appear. The contents
118
+ of the NOTICE file are for informational purposes only and
119
+ do not modify the License. You may add Your own attribution
120
+ notices within Derivative Works that You distribute, alongside
121
+ or as an addendum to the NOTICE text from the Work, provided
122
+ that such additional attribution notices cannot be construed
123
+ as modifying the License.
124
+
125
+ You may add Your own copyright statement to Your modifications and
126
+ may provide additional or different license terms and conditions
127
+ for use, reproduction, or distribution of Your modifications, or
128
+ for any such Derivative Works as a whole, provided Your use,
129
+ reproduction, and distribution of the Work otherwise complies with
130
+ the conditions stated in this License.
131
+
132
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
133
+ any Contribution intentionally submitted for inclusion in the Work
134
+ by You to the Licensor shall be under the terms and conditions of
135
+ this License, without any additional terms or conditions.
136
+ Notwithstanding the above, nothing herein shall supersede or modify
137
+ the terms of any separate license agreement you may have executed
138
+ with Licensor regarding such Contributions.
139
+
140
+ 6. Trademarks. This License does not grant permission to use the trade
141
+ names, trademarks, service marks, or product names of the Licensor,
142
+ except as required for reasonable and customary use in describing the
143
+ origin of the Work and reproducing the content of the NOTICE file.
144
+
145
+ 7. Disclaimer of Warranty. Unless required by applicable law or
146
+ agreed to in writing, Licensor provides the Work (and each
147
+ Contributor provides its Contributions) on an "AS IS" BASIS,
148
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
149
+ implied, including, without limitation, any warranties or conditions
150
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
151
+ PARTICULAR PURPOSE. You are solely responsible for determining the
152
+ appropriateness of using or redistributing the Work and assume any
153
+ risks associated with Your exercise of permissions under this License.
154
+
155
+ 8. Limitation of Liability. In no event and under no legal theory,
156
+ whether in tort (including negligence), contract, or otherwise,
157
+ unless required by applicable law (such as deliberate and grossly
158
+ negligent acts) or agreed to in writing, shall any Contributor be
159
+ liable to You for damages, including any direct, indirect, special,
160
+ incidental, or consequential damages of any character arising as a
161
+ result of this License or out of the use or inability to use the
162
+ Work (including but not limited to damages for loss of goodwill,
163
+ work stoppage, computer failure or malfunction, or any and all
164
+ other commercial damages or losses), even if such Contributor
165
+ has been advised of the possibility of such damages.
166
+
167
+ 9. Accepting Warranty or Additional Liability. While redistributing
168
+ the Work or Derivative Works thereof, You may choose to offer,
169
+ and charge a fee for, acceptance of support, warranty, indemnity,
170
+ or other liability obligations and/or rights consistent with this
171
+ License. However, in accepting such obligations, You may act only
172
+ on Your own behalf and on Your sole responsibility, not on behalf
173
+ of any other Contributor, and only if You agree to indemnify,
174
+ defend, and hold each Contributor harmless for any liability
175
+ incurred by, or claims asserted against, such Contributor by reason
176
+ of your accepting any such warranty or additional liability.
177
+
178
+ END OF TERMS AND CONDITIONS
179
+
180
+ APPENDIX: How to apply the Apache License to your work.
181
+
182
+ To apply the Apache License to your work, attach the following
183
+ boilerplate notice, with the fields enclosed by brackets "[]"
184
+ replaced with your own identifying information. (Don't include
185
+ the brackets!) The text should be enclosed in the appropriate
186
+ comment syntax for the file format. We also recommend that a
187
+ file or class name and description of purpose be included on the
188
+ same "printed page" as the copyright notice for easier
189
+ identification within third-party archives.
190
+
191
+ Copyright 2026 The NitroFusion2 Authors.
192
+
193
+ Licensed under the Apache License, Version 2.0 (the "License");
194
+ you may not use this file except in compliance with the License.
195
+ You may obtain a copy of the License at
196
+
197
+ http://www.apache.org/licenses/LICENSE-2.0
198
+
199
+ Unless required by applicable law or agreed to in writing, software
200
+ distributed under the License is distributed on an "AS IS" BASIS,
201
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
202
+ See the License for the specific language governing permissions and
203
+ limitations under the License.
204
+
NitroFusion2-Controlnet-Union-lite.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8a6cb600703aeec9b65e5d76c331d745c2429a72e5c2cc3dbcc5cf8c6287165f
3
+ size 2016627488
NitroFusion2-Controlnet-Union.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fe69e4fe0158aaf6303c6cf13598ed20378349df16f1a34c8224a076468ef306
3
+ size 6712485600
README.md ADDED
@@ -0,0 +1,110 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ pipeline_tag: text-to-image
6
+ library_name: diffusion-single-file
7
+ base_model:
8
+ - ChenDY/NitroFusion2
9
+ - alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1
10
+ tags:
11
+ - controlnet
12
+ - text-to-image
13
+ - one-step
14
+ - few-step
15
+ - comfyui
16
+ - nitrofusion2
17
+ ---
18
+
19
+ # NitroFusion2-Controlnet-Union
20
+
21
+ [![Demo](https://img.shields.io/badge/%F0%9F%A4%97%20Demo-NitroFusion2-ff7a00?labelColor=555)](https://huggingface.co/spaces/ChenDY/NitroFusion2)
22
+ [![NitroFusion2](https://img.shields.io/badge/%F0%9F%A4%97%20Checkpoint-NitroFusion2-ffc107?labelColor=555)](https://huggingface.co/ChenDY/NitroFusion2)
23
+ [![GitHub](https://img.shields.io/badge/GitHub-NitroFusion2-24292f?logo=github&labelColor=555)](https://github.com/ChenDarYen/NitroFusion2)
24
+ [![Project Page](https://img.shields.io/badge/%F0%9F%8C%90%20Project%20Page-NitroFusion2-1f72b8?labelColor=555)](https://chendaryen.github.io/NitroFusion2.github.io/)
25
+ [![arXiv](https://img.shields.io/badge/arXiv-2611.05648-b31b1b?labelColor=555)](https://arxiv.org/abs/2611.05648)
26
+
27
+ NitroFusion2-Controlnet-Union is a finetuned version of [Z-Image-Turbo-Fun-Controlnet-Union-2.1](https://huggingface.co/alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1) (2602), adapted to [NitroFusion2](https://huggingface.co/ChenDY/NitroFusion2) for 1-step and 2-step generation. It must be used with NitroFusion2 as the base model.
28
+
29
+ Supported conditions: Canny, HED, Depth, Pose, Scribble, Gray, and inpainting.
30
+
31
+ ## ComfyUI
32
+
33
+ - 1 step: [`workflows/nitrofusion2_controlnet_1step.json`](./workflows/nitrofusion2_controlnet_1step.json)
34
+ - 2 steps: [`workflows/nitrofusion2_controlnet_2step.json`](./workflows/nitrofusion2_controlnet_2step.json)
35
+
36
+ ## Model Card
37
+
38
+ | Name | Description |
39
+ |--|--|
40
+ | NitroFusion2-Controlnet-Union.safetensors | Union 2602 (15 control layer blocks + 2 refiner blocks) adapted to NitroFusion2 for 1–2 step inference. |
41
+ | NitroFusion2-Controlnet-Union-lite.safetensors | Lite version (3 control layer blocks + 2 refiner blocks) adapted the same way; smaller and lighter on memory, with weaker control. |
42
+
43
+ ## Results
44
+
45
+ ### 1 step
46
+
47
+ <table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
48
+ <tr><td>Pose</td><td>Union 2.1-2602 + NitroFusion2</td><td><b>NitroFusion2-Controlnet-Union</b></td></tr>
49
+ <tr><td width="33%"><img src="asset/pose2.jpg" width="100%" alt="Pose control map" /></td><td width="33%"><img src="results/pose2_original.jpg" width="100%" alt="Union 2.1-2602 + NitroFusion2, 1 step" /></td><td width="33%"><img src="results/pose2_ours.jpg" width="100%" alt="NitroFusion2-Controlnet-Union, 1 step" /></td></tr>
50
+ <tr><td>Pose</td><td>Union 2.1-2602 + NitroFusion2</td><td><b>NitroFusion2-Controlnet-Union</b></td></tr>
51
+ <tr><td width="33%"><img src="asset/pose3.jpg" width="100%" alt="Pose control map" /></td><td width="33%"><img src="results/pose3_original.jpg" width="100%" alt="Union 2.1-2602 + NitroFusion2, 1 step" /></td><td width="33%"><img src="results/pose3_ours.jpg" width="100%" alt="NitroFusion2-Controlnet-Union, 1 step" /></td></tr>
52
+ <tr><td>Canny</td><td>Union 2.1-2602 + NitroFusion2</td><td><b>NitroFusion2-Controlnet-Union</b></td></tr>
53
+ <tr><td width="33%"><img src="asset/canny.jpg" width="100%" alt="Canny control map" /></td><td width="33%"><img src="results/canny_original.jpg" width="100%" alt="Union 2.1-2602 + NitroFusion2, 1 step" /></td><td width="33%"><img src="results/canny_ours.jpg" width="100%" alt="NitroFusion2-Controlnet-Union, 1 step" /></td></tr>
54
+ <tr><td>Canny</td><td>Union 2.1-2602 + NitroFusion2</td><td><b>NitroFusion2-Controlnet-Union</b></td></tr>
55
+ <tr><td width="33%"><img src="asset/canny2.jpg" width="100%" alt="Canny control map" /></td><td width="33%"><img src="results/canny2_original.jpg" width="100%" alt="Union 2.1-2602 + NitroFusion2, 1 step" /></td><td width="33%"><img src="results/canny2_ours.jpg" width="100%" alt="NitroFusion2-Controlnet-Union, 1 step" /></td></tr>
56
+ <tr><td>HED</td><td>Union 2.1-2602 + NitroFusion2</td><td><b>NitroFusion2-Controlnet-Union</b></td></tr>
57
+ <tr><td width="33%"><img src="asset/hed.jpg" width="100%" alt="HED control map" /></td><td width="33%"><img src="results/hed_original.jpg" width="100%" alt="Union 2.1-2602 + NitroFusion2, 1 step" /></td><td width="33%"><img src="results/hed_ours.jpg" width="100%" alt="NitroFusion2-Controlnet-Union, 1 step" /></td></tr>
58
+ <tr><td>HED</td><td>Union 2.1-2602 + NitroFusion2</td><td><b>NitroFusion2-Controlnet-Union</b></td></tr>
59
+ <tr><td width="33%"><img src="asset/hed2.jpg" width="100%" alt="HED control map" /></td><td width="33%"><img src="results/hed2_original.jpg" width="100%" alt="Union 2.1-2602 + NitroFusion2, 1 step" /></td><td width="33%"><img src="results/hed2_ours.jpg" width="100%" alt="NitroFusion2-Controlnet-Union, 1 step" /></td></tr>
60
+ <tr><td>Depth</td><td>Union 2.1-2602 + NitroFusion2</td><td><b>NitroFusion2-Controlnet-Union</b></td></tr>
61
+ <tr><td width="33%"><img src="asset/depth.jpg" width="100%" alt="Depth control map" /></td><td width="33%"><img src="results/depth_original.jpg" width="100%" alt="Union 2.1-2602 + NitroFusion2, 1 step" /></td><td width="33%"><img src="results/depth_ours.jpg" width="100%" alt="NitroFusion2-Controlnet-Union, 1 step" /></td></tr>
62
+ <tr><td>Depth</td><td>Union 2.1-2602 + NitroFusion2</td><td><b>NitroFusion2-Controlnet-Union</b></td></tr>
63
+ <tr><td width="33%"><img src="asset/depth2.jpg" width="100%" alt="Depth control map" /></td><td width="33%"><img src="results/depth2_original.jpg" width="100%" alt="Union 2.1-2602 + NitroFusion2, 1 step" /></td><td width="33%"><img src="results/depth2_ours.jpg" width="100%" alt="NitroFusion2-Controlnet-Union, 1 step" /></td></tr>
64
+ <tr><td>Pose + Inpaint</td><td>Union 2.1-2602 + NitroFusion2</td><td><b>NitroFusion2-Controlnet-Union</b></td></tr>
65
+ <tr><td width="33%"><img src="asset/pose_inpaint.jpg" width="100%" alt="Source image, masked image and pose map" /></td><td width="33%"><img src="results/pose_inpaint_original.jpg" width="100%" alt="Union 2.1-2602 + NitroFusion2, 1 step" /></td><td width="33%"><img src="results/pose_inpaint_ours.jpg" width="100%" alt="NitroFusion2-Controlnet-Union, 1 step" /></td></tr>
66
+ </table>
67
+
68
+ ### 2 steps
69
+
70
+ <table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
71
+ <tr><td>Pose</td><td>Union 2.1-2602 + NitroFusion2</td><td><b>NitroFusion2-Controlnet-Union</b></td></tr>
72
+ <tr><td width="33%"><img src="asset/pose2.jpg" width="100%" alt="Pose control map" /></td><td width="33%"><img src="results/2step/pose2_original.jpg" width="100%" alt="Union 2.1-2602 + NitroFusion2, 2 steps" /></td><td width="33%"><img src="results/2step/pose2_ours.jpg" width="100%" alt="NitroFusion2-Controlnet-Union, 2 steps" /></td></tr>
73
+ <tr><td>Pose</td><td>Union 2.1-2602 + NitroFusion2</td><td><b>NitroFusion2-Controlnet-Union</b></td></tr>
74
+ <tr><td width="33%"><img src="asset/pose3.jpg" width="100%" alt="Pose control map" /></td><td width="33%"><img src="results/2step/pose3_original.jpg" width="100%" alt="Union 2.1-2602 + NitroFusion2, 2 steps" /></td><td width="33%"><img src="results/2step/pose3_ours.jpg" width="100%" alt="NitroFusion2-Controlnet-Union, 2 steps" /></td></tr>
75
+ <tr><td>Canny</td><td>Union 2.1-2602 + NitroFusion2</td><td><b>NitroFusion2-Controlnet-Union</b></td></tr>
76
+ <tr><td width="33%"><img src="asset/canny.jpg" width="100%" alt="Canny control map" /></td><td width="33%"><img src="results/2step/canny_original.jpg" width="100%" alt="Union 2.1-2602 + NitroFusion2, 2 steps" /></td><td width="33%"><img src="results/2step/canny_ours.jpg" width="100%" alt="NitroFusion2-Controlnet-Union, 2 steps" /></td></tr>
77
+ <tr><td>Canny</td><td>Union 2.1-2602 + NitroFusion2</td><td><b>NitroFusion2-Controlnet-Union</b></td></tr>
78
+ <tr><td width="33%"><img src="asset/canny2.jpg" width="100%" alt="Canny control map" /></td><td width="33%"><img src="results/2step/canny2_original.jpg" width="100%" alt="Union 2.1-2602 + NitroFusion2, 2 steps" /></td><td width="33%"><img src="results/2step/canny2_ours.jpg" width="100%" alt="NitroFusion2-Controlnet-Union, 2 steps" /></td></tr>
79
+ <tr><td>HED</td><td>Union 2.1-2602 + NitroFusion2</td><td><b>NitroFusion2-Controlnet-Union</b></td></tr>
80
+ <tr><td width="33%"><img src="asset/hed.jpg" width="100%" alt="HED control map" /></td><td width="33%"><img src="results/2step/hed_original.jpg" width="100%" alt="Union 2.1-2602 + NitroFusion2, 2 steps" /></td><td width="33%"><img src="results/2step/hed_ours.jpg" width="100%" alt="NitroFusion2-Controlnet-Union, 2 steps" /></td></tr>
81
+ <tr><td>HED</td><td>Union 2.1-2602 + NitroFusion2</td><td><b>NitroFusion2-Controlnet-Union</b></td></tr>
82
+ <tr><td width="33%"><img src="asset/hed2.jpg" width="100%" alt="HED control map" /></td><td width="33%"><img src="results/2step/hed2_original.jpg" width="100%" alt="Union 2.1-2602 + NitroFusion2, 2 steps" /></td><td width="33%"><img src="results/2step/hed2_ours.jpg" width="100%" alt="NitroFusion2-Controlnet-Union, 2 steps" /></td></tr>
83
+ <tr><td>Depth</td><td>Union 2.1-2602 + NitroFusion2</td><td><b>NitroFusion2-Controlnet-Union</b></td></tr>
84
+ <tr><td width="33%"><img src="asset/depth.jpg" width="100%" alt="Depth control map" /></td><td width="33%"><img src="results/2step/depth_original.jpg" width="100%" alt="Union 2.1-2602 + NitroFusion2, 2 steps" /></td><td width="33%"><img src="results/2step/depth_ours.jpg" width="100%" alt="NitroFusion2-Controlnet-Union, 2 steps" /></td></tr>
85
+ <tr><td>Depth</td><td>Union 2.1-2602 + NitroFusion2</td><td><b>NitroFusion2-Controlnet-Union</b></td></tr>
86
+ <tr><td width="33%"><img src="asset/depth2.jpg" width="100%" alt="Depth control map" /></td><td width="33%"><img src="results/2step/depth2_original.jpg" width="100%" alt="Union 2.1-2602 + NitroFusion2, 2 steps" /></td><td width="33%"><img src="results/2step/depth2_ours.jpg" width="100%" alt="NitroFusion2-Controlnet-Union, 2 steps" /></td></tr>
87
+ <tr><td>Pose + Inpaint</td><td>Union 2.1-2602 + NitroFusion2</td><td><b>NitroFusion2-Controlnet-Union</b></td></tr>
88
+ <tr><td width="33%"><img src="asset/pose_inpaint.jpg" width="100%" alt="Source image, masked image and pose map" /></td><td width="33%"><img src="results/2step/pose_inpaint_original.jpg" width="100%" alt="Union 2.1-2602 + NitroFusion2, 2 steps" /></td><td width="33%"><img src="results/2step/pose_inpaint_ours.jpg" width="100%" alt="NitroFusion2-Controlnet-Union, 2 steps" /></td></tr>
89
+ </table>
90
+
91
+ ## Citation
92
+
93
+ If you find NitroFusion2 useful in your research, please cite our paper:
94
+
95
+ ```bib
96
+ @article{chen2026nitrofusion2,
97
+ title={NitroFusion2: Bringing Adversarial Loss to the Text Stream of Multimodal Diffusion Transformers},
98
+ author={Chen, Dar-Yen and Song, Yi-Zhe},
99
+ journal={arXiv preprint arXiv:2611.05648},
100
+ year={2026}
101
+ }
102
+ ```
103
+
104
+ ## Acknowledgements
105
+
106
+ This model is adapted from [alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1](https://huggingface.co/alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1). Many thanks to the authors for releasing the ControlNet.
107
+
108
+ ## License
109
+
110
+ [Apache 2.0](./LICENSE)
asset/canny.jpg ADDED

Git LFS Details

  • SHA256: 7e99213635d00be6d5a17e7ca0fccee123b612b6e35fefc7dc2fec1964f6103f
  • Pointer size: 131 Bytes
  • Size of remote file: 217 kB
asset/canny2.jpg ADDED

Git LFS Details

  • SHA256: 7f88224d0b084e10aa2f91df5fc10adbf0fd433c5e7bfb10986c5bd2acccb1d2
  • Pointer size: 131 Bytes
  • Size of remote file: 275 kB
asset/depth.jpg ADDED
asset/depth2.jpg ADDED
asset/hed.jpg ADDED

Git LFS Details

  • SHA256: 03dd7aa58adbc5393fe9bc0bd216a5f64088bc16ad63f6ec943726127a1944ec
  • Pointer size: 131 Bytes
  • Size of remote file: 100 kB
asset/hed2.jpg ADDED

Git LFS Details

  • SHA256: ca84566ab7092e69e3843f7f7a878bdda81bcf304d68048324b6ecdae3c73b4f
  • Pointer size: 131 Bytes
  • Size of remote file: 141 kB
asset/pose.jpg ADDED
asset/pose2.jpg ADDED
asset/pose3.jpg ADDED
asset/pose_inpaint.jpg ADDED
asset/pose_inpaint_mask.png ADDED
asset/pose_inpaint_pose.jpg ADDED
asset/pose_inpaint_source.png ADDED

Git LFS Details

  • SHA256: 0e6965c986c6c163b2c9ff512f2a0f3ed7da8d5f382874ae288203416ef905e9
  • Pointer size: 131 Bytes
  • Size of remote file: 581 kB
results/2step/canny2_original.jpg ADDED

Git LFS Details

  • SHA256: 0c4577337c6c552eeb9a743cea1f58e8f04334444b9396b12286c9df3551936a
  • Pointer size: 131 Bytes
  • Size of remote file: 202 kB
results/2step/canny2_ours.jpg ADDED

Git LFS Details

  • SHA256: 90caa5eabc164c108b18cd5513864648de27846d7d17ff798fb240a1f09623bd
  • Pointer size: 131 Bytes
  • Size of remote file: 229 kB
results/2step/canny_original.jpg ADDED

Git LFS Details

  • SHA256: 5e17aa0e0f51d4053b86adbf2e5843c266aeb0e8240511765c1080bc2f479f8e
  • Pointer size: 131 Bytes
  • Size of remote file: 212 kB
results/2step/canny_ours.jpg ADDED

Git LFS Details

  • SHA256: 35bc8307d1ec4ad6d97cb7ad65250158dd54496f88d21e9b04872e5ea0c65d5f
  • Pointer size: 131 Bytes
  • Size of remote file: 232 kB
results/2step/depth2_original.jpg ADDED

Git LFS Details

  • SHA256: a760b1ed2e138391d04c2dc952e87a4d6cad745f49630c1932d616f83edbee95
  • Pointer size: 131 Bytes
  • Size of remote file: 167 kB
results/2step/depth2_ours.jpg ADDED

Git LFS Details

  • SHA256: 61f8e5218d6fdd8cf05907308f0d448ea1187a2426d03c12072b2fb40064b483
  • Pointer size: 131 Bytes
  • Size of remote file: 264 kB
results/2step/depth_original.jpg ADDED

Git LFS Details

  • SHA256: bfebc8cf3a1adb47659877a70eda95877f24204130f95e25df75e5d19fe5c4bf
  • Pointer size: 131 Bytes
  • Size of remote file: 191 kB
results/2step/depth_ours.jpg ADDED

Git LFS Details

  • SHA256: e756fa5cbbbb2fd57ba4617b37cfcfb357b6ad56c2ec51abbd83a45d0217aa15
  • Pointer size: 131 Bytes
  • Size of remote file: 230 kB
results/2step/hed2_original.jpg ADDED

Git LFS Details

  • SHA256: a058ddb615f0ed7bb3105ca924c6389dc8adcc0377f235b3cf80f0d3223bbc07
  • Pointer size: 131 Bytes
  • Size of remote file: 132 kB
results/2step/hed2_ours.jpg ADDED

Git LFS Details

  • SHA256: a694827d668950ca93c8040f39a60417605a87ed421a9a8e74cc4cc34ba5b81d
  • Pointer size: 131 Bytes
  • Size of remote file: 166 kB
results/2step/hed_original.jpg ADDED

Git LFS Details

  • SHA256: 9f941b102ebaf618eb15d8619928cf3085104533c9e9bb565fff98f8c283425d
  • Pointer size: 131 Bytes
  • Size of remote file: 141 kB
results/2step/hed_ours.jpg ADDED

Git LFS Details

  • SHA256: 4d2abfad29e54f5330d220a1703366f0a08b69f8b77b5e8d77350548ce5c4272
  • Pointer size: 131 Bytes
  • Size of remote file: 170 kB
results/2step/pose2_original.jpg ADDED

Git LFS Details

  • SHA256: 1c38e18883c06c2d6805b8e760c971f0caf8dad8d0844d41ccbdcdb2574863fb
  • Pointer size: 131 Bytes
  • Size of remote file: 212 kB
results/2step/pose2_ours.jpg ADDED

Git LFS Details

  • SHA256: 7b6d27ee30e5211e715f5d59c1957b605996b42dc7a89537f0e9cc376dcf6ad4
  • Pointer size: 131 Bytes
  • Size of remote file: 249 kB
results/2step/pose3_original.jpg ADDED

Git LFS Details

  • SHA256: 99e9fd3960f91f2e283df038781cc78cf7278bef2c4d9a32ab20131b87db865e
  • Pointer size: 131 Bytes
  • Size of remote file: 222 kB
results/2step/pose3_ours.jpg ADDED

Git LFS Details

  • SHA256: 59827ab34d55cbd3aba57e566caf56e8f19b2e73dae4e8250a59e0a1dac9f6ad
  • Pointer size: 131 Bytes
  • Size of remote file: 255 kB
results/2step/pose_inpaint_original.jpg ADDED

Git LFS Details

  • SHA256: dfa1ebdf2a90642ca727728b55d27369475a0b49aa87ff9d276dbd038bbc147c
  • Pointer size: 131 Bytes
  • Size of remote file: 181 kB
results/2step/pose_inpaint_ours.jpg ADDED

Git LFS Details

  • SHA256: 8b6edd7d4a1415b9e5f8e96e87ea0103ca43f0156443683d010f719e3668809e
  • Pointer size: 131 Bytes
  • Size of remote file: 184 kB
results/canny2_original.jpg ADDED

Git LFS Details

  • SHA256: 669f530b7943162edaf81f2f83da726b2443eef9b4a049994e22bbd20f65dfdc
  • Pointer size: 131 Bytes
  • Size of remote file: 228 kB
results/canny2_ours.jpg ADDED

Git LFS Details

  • SHA256: 867296d0739338a16fdc8c3f7ceb3cd76a7e6822ee5c4933edc5b115a5812e17
  • Pointer size: 131 Bytes
  • Size of remote file: 245 kB
results/canny_original.jpg ADDED

Git LFS Details

  • SHA256: 19afd090651ec5a4a66101b236540cab9a050c6fe1e76b336bef62881cb5a9cd
  • Pointer size: 131 Bytes
  • Size of remote file: 247 kB
results/canny_ours.jpg ADDED

Git LFS Details

  • SHA256: 603076fd392acaa1f2c551f712826f16e21496f7b2387f7f33355456f5c22d84
  • Pointer size: 131 Bytes
  • Size of remote file: 268 kB
results/depth2_original.jpg ADDED

Git LFS Details

  • SHA256: bcf160249db312c20338c0d29318e9e72a8c83aa35a91000aaf09fa3329ae85c
  • Pointer size: 131 Bytes
  • Size of remote file: 183 kB
results/depth2_ours.jpg ADDED

Git LFS Details

  • SHA256: c418872bddcdbc6e161ac80e0b735f15dc814258c11681efc572b2e822ddcea4
  • Pointer size: 131 Bytes
  • Size of remote file: 282 kB
results/depth_original.jpg ADDED

Git LFS Details

  • SHA256: 66df54ac217bd05991330c37916685367e434ce5ee164c6794444ee00b706fdd
  • Pointer size: 131 Bytes
  • Size of remote file: 204 kB
results/depth_ours.jpg ADDED

Git LFS Details

  • SHA256: 2cb9b57d9d1c74311f698947bbdb76e41f111c404e0fa4be94a5bf98edfac733
  • Pointer size: 131 Bytes
  • Size of remote file: 242 kB
results/hed2_original.jpg ADDED

Git LFS Details

  • SHA256: ac20632d339ef40c08a0b77dc6ba88bac5f107e0958c8505864f080d4526181e
  • Pointer size: 131 Bytes
  • Size of remote file: 156 kB
results/hed2_ours.jpg ADDED

Git LFS Details

  • SHA256: 8c11592a469143dca5eb04ac60558412a945fb47306fc0c90a75cca43bea18c9
  • Pointer size: 131 Bytes
  • Size of remote file: 190 kB
results/hed_original.jpg ADDED

Git LFS Details

  • SHA256: 211efbec445c3d02ef3c3dccb59e138c1ed0f8541e177a8a5c7acb16f01079ae
  • Pointer size: 131 Bytes
  • Size of remote file: 167 kB
results/hed_ours.jpg ADDED

Git LFS Details

  • SHA256: a517c5f2030d968766f0d4cf8baeec908cefbb9958a1dbb2074044b54a924df0
  • Pointer size: 131 Bytes
  • Size of remote file: 195 kB
results/pose2_original.jpg ADDED

Git LFS Details

  • SHA256: edb3b7762e56be8d1c3cc5d27537e8f3756bc44f383f51eef1ff90ad1c9a315e
  • Pointer size: 131 Bytes
  • Size of remote file: 215 kB
results/pose2_ours.jpg ADDED

Git LFS Details

  • SHA256: 86535e649d6785019f1a6a9ac1cd7241266bfe4bd9a422d0d51351bf7078b292
  • Pointer size: 131 Bytes
  • Size of remote file: 281 kB