--- license: apache-2.0 language: - en pipeline_tag: text-to-image library_name: diffusion-single-file base_model: - ChenDY/NitroFusion2 - alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1 tags: - controlnet - text-to-image - one-step - few-step - comfyui - nitrofusion2 --- # NitroFusion2-Controlnet-Union [![Demo](https://img.shields.io/badge/%F0%9F%A4%97%20Demo-NitroFusion2-ff7a00?labelColor=555)](https://huggingface.co/spaces/ChenDY/NitroFusion2) [![NitroFusion2](https://img.shields.io/badge/%F0%9F%A4%97%20Checkpoint-NitroFusion2-ffc107?labelColor=555)](https://huggingface.co/ChenDY/NitroFusion2) [![GitHub](https://img.shields.io/badge/GitHub-NitroFusion2-24292f?logo=github&labelColor=555)](https://github.com/ChenDarYen/NitroFusion2) [![Project Page](https://img.shields.io/badge/%F0%9F%8C%90%20Project%20Page-NitroFusion2-1f72b8?labelColor=555)](https://chendaryen.github.io/NitroFusion2.github.io/) [![arXiv](https://img.shields.io/badge/arXiv-2611.05648-b31b1b?labelColor=555)](https://arxiv.org/abs/2611.05648) NitroFusion2-Controlnet-Union is a finetuned version of [Z-Image-Turbo-Fun-Controlnet-Union-2.1](https://huggingface.co/alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1) (2602), adapted to [NitroFusion2](https://huggingface.co/ChenDY/NitroFusion2) for 1-step and 2-step generation. It must be used with NitroFusion2 as the base model. Supported conditions: Canny, HED, Depth, Pose, Scribble, Gray, and inpainting. ## ComfyUI - 1 step: [`workflows/nitrofusion2_controlnet_1step.json`](./workflows/nitrofusion2_controlnet_1step.json) - 2 steps: [`workflows/nitrofusion2_controlnet_2step.json`](./workflows/nitrofusion2_controlnet_2step.json) The 2-step workflow loads the LoRA with [ComfyUI-BypassLoRA](https://github.com/ChenDarYen/ComfyUI-BypassLoRA). It applies the LoRA without merging it into the weights, which makes 2-step generation 1.5–3.5× faster. The node includes the fix from [ComfyUI PR #17408](https://github.com/Comfy-Org/ComfyUI/pull/17408) until ComfyUI ships it. ## Model Card | Name | Description | |--|--| | NitroFusion2-Controlnet-Union.safetensors | Union 2602 (15 control layer blocks + 2 refiner blocks) adapted to NitroFusion2 for 1–2 step inference. | | NitroFusion2-Controlnet-Union-lite.safetensors | Lite version (3 control layer blocks + 2 refiner blocks) adapted the same way; smaller and lighter on memory, with weaker control. | ## Results ### 1 step
PoseUnion 2.1-2602 + NitroFusion2NitroFusion2-Controlnet-Union
Pose control mapUnion 2.1-2602 + NitroFusion2, 1 stepNitroFusion2-Controlnet-Union, 1 step
PoseUnion 2.1-2602 + NitroFusion2NitroFusion2-Controlnet-Union
Pose control mapUnion 2.1-2602 + NitroFusion2, 1 stepNitroFusion2-Controlnet-Union, 1 step
CannyUnion 2.1-2602 + NitroFusion2NitroFusion2-Controlnet-Union
Canny control mapUnion 2.1-2602 + NitroFusion2, 1 stepNitroFusion2-Controlnet-Union, 1 step
CannyUnion 2.1-2602 + NitroFusion2NitroFusion2-Controlnet-Union
Canny control mapUnion 2.1-2602 + NitroFusion2, 1 stepNitroFusion2-Controlnet-Union, 1 step
HEDUnion 2.1-2602 + NitroFusion2NitroFusion2-Controlnet-Union
HED control mapUnion 2.1-2602 + NitroFusion2, 1 stepNitroFusion2-Controlnet-Union, 1 step
HEDUnion 2.1-2602 + NitroFusion2NitroFusion2-Controlnet-Union
HED control mapUnion 2.1-2602 + NitroFusion2, 1 stepNitroFusion2-Controlnet-Union, 1 step
DepthUnion 2.1-2602 + NitroFusion2NitroFusion2-Controlnet-Union
Depth control mapUnion 2.1-2602 + NitroFusion2, 1 stepNitroFusion2-Controlnet-Union, 1 step
DepthUnion 2.1-2602 + NitroFusion2NitroFusion2-Controlnet-Union
Depth control mapUnion 2.1-2602 + NitroFusion2, 1 stepNitroFusion2-Controlnet-Union, 1 step
Pose + InpaintUnion 2.1-2602 + NitroFusion2NitroFusion2-Controlnet-Union
Source image, masked image and pose mapUnion 2.1-2602 + NitroFusion2, 1 stepNitroFusion2-Controlnet-Union, 1 step
### 2 steps
PoseUnion 2.1-2602 + NitroFusion2NitroFusion2-Controlnet-Union
Pose control mapUnion 2.1-2602 + NitroFusion2, 2 stepsNitroFusion2-Controlnet-Union, 2 steps
PoseUnion 2.1-2602 + NitroFusion2NitroFusion2-Controlnet-Union
Pose control mapUnion 2.1-2602 + NitroFusion2, 2 stepsNitroFusion2-Controlnet-Union, 2 steps
CannyUnion 2.1-2602 + NitroFusion2NitroFusion2-Controlnet-Union
Canny control mapUnion 2.1-2602 + NitroFusion2, 2 stepsNitroFusion2-Controlnet-Union, 2 steps
CannyUnion 2.1-2602 + NitroFusion2NitroFusion2-Controlnet-Union
Canny control mapUnion 2.1-2602 + NitroFusion2, 2 stepsNitroFusion2-Controlnet-Union, 2 steps
HEDUnion 2.1-2602 + NitroFusion2NitroFusion2-Controlnet-Union
HED control mapUnion 2.1-2602 + NitroFusion2, 2 stepsNitroFusion2-Controlnet-Union, 2 steps
HEDUnion 2.1-2602 + NitroFusion2NitroFusion2-Controlnet-Union
HED control mapUnion 2.1-2602 + NitroFusion2, 2 stepsNitroFusion2-Controlnet-Union, 2 steps
DepthUnion 2.1-2602 + NitroFusion2NitroFusion2-Controlnet-Union
Depth control mapUnion 2.1-2602 + NitroFusion2, 2 stepsNitroFusion2-Controlnet-Union, 2 steps
DepthUnion 2.1-2602 + NitroFusion2NitroFusion2-Controlnet-Union
Depth control mapUnion 2.1-2602 + NitroFusion2, 2 stepsNitroFusion2-Controlnet-Union, 2 steps
Pose + InpaintUnion 2.1-2602 + NitroFusion2NitroFusion2-Controlnet-Union
Source image, masked image and pose mapUnion 2.1-2602 + NitroFusion2, 2 stepsNitroFusion2-Controlnet-Union, 2 steps
## Citation If you find NitroFusion2 useful in your research, please cite our paper: ```bib @article{chen2026nitrofusion2, title={NitroFusion2: Bringing Adversarial Loss to the Text Stream of Multimodal Diffusion Transformers}, author={Chen, Dar-Yen and Song, Yi-Zhe}, journal={arXiv preprint arXiv:2611.05648}, year={2026} } ``` ## Acknowledgements This model is adapted from [alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1](https://huggingface.co/alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1). Many thanks to the authors for releasing the ControlNet. ## License [Apache 2.0](./LICENSE)