Instructions to use ChenDY/NitroFusion2-Controlnet-Union with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusion Single File
How to use ChenDY/NitroFusion2-Controlnet-Union with Diffusion Single File:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Download workflows/nitrofusion2_controlnet_2step.json from ChenDY/NitroFusion2-Controlnet-Union: direct link, hf CLI and curl.
- Browser
- Download file 49.1 kB
-
https://huggingface.co/ChenDY/NitroFusion2-Controlnet-Union/resolve/main/workflows/nitrofusion2_controlnet_2step.json
- Command line
-
hf download hf://ChenDY/NitroFusion2-Controlnet-Union/workflows/nitrofusion2_controlnet_2step.json
-
curl -L -o nitrofusion2_controlnet_2step.json https://huggingface.co/ChenDY/NitroFusion2-Controlnet-Union/resolve/main/workflows/nitrofusion2_controlnet_2step.json
49.1 kB
{
"id": "c6d9f3a5-2b8e-4c4a-9f73-0a1e4b7d2f95",
"revision": 0,
"last_node_id": 63,
"last_link_id": 65,
"nodes": [
{
"id": 35,
"type": "MarkdownNote",
"pos": [
-680,
200
],
"size": [
760,
1140
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [],
"title": "Read me",
"properties": {},
"widgets_values": [
"# NitroFusion2 ControlNet\n\n## Model files\n\n**text_encoders**\n- [qwen_3_4b.safetensors](https://huggingface.co/Comfy-Org/z_image_turbo/resolve/main/split_files/text_encoders/qwen_3_4b.safetensors) (7.49 GB)\n\n**diffusion_models**\n- [nf2-bf16.safetensors](https://huggingface.co/ChenDY/NitroFusion2/resolve/main/nf2-bf16.safetensors) (12.3 GB)\n\n**loras**\n- [nf2-dedistill-lora.safetensors](https://huggingface.co/ChenDY/NitroFusion2/resolve/main/nf2-dedistill-lora.safetensors) (381.6 MB)\n\n**model_patches**\n- [NitroFusion2-Controlnet-Union.safetensors](https://huggingface.co/ChenDY/NitroFusion2-Controlnet-Union/resolve/main/NitroFusion2-Controlnet-Union.safetensors) (6.7 GB)\n- [NitroFusion2-Controlnet-Union-lite.safetensors](https://huggingface.co/ChenDY/NitroFusion2-Controlnet-Union/resolve/main/NitroFusion2-Controlnet-Union-lite.safetensors) (2.0 GB, optional)\n\n**vae**\n- [ae.safetensors](https://huggingface.co/Comfy-Org/z_image_turbo/resolve/main/split_files/vae/ae.safetensors) (319.8 MB)\n\n## Folder structure\n\n```\nπ ComfyUI/\nβββ π input/\nβ βββ pose_inpaint_source.png\nβ βββ pose_inpaint_mask.png\nβ βββ pose_inpaint_pose.jpg\nβββ π models/\n βββ π text_encoders/\n β βββ qwen_3_4b.safetensors\n βββ π diffusion_models/\n β βββ nf2-bf16.safetensors\n βββ π loras/\n β βββ nf2-dedistill-lora.safetensors\n βββ π model_patches/\n β βββ NitroFusion2-Controlnet-Union.safetensors\n β βββ NitroFusion2-Controlnet-Union-lite.safetensors\n βββ π vae/\n βββ ae.safetensors\n```\n\n## How to use\n\n1. Load your control map (canny, HED, depth, pose, scribble or gray) in **Load Image (pose map)**. The example map is [asset/pose.jpg](https://huggingface.co/ChenDY/NitroFusion2-Controlnet-Union/resolve/main/asset/pose.jpg).\n2. Set **width** and **height** to the control map's aspect ratio (about 1 megapixel, multiples of 16).\n3. **control strength**: 1.0 is recommended.\n4. **controlnet**: NitroFusion2-Controlnet-Union.safetensors, or the lite file for lower memory with weaker control.\n\n## Inpainting\n\n- **Load Image (source to inpaint)**: the image to edit (pose_inpaint_source.png).\n- **Load Mask (white = repaint)**: white marks the area to repaint, black is kept; the red channel is read (pose_inpaint_mask.png).\n- **Load Image (pose map)**: the pose control for the repainted area (pose_inpaint_pose.jpg).\n- Set width and height to the source image's aspect ratio. Download all three from the asset/ folder of this repo and put them in ComfyUI/input/.\n\n**Without inpainting** (plain ControlNet, any condition): select **Load Image (source to inpaint)** and **Load Mask (white = repaint)** and bypass them (Ctrl+B), then load your control map in **Load Image (pose map)**.\n\n## Steps\n\nTwo steps on one shared sigma table (euler, simple). The shift (6) is preset. The step LoRA is applied on step 1 only (scale 0.5); step 2 runs NitroFusion2 alone. Keep cfg = 1.\n\nThe LoRA is loaded with **Load LoRA (Bypass, PR #17408, Model Only)** from [ComfyUI-BypassLoRA](https://github.com/ChenDarYen/ComfyUI-BypassLoRA), which applies it without merging it into the weights, so changing the scale between steps costs nothing (2-step generation is 1.5β3.5Γ faster). Install it with `git clone https://github.com/ChenDarYen/ComfyUI-BypassLoRA` in `ComfyUI/custom_nodes`, or with Install via Git URL in ComfyUI-Manager.\n\n## Report workflow issue\n\nIf you found any issues when running this workflow, [report them here](https://huggingface.co/ChenDY/NitroFusion2-Controlnet-Union/discussions).\n"
],
"color": "#222",
"bgcolor": "#000"
},
{
"id": 57,
"type": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2a1b44",
"pos": [
130,
200
],
"size": [
400,
560
],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{
"label": "prompt",
"name": "text",
"type": "STRING",
"widget": {
"name": "text"
},
"link": null
},
{
"label": "control image",
"name": "image",
"type": "IMAGE",
"link": 63
},
{
"label": "inpaint image",
"name": "inpaint_image",
"type": "IMAGE",
"link": 64
},
{
"label": "inpaint mask",
"name": "mask",
"type": "MASK",
"link": 65
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
62
]
}
],
"properties": {
"proxyWidgets": [
[
"6",
"text"
],
[
"9",
"width"
],
[
"9",
"height"
],
[
"10",
"noise_seed"
],
[
"1",
"unet_name"
],
[
"2",
"strength_model"
],
[
"14",
"strength_model"
],
[
"2",
"lora_name"
],
[
"5",
"clip_name"
],
[
"8",
"vae_name"
],
[
"20",
"name"
],
[
"21",
"strength"
],
[
"10",
"control_after_generate"
]
],
"cnr_id": "comfy-core"
},
"widgets_values": []
},
{
"id": 60,
"type": "SaveImage",
"pos": [
570,
200
],
"size": [
780,
660
],
"flags": {},
"order": 2,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 62
}
],
"outputs": [],
"properties": {
"Node name for S&R": "SaveImage",
"cnr_id": "comfy-core"
},
"widgets_values": [
"NitroFusion2"
]
},
{
"id": 61,
"type": "LoadImage",
"pos": [
130,
800
],
"size": [
380,
420
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"localized_name": "IMAGE",
"name": "IMAGE",
"type": "IMAGE",
"links": [
63
]
},
{
"localized_name": "MASK",
"name": "MASK",
"type": "MASK",
"links": null
}
],
"properties": {
"Node name for S&R": "LoadImage",
"cnr_id": "comfy-core"
},
"widgets_values": [
"pose_inpaint_pose.jpg",
"image"
],
"title": "Load Image (pose map)"
},
{
"id": 62,
"type": "LoadImage",
"pos": [
130,
1260
],
"size": [
380,
420
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"localized_name": "IMAGE",
"name": "IMAGE",
"type": "IMAGE",
"links": [
64
]
},
{
"localized_name": "MASK",
"name": "MASK",
"type": "MASK",
"links": null
}
],
"properties": {
"Node name for S&R": "LoadImage",
"cnr_id": "comfy-core"
},
"widgets_values": [
"pose_inpaint_source.png",
"image"
],
"title": "Load Image (source to inpaint)"
},
{
"id": 63,
"type": "LoadImageMask",
"pos": [
130,
1720
],
"size": [
380,
440
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [
{
"localized_name": "image",
"name": "image",
"type": "COMBO",
"widget": {
"name": "image"
},
"link": null
}
],
"outputs": [
{
"localized_name": "MASK",
"name": "MASK",
"type": "MASK",
"links": [
65
]
}
],
"properties": {
"Node name for S&R": "LoadImageMask",
"cnr_id": "comfy-core"
},
"widgets_values": [
"pose_inpaint_mask.png",
"red",
"image"
],
"title": "Load Mask (white = repaint)"
}
],
"links": [
[
62,
57,
0,
60,
0,
"IMAGE"
],
[
63,
61,
0,
57,
1,
"IMAGE"
],
[
64,
62,
0,
57,
2,
"IMAGE"
],
[
65,
63,
0,
57,
3,
"MASK"
]
],
"groups": [],
"definitions": {
"subgraphs": [
{
"id": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2a1b44",
"version": 1,
"state": {
"lastGroupId": 3,
"lastNodeId": 40,
"lastLinkId": 144,
"lastRerouteId": 0
},
"revision": 0,
"config": {},
"name": "Control to Image (NitroFusion2 ControlNet, 2 steps)",
"inputNode": {
"id": -10,
"bounding": [
-560,
480,
120,
360
]
},
"outputNode": {
"id": -20,
"bounding": [
1985,
320,
120,
60
]
},
"inputs": [
{
"id": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2a0000",
"name": "text",
"type": "STRING",
"linkIds": [
117
],
"pos": [
-460,
500
],
"label": "prompt"
},
{
"id": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2a0001",
"name": "width",
"type": "INT",
"linkIds": [
118
],
"pos": [
-460,
520
]
},
{
"id": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2a0002",
"name": "height",
"type": "INT",
"linkIds": [
119
],
"pos": [
-460,
540
]
},
{
"id": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2a0003",
"name": "seed",
"type": "INT",
"linkIds": [
120,
121
],
"pos": [
-460,
560
]
},
{
"id": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2a0004",
"name": "unet_name",
"type": "COMBO",
"linkIds": [
122
],
"pos": [
-460,
580
]
},
{
"id": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2a0005",
"name": "lora_scale_1",
"type": "FLOAT",
"linkIds": [
123
],
"pos": [
-460,
600
],
"label": "LoRA scale Β· step 1"
},
{
"id": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2a0006",
"name": "lora_scale_2",
"type": "FLOAT",
"linkIds": [
124
],
"pos": [
-460,
620
],
"label": "LoRA scale Β· step 2"
},
{
"id": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2a0007",
"name": "lora_name",
"type": "COMBO",
"linkIds": [
125,
126
],
"pos": [
-460,
640
]
},
{
"id": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2a0008",
"name": "clip_name",
"type": "COMBO",
"linkIds": [
127
],
"pos": [
-460,
660
]
},
{
"id": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2a0009",
"name": "vae_name",
"type": "COMBO",
"linkIds": [
128
],
"pos": [
-460,
680
]
},
{
"id": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2a0010",
"name": "image",
"type": "IMAGE",
"linkIds": [
136,
137
],
"pos": [
-460,
700
],
"label": "control image"
},
{
"id": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2a0011",
"name": "name",
"type": "COMBO",
"linkIds": [
138
],
"pos": [
-460,
720
],
"label": "controlnet"
},
{
"id": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2a0012",
"name": "strength",
"type": "FLOAT",
"linkIds": [
139,
140
],
"pos": [
-460,
740
],
"label": "control strength"
},
{
"id": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2a0013",
"name": "inpaint_image",
"type": "IMAGE",
"linkIds": [
141,
142
],
"pos": [
-460,
760
],
"label": "inpaint image"
},
{
"id": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2a0014",
"name": "mask",
"type": "MASK",
"linkIds": [
143,
144
],
"pos": [
-460,
780
],
"label": "inpaint mask"
}
],
"outputs": [
{
"id": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2affff",
"name": "IMAGE",
"type": "IMAGE",
"linkIds": [
129
],
"localized_name": "IMAGE",
"pos": [
0,
0
]
}
],
"widgets": [],
"nodes": [
{
"id": 1,
"type": "UNETLoader",
"pos": [
30,
230
],
"size": [
270,
110
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [
{
"localized_name": "unet_name",
"name": "unet_name",
"type": "COMBO",
"widget": {
"name": "unet_name"
},
"link": 122
}
],
"outputs": [
{
"localized_name": "MODEL",
"name": "MODEL",
"type": "MODEL",
"links": [
101,
102
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "UNETLoader",
"cnr_id": "comfy-core"
},
"widgets_values": [
"nf2-bf16.safetensors",
"default"
]
},
{
"id": 2,
"type": "LoraLoaderBypassPR17408ModelOnly",
"pos": [
30,
370
],
"size": [
270,
110
],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{
"localized_name": "model",
"name": "model",
"type": "MODEL",
"link": 101
},
{
"localized_name": "strength_model",
"name": "strength_model",
"type": "FLOAT",
"widget": {
"name": "strength_model"
},
"link": 123
},
{
"localized_name": "lora_name",
"name": "lora_name",
"type": "COMBO",
"widget": {
"name": "lora_name"
},
"link": 125
}
],
"outputs": [
{
"localized_name": "MODEL",
"name": "MODEL",
"type": "MODEL",
"links": [
104
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "LoraLoaderBypassPR17408ModelOnly",
"aux_id": "ChenDarYen/ComfyUI-BypassLoRA"
},
"widgets_values": [
"nf2-dedistill-lora.safetensors",
0.5
],
"title": "De-distillation LoRA Β· step 1"
},
{
"id": 14,
"type": "LoraLoaderBypassPR17408ModelOnly",
"pos": [
30,
510
],
"size": [
270,
110
],
"flags": {},
"order": 2,
"mode": 0,
"inputs": [
{
"localized_name": "model",
"name": "model",
"type": "MODEL",
"link": 102
},
{
"localized_name": "strength_model",
"name": "strength_model",
"type": "FLOAT",
"widget": {
"name": "strength_model"
},
"link": 124
},
{
"localized_name": "lora_name",
"name": "lora_name",
"type": "COMBO",
"widget": {
"name": "lora_name"
},
"link": 126
}
],
"outputs": [
{
"localized_name": "MODEL",
"name": "MODEL",
"type": "MODEL",
"links": [
103
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "LoraLoaderBypassPR17408ModelOnly",
"aux_id": "ChenDarYen/ComfyUI-BypassLoRA"
},
"widgets_values": [
"nf2-dedistill-lora.safetensors",
0.0
],
"title": "De-distillation LoRA Β· step 2"
},
{
"id": 3,
"type": "ModelSamplingAuraFlow",
"pos": [
1310,
230
],
"size": [
310,
104
],
"flags": {},
"order": 3,
"mode": 0,
"inputs": [
{
"localized_name": "model",
"name": "model",
"type": "MODEL",
"link": 133
}
],
"outputs": [
{
"localized_name": "MODEL",
"name": "MODEL",
"type": "MODEL",
"links": [
111
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "ModelSamplingAuraFlow",
"cnr_id": "comfy-core"
},
"widgets_values": [
6
]
},
{
"id": 4,
"type": "ModelSamplingAuraFlow",
"pos": [
950,
230
],
"size": [
310,
104
],
"flags": {},
"order": 4,
"mode": 0,
"inputs": [
{
"localized_name": "model",
"name": "model",
"type": "MODEL",
"link": 130
}
],
"outputs": [
{
"localized_name": "MODEL",
"name": "MODEL",
"type": "MODEL",
"links": [
107
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "ModelSamplingAuraFlow",
"cnr_id": "comfy-core"
},
"widgets_values": [
6
]
},
{
"id": 5,
"type": "CLIPLoader",
"pos": [
30,
650
],
"size": [
270,
140
],
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"localized_name": "clip_name",
"name": "clip_name",
"type": "COMBO",
"widget": {
"name": "clip_name"
},
"link": 127
}
],
"outputs": [
{
"localized_name": "CLIP",
"name": "CLIP",
"type": "CLIP",
"links": [
105
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "CLIPLoader",
"cnr_id": "comfy-core"
},
"widgets_values": [
"qwen_3_4b.safetensors",
"lumina2",
"default"
]
},
{
"id": 6,
"type": "CLIPTextEncode",
"pos": [
400,
230
],
"size": [
450,
650
],
"flags": {},
"order": 6,
"mode": 0,
"inputs": [
{
"localized_name": "clip",
"name": "clip",
"type": "CLIP",
"link": 105
},
{
"localized_name": "text",
"name": "text",
"type": "STRING",
"widget": {
"name": "text"
},
"link": 117
}
],
"outputs": [
{
"localized_name": "CONDITIONING",
"name": "CONDITIONING",
"type": "CONDITIONING",
"links": [
106,
108,
112
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "CLIPTextEncode",
"cnr_id": "comfy-core"
},
"widgets_values": [
"A young woman with long wavy violet-purple hair and straight bangs, two small black ribbon bows in her hair, smiling softly at the camera. She wears a white sleeveless summer dress with thin straps, a ruffled sweetheart neckline and a light, flowing chiffon skirt."
]
},
{
"id": 7,
"type": "ConditioningZeroOut",
"pos": [
630,
960
],
"size": [
225,
72
],
"flags": {},
"order": 7,
"mode": 0,
"inputs": [
{
"localized_name": "conditioning",
"name": "conditioning",
"type": "CONDITIONING",
"link": 106
}
],
"outputs": [
{
"localized_name": "CONDITIONING",
"name": "CONDITIONING",
"type": "CONDITIONING",
"links": [
109,
113
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "ConditioningZeroOut",
"cnr_id": "comfy-core"
},
"widgets_values": []
},
{
"id": 8,
"type": "VAELoader",
"pos": [
30,
830
],
"size": [
270,
110
],
"flags": {},
"order": 8,
"mode": 0,
"inputs": [
{
"localized_name": "vae_name",
"name": "vae_name",
"type": "COMBO",
"widget": {
"name": "vae_name"
},
"link": 128
}
],
"outputs": [
{
"localized_name": "VAE",
"name": "VAE",
"type": "VAE",
"links": [
116,
132,
135
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "VAELoader",
"cnr_id": "comfy-core"
},
"widgets_values": [
"ae.safetensors"
]
},
{
"id": 9,
"type": "EmptySD3LatentImage",
"pos": [
40,
1050
],
"size": [
260,
170
],
"flags": {},
"order": 9,
"mode": 0,
"inputs": [
{
"localized_name": "width",
"name": "width",
"type": "INT",
"widget": {
"name": "width"
},
"link": 118
},
{
"localized_name": "height",
"name": "height",
"type": "INT",
"widget": {
"name": "height"
},
"link": 119
}
],
"outputs": [
{
"localized_name": "LATENT",
"name": "LATENT",
"type": "LATENT",
"links": [
110
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "EmptySD3LatentImage",
"cnr_id": "comfy-core"
},
"widgets_values": [
768,
1328,
1
]
},
{
"id": 10,
"type": "KSamplerAdvanced",
"pos": [
950,
400
],
"size": [
320,
400
],
"flags": {},
"order": 10,
"mode": 0,
"inputs": [
{
"localized_name": "model",
"name": "model",
"type": "MODEL",
"link": 107
},
{
"localized_name": "positive",
"name": "positive",
"type": "CONDITIONING",
"link": 108
},
{
"localized_name": "negative",
"name": "negative",
"type": "CONDITIONING",
"link": 109
},
{
"localized_name": "latent_image",
"name": "latent_image",
"type": "LATENT",
"link": 110
},
{
"localized_name": "noise_seed",
"name": "noise_seed",
"type": "INT",
"widget": {
"name": "noise_seed"
},
"link": 120
}
],
"outputs": [
{
"localized_name": "LATENT",
"name": "LATENT",
"type": "LATENT",
"links": [
114
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "KSamplerAdvanced",
"cnr_id": "comfy-core"
},
"widgets_values": [
"enable",
3,
"fixed",
2,
1,
"euler",
"simple",
0,
1,
"enable"
],
"title": "Step 1 Β· NitroFusion2 + de-distillation LoRA"
},
{
"id": 11,
"type": "KSamplerAdvanced",
"pos": [
1310,
400
],
"size": [
320,
400
],
"flags": {},
"order": 11,
"mode": 0,
"inputs": [
{
"localized_name": "model",
"name": "model",
"type": "MODEL",
"link": 111
},
{
"localized_name": "positive",
"name": "positive",
"type": "CONDITIONING",
"link": 112
},
{
"localized_name": "negative",
"name": "negative",
"type": "CONDITIONING",
"link": 113
},
{
"localized_name": "latent_image",
"name": "latent_image",
"type": "LATENT",
"link": 114
},
{
"localized_name": "noise_seed",
"name": "noise_seed",
"type": "INT",
"widget": {
"name": "noise_seed"
},
"link": 121
}
],
"outputs": [
{
"localized_name": "LATENT",
"name": "LATENT",
"type": "LATENT",
"links": [
115
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "KSamplerAdvanced",
"cnr_id": "comfy-core"
},
"widgets_values": [
"disable",
3,
"fixed",
2,
1,
"euler",
"simple",
1,
2,
"disable"
],
"title": "Step 2 Β· NitroFusion2"
},
{
"id": 12,
"type": "VAEDecode",
"pos": [
1680,
230
],
"size": [
225,
96
],
"flags": {},
"order": 12,
"mode": 0,
"inputs": [
{
"localized_name": "samples",
"name": "samples",
"type": "LATENT",
"link": 115
},
{
"localized_name": "vae",
"name": "vae",
"type": "VAE",
"link": 116
}
],
"outputs": [
{
"localized_name": "IMAGE",
"name": "IMAGE",
"type": "IMAGE",
"links": [
129
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "VAEDecode",
"cnr_id": "comfy-core"
},
"widgets_values": []
},
{
"id": 20,
"type": "ModelPatchLoader",
"pos": [
30,
1330
],
"size": [
270,
82
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [
{
"localized_name": "name",
"name": "name",
"type": "COMBO",
"widget": {
"name": "name"
},
"link": 138
}
],
"outputs": [
{
"localized_name": "MODEL_PATCH",
"name": "MODEL_PATCH",
"type": "MODEL_PATCH",
"links": [
131,
134
]
}
],
"properties": {
"Node name for S&R": "ModelPatchLoader",
"cnr_id": "comfy-core"
},
"widgets_values": [
"NitroFusion2-Controlnet-Union.safetensors"
]
},
{
"id": 21,
"type": "ZImageFunControlnet",
"pos": [
950,
860
],
"size": [
320,
190
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [
{
"localized_name": "model",
"name": "model",
"type": "MODEL",
"link": 104
},
{
"localized_name": "model_patch",
"name": "model_patch",
"type": "MODEL_PATCH",
"link": 131
},
{
"localized_name": "vae",
"name": "vae",
"type": "VAE",
"link": 132
},
{
"localized_name": "image",
"name": "image",
"shape": 7,
"type": "IMAGE",
"link": 136
},
{
"localized_name": "inpaint_image",
"name": "inpaint_image",
"shape": 7,
"type": "IMAGE",
"link": 141
},
{
"localized_name": "mask",
"name": "mask",
"shape": 7,
"type": "MASK",
"link": 143
},
{
"localized_name": "strength",
"name": "strength",
"type": "FLOAT",
"widget": {
"name": "strength"
},
"link": 139
}
],
"outputs": [
{
"localized_name": "MODEL",
"name": "MODEL",
"type": "MODEL",
"links": [
130
]
}
],
"properties": {
"Node name for S&R": "ZImageFunControlnet",
"cnr_id": "comfy-core"
},
"widgets_values": [
1.0,
0.0,
1.0
]
},
{
"id": 22,
"type": "ZImageFunControlnet",
"pos": [
1310,
860
],
"size": [
320,
190
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [
{
"localized_name": "model",
"name": "model",
"type": "MODEL",
"link": 103
},
{
"localized_name": "model_patch",
"name": "model_patch",
"type": "MODEL_PATCH",
"link": 134
},
{
"localized_name": "vae",
"name": "vae",
"type": "VAE",
"link": 135
},
{
"localized_name": "image",
"name": "image",
"shape": 7,
"type": "IMAGE",
"link": 137
},
{
"localized_name": "inpaint_image",
"name": "inpaint_image",
"shape": 7,
"type": "IMAGE",
"link": 142
},
{
"localized_name": "mask",
"name": "mask",
"shape": 7,
"type": "MASK",
"link": 144
},
{
"localized_name": "strength",
"name": "strength",
"type": "FLOAT",
"widget": {
"name": "strength"
},
"link": 140
}
],
"outputs": [
{
"localized_name": "MODEL",
"name": "MODEL",
"type": "MODEL",
"links": [
133
]
}
],
"properties": {
"Node name for S&R": "ZImageFunControlnet",
"cnr_id": "comfy-core"
},
"widgets_values": [
1.0,
0.0,
1.0
]
}
],
"groups": [
{
"id": 1,
"title": "Step2 - Image size",
"bounding": [
10,
980,
320,
280
],
"color": "#3f789e",
"font_size": 24,
"flags": {}
},
{
"id": 2,
"title": "Step3 - Prompt",
"bounding": [
360,
130,
530,
1130
],
"color": "#3f789e",
"font_size": 24,
"flags": {}
},
{
"id": 3,
"title": "Step1 - Load models",
"bounding": [
0,
130,
330,
830
],
"color": "#3f789e",
"font_size": 24,
"flags": {}
},
{
"id": 4,
"title": "Step4 - ControlNet",
"bounding": [
0,
1280,
330,
150
],
"color": "#3f789e",
"font_size": 24,
"flags": {}
},
{
"id": 5,
"title": "ControlNet",
"bounding": [
940,
790,
730,
280
],
"color": "#3f789e",
"font_size": 24,
"flags": {}
}
],
"links": [
{
"id": 101,
"origin_id": 1,
"origin_slot": 0,
"target_id": 2,
"target_slot": 0,
"type": "MODEL"
},
{
"id": 102,
"origin_id": 1,
"origin_slot": 0,
"target_id": 14,
"target_slot": 0,
"type": "MODEL"
},
{
"id": 103,
"origin_id": 14,
"origin_slot": 0,
"target_id": 22,
"target_slot": 0,
"type": "MODEL"
},
{
"id": 104,
"origin_id": 2,
"origin_slot": 0,
"target_id": 21,
"target_slot": 0,
"type": "MODEL"
},
{
"id": 105,
"origin_id": 5,
"origin_slot": 0,
"target_id": 6,
"target_slot": 0,
"type": "CLIP"
},
{
"id": 106,
"origin_id": 6,
"origin_slot": 0,
"target_id": 7,
"target_slot": 0,
"type": "CONDITIONING"
},
{
"id": 107,
"origin_id": 4,
"origin_slot": 0,
"target_id": 10,
"target_slot": 0,
"type": "MODEL"
},
{
"id": 108,
"origin_id": 6,
"origin_slot": 0,
"target_id": 10,
"target_slot": 1,
"type": "CONDITIONING"
},
{
"id": 109,
"origin_id": 7,
"origin_slot": 0,
"target_id": 10,
"target_slot": 2,
"type": "CONDITIONING"
},
{
"id": 110,
"origin_id": 9,
"origin_slot": 0,
"target_id": 10,
"target_slot": 3,
"type": "LATENT"
},
{
"id": 111,
"origin_id": 3,
"origin_slot": 0,
"target_id": 11,
"target_slot": 0,
"type": "MODEL"
},
{
"id": 112,
"origin_id": 6,
"origin_slot": 0,
"target_id": 11,
"target_slot": 1,
"type": "CONDITIONING"
},
{
"id": 113,
"origin_id": 7,
"origin_slot": 0,
"target_id": 11,
"target_slot": 2,
"type": "CONDITIONING"
},
{
"id": 114,
"origin_id": 10,
"origin_slot": 0,
"target_id": 11,
"target_slot": 3,
"type": "LATENT"
},
{
"id": 115,
"origin_id": 11,
"origin_slot": 0,
"target_id": 12,
"target_slot": 0,
"type": "LATENT"
},
{
"id": 116,
"origin_id": 8,
"origin_slot": 0,
"target_id": 12,
"target_slot": 1,
"type": "VAE"
},
{
"id": 117,
"origin_id": -10,
"origin_slot": 0,
"target_id": 6,
"target_slot": 1,
"type": "STRING"
},
{
"id": 118,
"origin_id": -10,
"origin_slot": 1,
"target_id": 9,
"target_slot": 0,
"type": "INT"
},
{
"id": 119,
"origin_id": -10,
"origin_slot": 2,
"target_id": 9,
"target_slot": 1,
"type": "INT"
},
{
"id": 120,
"origin_id": -10,
"origin_slot": 3,
"target_id": 10,
"target_slot": 4,
"type": "INT"
},
{
"id": 121,
"origin_id": -10,
"origin_slot": 3,
"target_id": 11,
"target_slot": 4,
"type": "INT"
},
{
"id": 122,
"origin_id": -10,
"origin_slot": 4,
"target_id": 1,
"target_slot": 0,
"type": "COMBO"
},
{
"id": 123,
"origin_id": -10,
"origin_slot": 5,
"target_id": 2,
"target_slot": 1,
"type": "FLOAT"
},
{
"id": 124,
"origin_id": -10,
"origin_slot": 6,
"target_id": 14,
"target_slot": 1,
"type": "FLOAT"
},
{
"id": 125,
"origin_id": -10,
"origin_slot": 7,
"target_id": 2,
"target_slot": 2,
"type": "COMBO"
},
{
"id": 126,
"origin_id": -10,
"origin_slot": 7,
"target_id": 14,
"target_slot": 2,
"type": "COMBO"
},
{
"id": 127,
"origin_id": -10,
"origin_slot": 8,
"target_id": 5,
"target_slot": 0,
"type": "COMBO"
},
{
"id": 128,
"origin_id": -10,
"origin_slot": 9,
"target_id": 8,
"target_slot": 0,
"type": "COMBO"
},
{
"id": 129,
"origin_id": 12,
"origin_slot": 0,
"target_id": -20,
"target_slot": 0,
"type": "IMAGE"
},
{
"id": 130,
"origin_id": 21,
"origin_slot": 0,
"target_id": 4,
"target_slot": 0,
"type": "MODEL"
},
{
"id": 131,
"origin_id": 20,
"origin_slot": 0,
"target_id": 21,
"target_slot": 1,
"type": "MODEL_PATCH"
},
{
"id": 132,
"origin_id": 8,
"origin_slot": 0,
"target_id": 21,
"target_slot": 2,
"type": "VAE"
},
{
"id": 133,
"origin_id": 22,
"origin_slot": 0,
"target_id": 3,
"target_slot": 0,
"type": "MODEL"
},
{
"id": 134,
"origin_id": 20,
"origin_slot": 0,
"target_id": 22,
"target_slot": 1,
"type": "MODEL_PATCH"
},
{
"id": 135,
"origin_id": 8,
"origin_slot": 0,
"target_id": 22,
"target_slot": 2,
"type": "VAE"
},
{
"id": 136,
"origin_id": -10,
"origin_slot": 10,
"target_id": 21,
"target_slot": 3,
"type": "IMAGE"
},
{
"id": 137,
"origin_id": -10,
"origin_slot": 10,
"target_id": 22,
"target_slot": 3,
"type": "IMAGE"
},
{
"id": 138,
"origin_id": -10,
"origin_slot": 11,
"target_id": 20,
"target_slot": 0,
"type": "COMBO"
},
{
"id": 139,
"origin_id": -10,
"origin_slot": 12,
"target_id": 21,
"target_slot": 6,
"type": "FLOAT"
},
{
"id": 140,
"origin_id": -10,
"origin_slot": 12,
"target_id": 22,
"target_slot": 6,
"type": "FLOAT"
},
{
"id": 141,
"origin_id": -10,
"origin_slot": 13,
"target_id": 21,
"target_slot": 4,
"type": "IMAGE"
},
{
"id": 142,
"origin_id": -10,
"origin_slot": 13,
"target_id": 22,
"target_slot": 4,
"type": "IMAGE"
},
{
"id": 143,
"origin_id": -10,
"origin_slot": 14,
"target_id": 21,
"target_slot": 5,
"type": "MASK"
},
{
"id": 144,
"origin_id": -10,
"origin_slot": 14,
"target_id": 22,
"target_slot": 5,
"type": "MASK"
}
],
"extra": {
"workflowRendererVersion": "LG"
}
}
]
},
"config": {},
"extra": {
"ds": {
"scale": 0.67,
"offset": [
840,
310
]
},
"workflowRendererVersion": "LG"
},
"version": 0.4
}
{
"id": "c6d9f3a5-2b8e-4c4a-9f73-0a1e4b7d2f95",
"revision": 0,
"last_node_id": 63,
"last_link_id": 65,
"nodes": [
{
"id": 35,
"type": "MarkdownNote",
"pos": [
-680,
200
],
"size": [
760,
1140
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [],
"title": "Read me",
"properties": {},
"widgets_values": [
"# NitroFusion2 ControlNet\n\n## Model files\n\n**text_encoders**\n- [qwen_3_4b.safetensors](https://huggingface.co/Comfy-Org/z_image_turbo/resolve/main/split_files/text_encoders/qwen_3_4b.safetensors) (7.49 GB)\n\n**diffusion_models**\n- [nf2-bf16.safetensors](https://huggingface.co/ChenDY/NitroFusion2/resolve/main/nf2-bf16.safetensors) (12.3 GB)\n\n**loras**\n- [nf2-dedistill-lora.safetensors](https://huggingface.co/ChenDY/NitroFusion2/resolve/main/nf2-dedistill-lora.safetensors) (381.6 MB)\n\n**model_patches**\n- [NitroFusion2-Controlnet-Union.safetensors](https://huggingface.co/ChenDY/NitroFusion2-Controlnet-Union/resolve/main/NitroFusion2-Controlnet-Union.safetensors) (6.7 GB)\n- [NitroFusion2-Controlnet-Union-lite.safetensors](https://huggingface.co/ChenDY/NitroFusion2-Controlnet-Union/resolve/main/NitroFusion2-Controlnet-Union-lite.safetensors) (2.0 GB, optional)\n\n**vae**\n- [ae.safetensors](https://huggingface.co/Comfy-Org/z_image_turbo/resolve/main/split_files/vae/ae.safetensors) (319.8 MB)\n\n## Folder structure\n\n```\nπ ComfyUI/\nβββ π input/\nβ βββ pose_inpaint_source.png\nβ βββ pose_inpaint_mask.png\nβ βββ pose_inpaint_pose.jpg\nβββ π models/\n βββ π text_encoders/\n β βββ qwen_3_4b.safetensors\n βββ π diffusion_models/\n β βββ nf2-bf16.safetensors\n βββ π loras/\n β βββ nf2-dedistill-lora.safetensors\n βββ π model_patches/\n β βββ NitroFusion2-Controlnet-Union.safetensors\n β βββ NitroFusion2-Controlnet-Union-lite.safetensors\n βββ π vae/\n βββ ae.safetensors\n```\n\n## How to use\n\n1. Load your control map (canny, HED, depth, pose, scribble or gray) in **Load Image (pose map)**. The example map is [asset/pose.jpg](https://huggingface.co/ChenDY/NitroFusion2-Controlnet-Union/resolve/main/asset/pose.jpg).\n2. Set **width** and **height** to the control map's aspect ratio (about 1 megapixel, multiples of 16).\n3. **control strength**: 1.0 is recommended.\n4. **controlnet**: NitroFusion2-Controlnet-Union.safetensors, or the lite file for lower memory with weaker control.\n\n## Inpainting\n\n- **Load Image (source to inpaint)**: the image to edit (pose_inpaint_source.png).\n- **Load Mask (white = repaint)**: white marks the area to repaint, black is kept; the red channel is read (pose_inpaint_mask.png).\n- **Load Image (pose map)**: the pose control for the repainted area (pose_inpaint_pose.jpg).\n- Set width and height to the source image's aspect ratio. Download all three from the asset/ folder of this repo and put them in ComfyUI/input/.\n\n**Without inpainting** (plain ControlNet, any condition): select **Load Image (source to inpaint)** and **Load Mask (white = repaint)** and bypass them (Ctrl+B), then load your control map in **Load Image (pose map)**.\n\n## Steps\n\nTwo steps on one shared sigma table (euler, simple). The shift (6) is preset. The step LoRA is applied on step 1 only (scale 0.5); step 2 runs NitroFusion2 alone. Keep cfg = 1.\n\nThe LoRA is loaded with **Load LoRA (Bypass, PR #17408, Model Only)** from [ComfyUI-BypassLoRA](https://github.com/ChenDarYen/ComfyUI-BypassLoRA), which applies it without merging it into the weights, so changing the scale between steps costs nothing (2-step generation is 1.5β3.5Γ faster). Install it with `git clone https://github.com/ChenDarYen/ComfyUI-BypassLoRA` in `ComfyUI/custom_nodes`, or with Install via Git URL in ComfyUI-Manager.\n\n## Report workflow issue\n\nIf you found any issues when running this workflow, [report them here](https://huggingface.co/ChenDY/NitroFusion2-Controlnet-Union/discussions).\n"
],
"color": "#222",
"bgcolor": "#000"
},
{
"id": 57,
"type": "9a1f6d38-2e4c-4b7a-b3d5-7c0e8f2a1b44",
"pos": [
130,
200
],
"size": [
400,
560
],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{
"label": "prompt",
"name": "text",
"type": "STRING",
"widget": {
"name": "text"
},
"link": null
},
{
"label": "control image",
"name": "image",
"type": "IMAGE",
"link": 63
},
{
"label": "inpaint image",
"name": "inpaint_image",
"type": "IMAGE",
"link": 64
},
{
"label": "inpaint mask",
"name": "mask",
"type": "MASK",
"link": 65
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
62
]
}
],
"properties": {
"proxyWidgets": [
[
"6",
"text"
],
[
"9",
"width"
],
[
"9",
"height"
],
[
"10",
"noise_seed"
],