ussoewwin commited on
Commit
0876629
Β·
verified Β·
1 Parent(s): 80ddf5e

docs: restore the previous card

Browse files
Files changed (1) hide show
  1. README.md +67 -12
README.md CHANGED
@@ -34,7 +34,38 @@ High-fidelity **ConvRot INT8** quantized weights for multi-condition and dedicat
34
  - **Technical Details & Quantization Framework:** [Hybrid-Sensitivity-Weighted-Quantization (HSWQ)](https://github.com/ussoewwin/Hybrid-Sensitivity-Weighted-Quantization)
35
  - **ComfyUI Loader for ConvRot INT8:** [ComfyUI-HSWQ-Loader-and-Tools](https://github.com/ussoewwin/ComfyUI-HSWQ-Loader-and-Tools)
36
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
  ## πŸ“¦ Available Models
 
38
  Sizes are decimal (1 GB = 10\^9 bytes) and exact; the `SHA256` column is the pack's own hash, so a download can be verified against it.
39
 
40
  | Filename | Base Architecture | Base Model / Author | Supported Conditions | Quantization | File Size | SHA256 | License |
@@ -51,7 +82,41 @@ Sizes are decimal (1 GB = 10\^9 bytes) and exact; the `SHA256` column is the pac
51
  | `Z-Image-Turbo-Fun-Controlnet-Union-2.1-lite-2602-8steps_hswq_convrot_int8.safetensors` | Z-Image-Turbo | [alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1](https://huggingface.co/alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1) | Multi-condition (Canny, Depth, Pose, Color, Scribble, Inpaint) | ConvRot INT8 | 1009.7 MB | `1dd12b2d028b12df6b2a4f93856f8ab161c1edc05125e3b1c987a1694e99dba4` | Apache-2.0 |
52
  | `Z-Image-Turbo-Fun-Controlnet-Tile-2.1-lite-2601-8steps_hswq_convrot_int8.safetensors` | Z-Image-Turbo | [alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1](https://huggingface.co/alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1) | Tile, Super-Resolution, Detail Enhancing | ConvRot INT8 | 1009.7 MB | `2998f83ef65903b4b6b22fb7ac270ed66f9a2443220563aa43db4188024763a8` | Apache-2.0 |
53
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
54
  ## πŸš€ Usage in ComfyUI
 
55
  Install the loader extension:
56
 
57
  ```bash
@@ -64,6 +129,8 @@ Then place each file in the directory its type requires (see the table at the to
64
  - **ControlNet** files go in `ComfyUI/models/controlnet/` and load with `ControlNetLoader`.
65
  - **Model patch** files go in `ComfyUI/models/model_patches/` and load with **`HSWQ Model Patch Loader`** (the stock `ModelPatchLoader` reads the INT8 tensors as plain weights), then apply with `ZImageFunControlnet` / `QwenImageDiffsynthControlnet`.
66
 
 
 
67
  ## πŸ“œ Credits & License
68
 
69
  ### Base Models & Research
@@ -78,15 +145,3 @@ Then place each file in the directory its type requires (see the table at the to
78
  ---
79
 
80
  **Disclaimer:** These models are provided for optimization, workflow acceleration, and research purposes. Please adhere to the licenses and terms of the respective base models.
81
-
82
- ## πŸ“š Detailed documentation
83
-
84
- - [ControlNet ConvRot INT8 trajectory impact guide](https://github.com/ussoewwin/Hybrid-Sensitivity-Weighted-Quantization/blob/main/md/ControlNet_ConvRot_INT8_Trajectory_Impact_Technical_Guide.md)
85
- - [Backdoor Audit Verdicts](https://github.com/ussoewwin/Hybrid-Sensitivity-Weighted-Quantization/blob/main/md/Backdoor_Audit_Verdicts.md)
86
-
87
- ## πŸ”’ Security
88
-
89
- **Backdoor audit.** Each file's audit status - verdict strength, weight coverage, the record behind it,
90
- and the limits of what that measurement can show - is in
91
- [Backdoor Audit Verdicts](https://github.com/ussoewwin/Hybrid-Sensitivity-Weighted-Quantization/blob/main/md/Backdoor_Audit_Verdicts.md). The audit code, the procedure and the raw records are in the same
92
- repository.
 
34
  - **Technical Details & Quantization Framework:** [Hybrid-Sensitivity-Weighted-Quantization (HSWQ)](https://github.com/ussoewwin/Hybrid-Sensitivity-Weighted-Quantization)
35
  - **ComfyUI Loader for ConvRot INT8:** [ComfyUI-HSWQ-Loader-and-Tools](https://github.com/ussoewwin/ComfyUI-HSWQ-Loader-and-Tools)
36
 
37
+ ---
38
+
39
+ ## ⚠️ Where each pack goes, and how it loads
40
+
41
+ **This repository mixes two runtime object types.** They use different loaders, and getting this wrong means the file silently fails to apply.
42
+
43
+ | group | the runtime object | the loader | the directory |
44
+ | :--- | :--- | :--- | :--- |
45
+ | **ControlNet** (SDXL anytest A/B, SDXL union-pro-max, Qwen-Image-2512 Union 2602, Qwen-Image Inpainting, FLUX.1-dev Union Pro 2.0) | a ControlNet | `ControlNetLoader` (stock) or the HSWQ ControlNet loader | `ComfyUI/models/controlnet/` |
46
+ | **Model patch** (Z-Image-Turbo Fun Controlnet Union/Tile, Qwen-Image DiffSynth canny/depth/inpaint) | a **model patch**, despite the "ControlNet" name | **`HSWQ Model Patch Loader`** (`HSWQModelPatchLoaderCustom`). The **stock `ModelPatchLoader` cannot read the INT8 layout** | `ComfyUI/models/model_patches/` |
47
+
48
+ The model patches are applied by the dedicated apply nodes (`ZImageFunControlnet`, `QwenImageDiffsynthControlnet`), not by `Apply ControlNet`.
49
+
50
+ ---
51
+
52
+ ## 🌟 Model Overview
53
+
54
+ This repository hosts high-quality **ConvRot INT8** quantized weights for all-in-one ControlNet Union and dedicated condition-specific ControlNet models. Orthogonal Hadamard rotation is applied to each 2D weight before per-channel INT8 quantization, which suppresses activation outlier distortion and cuts the disk footprint roughly in half while preserving structural control fidelity:
55
+
56
+ - **CN-anytest4_illustrious2 (Variants A & B)**: Multi-purpose all-in-one Anytest v4 ControlNet models fine-tuned for **Illustrious-XL / SDXL** by **2vXpSwA7**, offering high-precision anime and illustration structure guiding.
57
+ - **controlnet-union-pro-max-sdxl-1.0**: The comprehensive all-in-one ControlNet Union model for **SDXL 1.0** by **xinsir**, supporting 10+ control conditions in a single compact file.
58
+ - **Z-Image-Turbo-Fun-Controlnet-Union-2.1-lite-2602-8steps**: Multi-condition ControlNet Union model (lite architecture, 8-step distilled) for the **Z-Image-Turbo** architecture by **alibaba-pai**, providing lightweight and fast structure control (Canny, Depth, Pose, Color, etc.). **Model patch.**
59
+ - **Z-Image-Turbo-Fun-Controlnet-Tile-2.1-lite-2601-8steps**: Dedicated high-resolution Tile and upscaling ControlNet model (lite architecture, 8-step distilled) for the **Z-Image-Turbo** architecture by **alibaba-pai**, optimized for super-resolution detail enhancement. **Model patch.**
60
+ - **Qwen-Image-2512-Fun-Controlnet-Union-2602**: Multi-condition ControlNet Union model (5 layer blocks) for the **Qwen-Image-2512** architecture.
61
+ - **Qwen-Image-ControlNet-Inpainting**: Dedicated inpainting and editing ControlNet model for the **Qwen-Image** architecture.
62
+ - **Qwen-Image DiffSynth ControlNet Suite** (canny / depth / inpaint): Dedicated single-condition ControlNet models for the **Qwen-Image** architecture developed by **DiffSynth-Studio**. **Model patches.**
63
+ - **FLUX.1-dev-ControlNet-Union-Pro-2.0**: Next-generation unified 7-in-1 ControlNet for the **FLUX.1-dev** architecture by **Shakker Labs**.
64
+
65
+ ---
66
+
67
  ## πŸ“¦ Available Models
68
+
69
  Sizes are decimal (1 GB = 10\^9 bytes) and exact; the `SHA256` column is the pack's own hash, so a download can be verified against it.
70
 
71
  | Filename | Base Architecture | Base Model / Author | Supported Conditions | Quantization | File Size | SHA256 | License |
 
82
  | `Z-Image-Turbo-Fun-Controlnet-Union-2.1-lite-2602-8steps_hswq_convrot_int8.safetensors` | Z-Image-Turbo | [alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1](https://huggingface.co/alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1) | Multi-condition (Canny, Depth, Pose, Color, Scribble, Inpaint) | ConvRot INT8 | 1009.7 MB | `1dd12b2d028b12df6b2a4f93856f8ab161c1edc05125e3b1c987a1694e99dba4` | Apache-2.0 |
83
  | `Z-Image-Turbo-Fun-Controlnet-Tile-2.1-lite-2601-8steps_hswq_convrot_int8.safetensors` | Z-Image-Turbo | [alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1](https://huggingface.co/alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1) | Tile, Super-Resolution, Detail Enhancing | ConvRot INT8 | 1009.7 MB | `2998f83ef65903b4b6b22fb7ac270ed66f9a2443220563aa43db4188024763a8` | Apache-2.0 |
84
 
85
+ ---
86
+
87
+ ## πŸ“ Measured fidelity
88
+
89
+ Each pack's protection set was chosen by a **measured** aggregate contribution on the model's own output, over 8 seeds, and the acceptance was measured separately on a fixed 256-item set (seed 42) through the same loader the runtime uses. "Gate" is `cos.mean >= 0.99` and `cos.min >= 0.98`.
90
+
91
+ | Pack | the score (mean / min) | the gate | the note |
92
+ | :--- | :--- | :--- | :--- |
93
+ | `CN-anytest4_illustrious2_A` | 0.999626 (forward, relMSE 1.11e-03) | PASS | trajectory-impact selection, 36 protected |
94
+ | `CN-anytest4_illustrious2_B` | 0.992958 / 0.948279 (trajectory) | PASS | trajectory gate, 35 protected |
95
+ | `controlnet-union-pro-max-sdxl-1.0` | 0.999961 | PASS | curve flat, smallest satisfying pack taken (41 protected) |
96
+ | `Qwen-Image-2512-Fun-Controlnet-Union-2602` | 0.999158 / 0.998215 | PASS | floor case: all 77 layers INT8, no protection |
97
+ | `Qwen-Image-ControlNet-Inpainting` | 0.999545 / 0.998887 | PASS | floor case: all 95 layers INT8, no protection |
98
+ | `qwen_image_canny_diffsynth` | 0.9999924905 / 0.9999924096 | PASS | one protected layer (`img_in`), 598x above the runner-up |
99
+ | `qwen_image_depth_diffsynth` | 0.9999924109 / 0.9999923262 | PASS | one protected layer (`img_in`), 376x above the runner-up |
100
+ | `qwen_image_inpaint_diffsynth` | 0.9999921236 / 0.9999919542 | PASS | one protected layer (`img_in`), 147x above the runner-up |
101
+ | `Z-Image-Turbo-Fun-Controlnet-Union` | 0.999862 / 0.999491 (trajectory) | PASS | all 38 layers INT8, no protection |
102
+ | `Z-Image-Turbo-Fun-Controlnet-Tile` | 0.999956 / 0.999813 (trajectory) | PASS | all 38 layers INT8, no protection |
103
+ | `FLUX.1-dev-ControlNet-Union-Pro-2.0` | 0.999771 / 0.999488 | PASS | 12 protected; the all-INT8 state measured 0.999622 / 0.999020 |
104
+
105
+ The per-checkpoint reasoning and the full measured curves are in the HSWQ repository's `md/` guides.
106
+
107
+ ---
108
+
109
+ ## πŸ› οΈ Key Features
110
+
111
+ - **All-in-One & Dedicated Condition Control**: Unified multi-condition architectures alongside specialized single-condition models (Canny, Depth, Tile, Inpainting) for fine-grained workflow flexibility.
112
+ - **ConvRot INT8 Precision**: Orthogonal Hadamard rotations redistribute channel-wise outlier spikes across the input dimension before per-channel INT8 quantization, preventing quantization error buildup in deep control layers.
113
+ - **Measured, not assumed, protection**: Where a layer's measured contribution is large it stays FP16 (kept tensors are stored FP16, not FP32). Several packs here need no protection at all; the measured curves say so.
114
+ - **Footprint**: roughly half the unquantized checkpoint, with the INT8 execution path (Tensor Core INT8 matmul) as the additional runtime gain.
115
+
116
+ ---
117
+
118
  ## πŸš€ Usage in ComfyUI
119
+
120
  Install the loader extension:
121
 
122
  ```bash
 
129
  - **ControlNet** files go in `ComfyUI/models/controlnet/` and load with `ControlNetLoader`.
130
  - **Model patch** files go in `ComfyUI/models/model_patches/` and load with **`HSWQ Model Patch Loader`** (the stock `ModelPatchLoader` reads the INT8 tensors as plain weights), then apply with `ZImageFunControlnet` / `QwenImageDiffsynthControlnet`.
131
 
132
+ ---
133
+
134
  ## πŸ“œ Credits & License
135
 
136
  ### Base Models & Research
 
145
  ---
146
 
147
  **Disclaimer:** These models are provided for optimization, workflow acceleration, and research purposes. Please adhere to the licenses and terms of the respective base models.