ussoewwin commited on
Commit
80ddf5e
Β·
verified Β·
1 Parent(s): 0acd63f

docs: reduce the model card to a summary with links to the GitHub reports

Browse files
Files changed (1) hide show
  1. README.md +12 -67
README.md CHANGED
@@ -34,38 +34,7 @@ High-fidelity **ConvRot INT8** quantized weights for multi-condition and dedicat
34
  - **Technical Details & Quantization Framework:** [Hybrid-Sensitivity-Weighted-Quantization (HSWQ)](https://github.com/ussoewwin/Hybrid-Sensitivity-Weighted-Quantization)
35
  - **ComfyUI Loader for ConvRot INT8:** [ComfyUI-HSWQ-Loader-and-Tools](https://github.com/ussoewwin/ComfyUI-HSWQ-Loader-and-Tools)
36
 
37
- ---
38
-
39
- ## ⚠️ Where each pack goes, and how it loads
40
-
41
- **This repository mixes two runtime object types.** They use different loaders, and getting this wrong means the file silently fails to apply.
42
-
43
- | group | the runtime object | the loader | the directory |
44
- | :--- | :--- | :--- | :--- |
45
- | **ControlNet** (SDXL anytest A/B, SDXL union-pro-max, Qwen-Image-2512 Union 2602, Qwen-Image Inpainting, FLUX.1-dev Union Pro 2.0) | a ControlNet | `ControlNetLoader` (stock) or the HSWQ ControlNet loader | `ComfyUI/models/controlnet/` |
46
- | **Model patch** (Z-Image-Turbo Fun Controlnet Union/Tile, Qwen-Image DiffSynth canny/depth/inpaint) | a **model patch**, despite the "ControlNet" name | **`HSWQ Model Patch Loader`** (`HSWQModelPatchLoaderCustom`). The **stock `ModelPatchLoader` cannot read the INT8 layout** | `ComfyUI/models/model_patches/` |
47
-
48
- The model patches are applied by the dedicated apply nodes (`ZImageFunControlnet`, `QwenImageDiffsynthControlnet`), not by `Apply ControlNet`.
49
-
50
- ---
51
-
52
- ## 🌟 Model Overview
53
-
54
- This repository hosts high-quality **ConvRot INT8** quantized weights for all-in-one ControlNet Union and dedicated condition-specific ControlNet models. Orthogonal Hadamard rotation is applied to each 2D weight before per-channel INT8 quantization, which suppresses activation outlier distortion and cuts the disk footprint roughly in half while preserving structural control fidelity:
55
-
56
- - **CN-anytest4_illustrious2 (Variants A & B)**: Multi-purpose all-in-one Anytest v4 ControlNet models fine-tuned for **Illustrious-XL / SDXL** by **2vXpSwA7**, offering high-precision anime and illustration structure guiding.
57
- - **controlnet-union-pro-max-sdxl-1.0**: The comprehensive all-in-one ControlNet Union model for **SDXL 1.0** by **xinsir**, supporting 10+ control conditions in a single compact file.
58
- - **Z-Image-Turbo-Fun-Controlnet-Union-2.1-lite-2602-8steps**: Multi-condition ControlNet Union model (lite architecture, 8-step distilled) for the **Z-Image-Turbo** architecture by **alibaba-pai**, providing lightweight and fast structure control (Canny, Depth, Pose, Color, etc.). **Model patch.**
59
- - **Z-Image-Turbo-Fun-Controlnet-Tile-2.1-lite-2601-8steps**: Dedicated high-resolution Tile and upscaling ControlNet model (lite architecture, 8-step distilled) for the **Z-Image-Turbo** architecture by **alibaba-pai**, optimized for super-resolution detail enhancement. **Model patch.**
60
- - **Qwen-Image-2512-Fun-Controlnet-Union-2602**: Multi-condition ControlNet Union model (5 layer blocks) for the **Qwen-Image-2512** architecture.
61
- - **Qwen-Image-ControlNet-Inpainting**: Dedicated inpainting and editing ControlNet model for the **Qwen-Image** architecture.
62
- - **Qwen-Image DiffSynth ControlNet Suite** (canny / depth / inpaint): Dedicated single-condition ControlNet models for the **Qwen-Image** architecture developed by **DiffSynth-Studio**. **Model patches.**
63
- - **FLUX.1-dev-ControlNet-Union-Pro-2.0**: Next-generation unified 7-in-1 ControlNet for the **FLUX.1-dev** architecture by **Shakker Labs**.
64
-
65
- ---
66
-
67
  ## πŸ“¦ Available Models
68
-
69
  Sizes are decimal (1 GB = 10\^9 bytes) and exact; the `SHA256` column is the pack's own hash, so a download can be verified against it.
70
 
71
  | Filename | Base Architecture | Base Model / Author | Supported Conditions | Quantization | File Size | SHA256 | License |
@@ -82,41 +51,7 @@ Sizes are decimal (1 GB = 10\^9 bytes) and exact; the `SHA256` column is the pac
82
  | `Z-Image-Turbo-Fun-Controlnet-Union-2.1-lite-2602-8steps_hswq_convrot_int8.safetensors` | Z-Image-Turbo | [alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1](https://huggingface.co/alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1) | Multi-condition (Canny, Depth, Pose, Color, Scribble, Inpaint) | ConvRot INT8 | 1009.7 MB | `1dd12b2d028b12df6b2a4f93856f8ab161c1edc05125e3b1c987a1694e99dba4` | Apache-2.0 |
83
  | `Z-Image-Turbo-Fun-Controlnet-Tile-2.1-lite-2601-8steps_hswq_convrot_int8.safetensors` | Z-Image-Turbo | [alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1](https://huggingface.co/alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1) | Tile, Super-Resolution, Detail Enhancing | ConvRot INT8 | 1009.7 MB | `2998f83ef65903b4b6b22fb7ac270ed66f9a2443220563aa43db4188024763a8` | Apache-2.0 |
84
 
85
- ---
86
-
87
- ## πŸ“ Measured fidelity
88
-
89
- Each pack's protection set was chosen by a **measured** aggregate contribution on the model's own output, over 8 seeds, and the acceptance was measured separately on a fixed 256-item set (seed 42) through the same loader the runtime uses. "Gate" is `cos.mean >= 0.99` and `cos.min >= 0.98`.
90
-
91
- | Pack | the score (mean / min) | the gate | the note |
92
- | :--- | :--- | :--- | :--- |
93
- | `CN-anytest4_illustrious2_A` | 0.999626 (forward, relMSE 1.11e-03) | PASS | trajectory-impact selection, 36 protected |
94
- | `CN-anytest4_illustrious2_B` | 0.992958 / 0.948279 (trajectory) | PASS | trajectory gate, 35 protected |
95
- | `controlnet-union-pro-max-sdxl-1.0` | 0.999961 | PASS | curve flat, smallest satisfying pack taken (41 protected) |
96
- | `Qwen-Image-2512-Fun-Controlnet-Union-2602` | 0.999158 / 0.998215 | PASS | floor case: all 77 layers INT8, no protection |
97
- | `Qwen-Image-ControlNet-Inpainting` | 0.999545 / 0.998887 | PASS | floor case: all 95 layers INT8, no protection |
98
- | `qwen_image_canny_diffsynth` | 0.9999924905 / 0.9999924096 | PASS | one protected layer (`img_in`), 598x above the runner-up |
99
- | `qwen_image_depth_diffsynth` | 0.9999924109 / 0.9999923262 | PASS | one protected layer (`img_in`), 376x above the runner-up |
100
- | `qwen_image_inpaint_diffsynth` | 0.9999921236 / 0.9999919542 | PASS | one protected layer (`img_in`), 147x above the runner-up |
101
- | `Z-Image-Turbo-Fun-Controlnet-Union` | 0.999862 / 0.999491 (trajectory) | PASS | all 38 layers INT8, no protection |
102
- | `Z-Image-Turbo-Fun-Controlnet-Tile` | 0.999956 / 0.999813 (trajectory) | PASS | all 38 layers INT8, no protection |
103
- | `FLUX.1-dev-ControlNet-Union-Pro-2.0` | 0.999771 / 0.999488 | PASS | 12 protected; the all-INT8 state measured 0.999622 / 0.999020 |
104
-
105
- The per-checkpoint reasoning and the full measured curves are in the HSWQ repository's `md/` guides.
106
-
107
- ---
108
-
109
- ## πŸ› οΈ Key Features
110
-
111
- - **All-in-One & Dedicated Condition Control**: Unified multi-condition architectures alongside specialized single-condition models (Canny, Depth, Tile, Inpainting) for fine-grained workflow flexibility.
112
- - **ConvRot INT8 Precision**: Orthogonal Hadamard rotations redistribute channel-wise outlier spikes across the input dimension before per-channel INT8 quantization, preventing quantization error buildup in deep control layers.
113
- - **Measured, not assumed, protection**: Where a layer's measured contribution is large it stays FP16 (kept tensors are stored FP16, not FP32). Several packs here need no protection at all; the measured curves say so.
114
- - **Footprint**: roughly half the unquantized checkpoint, with the INT8 execution path (Tensor Core INT8 matmul) as the additional runtime gain.
115
-
116
- ---
117
-
118
  ## πŸš€ Usage in ComfyUI
119
-
120
  Install the loader extension:
121
 
122
  ```bash
@@ -129,8 +64,6 @@ Then place each file in the directory its type requires (see the table at the to
129
  - **ControlNet** files go in `ComfyUI/models/controlnet/` and load with `ControlNetLoader`.
130
  - **Model patch** files go in `ComfyUI/models/model_patches/` and load with **`HSWQ Model Patch Loader`** (the stock `ModelPatchLoader` reads the INT8 tensors as plain weights), then apply with `ZImageFunControlnet` / `QwenImageDiffsynthControlnet`.
131
 
132
- ---
133
-
134
  ## πŸ“œ Credits & License
135
 
136
  ### Base Models & Research
@@ -145,3 +78,15 @@ Then place each file in the directory its type requires (see the table at the to
145
  ---
146
 
147
  **Disclaimer:** These models are provided for optimization, workflow acceleration, and research purposes. Please adhere to the licenses and terms of the respective base models.
 
 
 
 
 
 
 
 
 
 
 
 
 
34
  - **Technical Details & Quantization Framework:** [Hybrid-Sensitivity-Weighted-Quantization (HSWQ)](https://github.com/ussoewwin/Hybrid-Sensitivity-Weighted-Quantization)
35
  - **ComfyUI Loader for ConvRot INT8:** [ComfyUI-HSWQ-Loader-and-Tools](https://github.com/ussoewwin/ComfyUI-HSWQ-Loader-and-Tools)
36
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
  ## πŸ“¦ Available Models
 
38
  Sizes are decimal (1 GB = 10\^9 bytes) and exact; the `SHA256` column is the pack's own hash, so a download can be verified against it.
39
 
40
  | Filename | Base Architecture | Base Model / Author | Supported Conditions | Quantization | File Size | SHA256 | License |
 
51
  | `Z-Image-Turbo-Fun-Controlnet-Union-2.1-lite-2602-8steps_hswq_convrot_int8.safetensors` | Z-Image-Turbo | [alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1](https://huggingface.co/alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1) | Multi-condition (Canny, Depth, Pose, Color, Scribble, Inpaint) | ConvRot INT8 | 1009.7 MB | `1dd12b2d028b12df6b2a4f93856f8ab161c1edc05125e3b1c987a1694e99dba4` | Apache-2.0 |
52
  | `Z-Image-Turbo-Fun-Controlnet-Tile-2.1-lite-2601-8steps_hswq_convrot_int8.safetensors` | Z-Image-Turbo | [alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1](https://huggingface.co/alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1) | Tile, Super-Resolution, Detail Enhancing | ConvRot INT8 | 1009.7 MB | `2998f83ef65903b4b6b22fb7ac270ed66f9a2443220563aa43db4188024763a8` | Apache-2.0 |
53
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
54
  ## πŸš€ Usage in ComfyUI
 
55
  Install the loader extension:
56
 
57
  ```bash
 
64
  - **ControlNet** files go in `ComfyUI/models/controlnet/` and load with `ControlNetLoader`.
65
  - **Model patch** files go in `ComfyUI/models/model_patches/` and load with **`HSWQ Model Patch Loader`** (the stock `ModelPatchLoader` reads the INT8 tensors as plain weights), then apply with `ZImageFunControlnet` / `QwenImageDiffsynthControlnet`.
66
 
 
 
67
  ## πŸ“œ Credits & License
68
 
69
  ### Base Models & Research
 
78
  ---
79
 
80
  **Disclaimer:** These models are provided for optimization, workflow acceleration, and research purposes. Please adhere to the licenses and terms of the respective base models.
81
+
82
+ ## πŸ“š Detailed documentation
83
+
84
+ - [ControlNet ConvRot INT8 trajectory impact guide](https://github.com/ussoewwin/Hybrid-Sensitivity-Weighted-Quantization/blob/main/md/ControlNet_ConvRot_INT8_Trajectory_Impact_Technical_Guide.md)
85
+ - [Backdoor Audit Verdicts](https://github.com/ussoewwin/Hybrid-Sensitivity-Weighted-Quantization/blob/main/md/Backdoor_Audit_Verdicts.md)
86
+
87
+ ## πŸ”’ Security
88
+
89
+ **Backdoor audit.** Each file's audit status - verdict strength, weight coverage, the record behind it,
90
+ and the limits of what that measurement can show - is in
91
+ [Backdoor Audit Verdicts](https://github.com/ussoewwin/Hybrid-Sensitivity-Weighted-Quantization/blob/main/md/Backdoor_Audit_Verdicts.md). The audit code, the procedure and the raw records are in the same
92
+ repository.