docs: reduce the model card to a summary with links to the GitHub reports
Browse files
README.md
CHANGED
|
@@ -34,38 +34,7 @@ High-fidelity **ConvRot INT8** quantized weights for multi-condition and dedicat
|
|
| 34 |
- **Technical Details & Quantization Framework:** [Hybrid-Sensitivity-Weighted-Quantization (HSWQ)](https://github.com/ussoewwin/Hybrid-Sensitivity-Weighted-Quantization)
|
| 35 |
- **ComfyUI Loader for ConvRot INT8:** [ComfyUI-HSWQ-Loader-and-Tools](https://github.com/ussoewwin/ComfyUI-HSWQ-Loader-and-Tools)
|
| 36 |
|
| 37 |
-
---
|
| 38 |
-
|
| 39 |
-
## β οΈ Where each pack goes, and how it loads
|
| 40 |
-
|
| 41 |
-
**This repository mixes two runtime object types.** They use different loaders, and getting this wrong means the file silently fails to apply.
|
| 42 |
-
|
| 43 |
-
| group | the runtime object | the loader | the directory |
|
| 44 |
-
| :--- | :--- | :--- | :--- |
|
| 45 |
-
| **ControlNet** (SDXL anytest A/B, SDXL union-pro-max, Qwen-Image-2512 Union 2602, Qwen-Image Inpainting, FLUX.1-dev Union Pro 2.0) | a ControlNet | `ControlNetLoader` (stock) or the HSWQ ControlNet loader | `ComfyUI/models/controlnet/` |
|
| 46 |
-
| **Model patch** (Z-Image-Turbo Fun Controlnet Union/Tile, Qwen-Image DiffSynth canny/depth/inpaint) | a **model patch**, despite the "ControlNet" name | **`HSWQ Model Patch Loader`** (`HSWQModelPatchLoaderCustom`). The **stock `ModelPatchLoader` cannot read the INT8 layout** | `ComfyUI/models/model_patches/` |
|
| 47 |
-
|
| 48 |
-
The model patches are applied by the dedicated apply nodes (`ZImageFunControlnet`, `QwenImageDiffsynthControlnet`), not by `Apply ControlNet`.
|
| 49 |
-
|
| 50 |
-
---
|
| 51 |
-
|
| 52 |
-
## π Model Overview
|
| 53 |
-
|
| 54 |
-
This repository hosts high-quality **ConvRot INT8** quantized weights for all-in-one ControlNet Union and dedicated condition-specific ControlNet models. Orthogonal Hadamard rotation is applied to each 2D weight before per-channel INT8 quantization, which suppresses activation outlier distortion and cuts the disk footprint roughly in half while preserving structural control fidelity:
|
| 55 |
-
|
| 56 |
-
- **CN-anytest4_illustrious2 (Variants A & B)**: Multi-purpose all-in-one Anytest v4 ControlNet models fine-tuned for **Illustrious-XL / SDXL** by **2vXpSwA7**, offering high-precision anime and illustration structure guiding.
|
| 57 |
-
- **controlnet-union-pro-max-sdxl-1.0**: The comprehensive all-in-one ControlNet Union model for **SDXL 1.0** by **xinsir**, supporting 10+ control conditions in a single compact file.
|
| 58 |
-
- **Z-Image-Turbo-Fun-Controlnet-Union-2.1-lite-2602-8steps**: Multi-condition ControlNet Union model (lite architecture, 8-step distilled) for the **Z-Image-Turbo** architecture by **alibaba-pai**, providing lightweight and fast structure control (Canny, Depth, Pose, Color, etc.). **Model patch.**
|
| 59 |
-
- **Z-Image-Turbo-Fun-Controlnet-Tile-2.1-lite-2601-8steps**: Dedicated high-resolution Tile and upscaling ControlNet model (lite architecture, 8-step distilled) for the **Z-Image-Turbo** architecture by **alibaba-pai**, optimized for super-resolution detail enhancement. **Model patch.**
|
| 60 |
-
- **Qwen-Image-2512-Fun-Controlnet-Union-2602**: Multi-condition ControlNet Union model (5 layer blocks) for the **Qwen-Image-2512** architecture.
|
| 61 |
-
- **Qwen-Image-ControlNet-Inpainting**: Dedicated inpainting and editing ControlNet model for the **Qwen-Image** architecture.
|
| 62 |
-
- **Qwen-Image DiffSynth ControlNet Suite** (canny / depth / inpaint): Dedicated single-condition ControlNet models for the **Qwen-Image** architecture developed by **DiffSynth-Studio**. **Model patches.**
|
| 63 |
-
- **FLUX.1-dev-ControlNet-Union-Pro-2.0**: Next-generation unified 7-in-1 ControlNet for the **FLUX.1-dev** architecture by **Shakker Labs**.
|
| 64 |
-
|
| 65 |
-
---
|
| 66 |
-
|
| 67 |
## π¦ Available Models
|
| 68 |
-
|
| 69 |
Sizes are decimal (1 GB = 10\^9 bytes) and exact; the `SHA256` column is the pack's own hash, so a download can be verified against it.
|
| 70 |
|
| 71 |
| Filename | Base Architecture | Base Model / Author | Supported Conditions | Quantization | File Size | SHA256 | License |
|
|
@@ -82,41 +51,7 @@ Sizes are decimal (1 GB = 10\^9 bytes) and exact; the `SHA256` column is the pac
|
|
| 82 |
| `Z-Image-Turbo-Fun-Controlnet-Union-2.1-lite-2602-8steps_hswq_convrot_int8.safetensors` | Z-Image-Turbo | [alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1](https://huggingface.co/alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1) | Multi-condition (Canny, Depth, Pose, Color, Scribble, Inpaint) | ConvRot INT8 | 1009.7 MB | `1dd12b2d028b12df6b2a4f93856f8ab161c1edc05125e3b1c987a1694e99dba4` | Apache-2.0 |
|
| 83 |
| `Z-Image-Turbo-Fun-Controlnet-Tile-2.1-lite-2601-8steps_hswq_convrot_int8.safetensors` | Z-Image-Turbo | [alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1](https://huggingface.co/alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1) | Tile, Super-Resolution, Detail Enhancing | ConvRot INT8 | 1009.7 MB | `2998f83ef65903b4b6b22fb7ac270ed66f9a2443220563aa43db4188024763a8` | Apache-2.0 |
|
| 84 |
|
| 85 |
-
---
|
| 86 |
-
|
| 87 |
-
## π Measured fidelity
|
| 88 |
-
|
| 89 |
-
Each pack's protection set was chosen by a **measured** aggregate contribution on the model's own output, over 8 seeds, and the acceptance was measured separately on a fixed 256-item set (seed 42) through the same loader the runtime uses. "Gate" is `cos.mean >= 0.99` and `cos.min >= 0.98`.
|
| 90 |
-
|
| 91 |
-
| Pack | the score (mean / min) | the gate | the note |
|
| 92 |
-
| :--- | :--- | :--- | :--- |
|
| 93 |
-
| `CN-anytest4_illustrious2_A` | 0.999626 (forward, relMSE 1.11e-03) | PASS | trajectory-impact selection, 36 protected |
|
| 94 |
-
| `CN-anytest4_illustrious2_B` | 0.992958 / 0.948279 (trajectory) | PASS | trajectory gate, 35 protected |
|
| 95 |
-
| `controlnet-union-pro-max-sdxl-1.0` | 0.999961 | PASS | curve flat, smallest satisfying pack taken (41 protected) |
|
| 96 |
-
| `Qwen-Image-2512-Fun-Controlnet-Union-2602` | 0.999158 / 0.998215 | PASS | floor case: all 77 layers INT8, no protection |
|
| 97 |
-
| `Qwen-Image-ControlNet-Inpainting` | 0.999545 / 0.998887 | PASS | floor case: all 95 layers INT8, no protection |
|
| 98 |
-
| `qwen_image_canny_diffsynth` | 0.9999924905 / 0.9999924096 | PASS | one protected layer (`img_in`), 598x above the runner-up |
|
| 99 |
-
| `qwen_image_depth_diffsynth` | 0.9999924109 / 0.9999923262 | PASS | one protected layer (`img_in`), 376x above the runner-up |
|
| 100 |
-
| `qwen_image_inpaint_diffsynth` | 0.9999921236 / 0.9999919542 | PASS | one protected layer (`img_in`), 147x above the runner-up |
|
| 101 |
-
| `Z-Image-Turbo-Fun-Controlnet-Union` | 0.999862 / 0.999491 (trajectory) | PASS | all 38 layers INT8, no protection |
|
| 102 |
-
| `Z-Image-Turbo-Fun-Controlnet-Tile` | 0.999956 / 0.999813 (trajectory) | PASS | all 38 layers INT8, no protection |
|
| 103 |
-
| `FLUX.1-dev-ControlNet-Union-Pro-2.0` | 0.999771 / 0.999488 | PASS | 12 protected; the all-INT8 state measured 0.999622 / 0.999020 |
|
| 104 |
-
|
| 105 |
-
The per-checkpoint reasoning and the full measured curves are in the HSWQ repository's `md/` guides.
|
| 106 |
-
|
| 107 |
-
---
|
| 108 |
-
|
| 109 |
-
## π οΈ Key Features
|
| 110 |
-
|
| 111 |
-
- **All-in-One & Dedicated Condition Control**: Unified multi-condition architectures alongside specialized single-condition models (Canny, Depth, Tile, Inpainting) for fine-grained workflow flexibility.
|
| 112 |
-
- **ConvRot INT8 Precision**: Orthogonal Hadamard rotations redistribute channel-wise outlier spikes across the input dimension before per-channel INT8 quantization, preventing quantization error buildup in deep control layers.
|
| 113 |
-
- **Measured, not assumed, protection**: Where a layer's measured contribution is large it stays FP16 (kept tensors are stored FP16, not FP32). Several packs here need no protection at all; the measured curves say so.
|
| 114 |
-
- **Footprint**: roughly half the unquantized checkpoint, with the INT8 execution path (Tensor Core INT8 matmul) as the additional runtime gain.
|
| 115 |
-
|
| 116 |
-
---
|
| 117 |
-
|
| 118 |
## π Usage in ComfyUI
|
| 119 |
-
|
| 120 |
Install the loader extension:
|
| 121 |
|
| 122 |
```bash
|
|
@@ -129,8 +64,6 @@ Then place each file in the directory its type requires (see the table at the to
|
|
| 129 |
- **ControlNet** files go in `ComfyUI/models/controlnet/` and load with `ControlNetLoader`.
|
| 130 |
- **Model patch** files go in `ComfyUI/models/model_patches/` and load with **`HSWQ Model Patch Loader`** (the stock `ModelPatchLoader` reads the INT8 tensors as plain weights), then apply with `ZImageFunControlnet` / `QwenImageDiffsynthControlnet`.
|
| 131 |
|
| 132 |
-
---
|
| 133 |
-
|
| 134 |
## π Credits & License
|
| 135 |
|
| 136 |
### Base Models & Research
|
|
@@ -145,3 +78,15 @@ Then place each file in the directory its type requires (see the table at the to
|
|
| 145 |
---
|
| 146 |
|
| 147 |
**Disclaimer:** These models are provided for optimization, workflow acceleration, and research purposes. Please adhere to the licenses and terms of the respective base models.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
- **Technical Details & Quantization Framework:** [Hybrid-Sensitivity-Weighted-Quantization (HSWQ)](https://github.com/ussoewwin/Hybrid-Sensitivity-Weighted-Quantization)
|
| 35 |
- **ComfyUI Loader for ConvRot INT8:** [ComfyUI-HSWQ-Loader-and-Tools](https://github.com/ussoewwin/ComfyUI-HSWQ-Loader-and-Tools)
|
| 36 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
## π¦ Available Models
|
|
|
|
| 38 |
Sizes are decimal (1 GB = 10\^9 bytes) and exact; the `SHA256` column is the pack's own hash, so a download can be verified against it.
|
| 39 |
|
| 40 |
| Filename | Base Architecture | Base Model / Author | Supported Conditions | Quantization | File Size | SHA256 | License |
|
|
|
|
| 51 |
| `Z-Image-Turbo-Fun-Controlnet-Union-2.1-lite-2602-8steps_hswq_convrot_int8.safetensors` | Z-Image-Turbo | [alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1](https://huggingface.co/alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1) | Multi-condition (Canny, Depth, Pose, Color, Scribble, Inpaint) | ConvRot INT8 | 1009.7 MB | `1dd12b2d028b12df6b2a4f93856f8ab161c1edc05125e3b1c987a1694e99dba4` | Apache-2.0 |
|
| 52 |
| `Z-Image-Turbo-Fun-Controlnet-Tile-2.1-lite-2601-8steps_hswq_convrot_int8.safetensors` | Z-Image-Turbo | [alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1](https://huggingface.co/alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.1) | Tile, Super-Resolution, Detail Enhancing | ConvRot INT8 | 1009.7 MB | `2998f83ef65903b4b6b22fb7ac270ed66f9a2443220563aa43db4188024763a8` | Apache-2.0 |
|
| 53 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 54 |
## π Usage in ComfyUI
|
|
|
|
| 55 |
Install the loader extension:
|
| 56 |
|
| 57 |
```bash
|
|
|
|
| 64 |
- **ControlNet** files go in `ComfyUI/models/controlnet/` and load with `ControlNetLoader`.
|
| 65 |
- **Model patch** files go in `ComfyUI/models/model_patches/` and load with **`HSWQ Model Patch Loader`** (the stock `ModelPatchLoader` reads the INT8 tensors as plain weights), then apply with `ZImageFunControlnet` / `QwenImageDiffsynthControlnet`.
|
| 66 |
|
|
|
|
|
|
|
| 67 |
## π Credits & License
|
| 68 |
|
| 69 |
### Base Models & Research
|
|
|
|
| 78 |
---
|
| 79 |
|
| 80 |
**Disclaimer:** These models are provided for optimization, workflow acceleration, and research purposes. Please adhere to the licenses and terms of the respective base models.
|
| 81 |
+
|
| 82 |
+
## π Detailed documentation
|
| 83 |
+
|
| 84 |
+
- [ControlNet ConvRot INT8 trajectory impact guide](https://github.com/ussoewwin/Hybrid-Sensitivity-Weighted-Quantization/blob/main/md/ControlNet_ConvRot_INT8_Trajectory_Impact_Technical_Guide.md)
|
| 85 |
+
- [Backdoor Audit Verdicts](https://github.com/ussoewwin/Hybrid-Sensitivity-Weighted-Quantization/blob/main/md/Backdoor_Audit_Verdicts.md)
|
| 86 |
+
|
| 87 |
+
## π Security
|
| 88 |
+
|
| 89 |
+
**Backdoor audit.** Each file's audit status - verdict strength, weight coverage, the record behind it,
|
| 90 |
+
and the limits of what that measurement can show - is in
|
| 91 |
+
[Backdoor Audit Verdicts](https://github.com/ussoewwin/Hybrid-Sensitivity-Weighted-Quantization/blob/main/md/Backdoor_Audit_Verdicts.md). The audit code, the procedure and the raw records are in the same
|
| 92 |
+
repository.
|