Kandinsky 6: replace raw transformers with int8_convrot and W6A8 quantisations

README.md

@@ -88,14 +88,18 @@ carries the Depth Anything V2 Small weights FlowSeek reads depth with. `LICENSE-
 ## models/kandinsky6/
 
 Kandinsky 6 for ComfyUI, in ComfyUI's own folder layout: each subfolder goes into the matching
-folder under `ComfyUI/models`. Every file is byte for byte the upstream file, and
-`models/kandinsky6/NOTICE.md` names the commit each was taken at, with its SHA-256. The
+folder under `ComfyUI/models`. The transformers are ComfyUI int8 ConvRot and W6A8 quantisations; everything else is byte for byte
+the upstream file. `models/kandinsky6/NOTICE.md` names each source commit and every SHA-256. The
 licence files sit beside the files they cover.
 
 | File | Original | Licence |
 |---|---|---|
-| `models/kandinsky6/diffusion_models/kandinsky6_lite_distill_5s.safetensors` | [kandinskylab/Kandinsky-6.0-Lite-distill-5s-Diffusers](https://huggingface.co/kandinskylab/Kandinsky-6.0-Lite-distill-5s-Diffusers), Kandinsky Lab | MIT |
-| `models/kandinsky6/diffusion_models/kandinsky6_lite_5s.safetensors` | [kandinskylab/Kandinsky-6.0-Lite-5s-Diffusers](https://huggingface.co/kandinskylab/Kandinsky-6.0-Lite-5s-Diffusers), Kandinsky Lab | MIT |
+| `models/kandinsky6/diffusion_models/kandinsky6_lite_distill_5s_int8_convrot.safetensors` | [kandinskylab/Kandinsky-6.0-Lite-distill-5s-Diffusers](https://huggingface.co/kandinskylab/Kandinsky-6.0-Lite-distill-5s-Diffusers), Kandinsky Lab, quantised for ComfyUI (int8_convrot) | MIT |
+| `models/kandinsky6/diffusion_models/kandinsky6_lite_distill_5s_w6a8.safetensors` | [kandinskylab/Kandinsky-6.0-Lite-distill-5s-Diffusers](https://huggingface.co/kandinskylab/Kandinsky-6.0-Lite-distill-5s-Diffusers), Kandinsky Lab, quantised for ComfyUI (w6a8) | MIT |
+| `models/kandinsky6/diffusion_models/kandinsky6_lite_5s_int8_convrot.safetensors` | [kandinskylab/Kandinsky-6.0-Lite-5s-Diffusers](https://huggingface.co/kandinskylab/Kandinsky-6.0-Lite-5s-Diffusers), Kandinsky Lab, quantised for ComfyUI (int8_convrot) | MIT |
+| `models/kandinsky6/diffusion_models/kandinsky6_lite_5s_w6a8.safetensors` | [kandinskylab/Kandinsky-6.0-Lite-5s-Diffusers](https://huggingface.co/kandinskylab/Kandinsky-6.0-Lite-5s-Diffusers), Kandinsky Lab, quantised for ComfyUI (w6a8) | MIT |
+| `models/kandinsky6/diffusion_models/kandinsky6_pro_distill_5s_int8_convrot.safetensors` | [kandinskylab/Kandinsky-6.0-Pro-distill-5s-Diffusers](https://huggingface.co/kandinskylab/Kandinsky-6.0-Pro-distill-5s-Diffusers), Kandinsky Lab, quantised for ComfyUI (int8_convrot) | MIT |
+| `models/kandinsky6/diffusion_models/kandinsky6_pro_distill_5s_w6a8.safetensors` | [kandinskylab/Kandinsky-6.0-Pro-distill-5s-Diffusers](https://huggingface.co/kandinskylab/Kandinsky-6.0-Pro-distill-5s-Diffusers), Kandinsky Lab, quantised for ComfyUI (w6a8) | MIT |
 | `models/kandinsky6/text_encoders/qwen_2.5_vl_7b_fp8_scaled.safetensors` | [Qwen/Qwen2.5-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct), as repackaged by [Comfy-Org](https://huggingface.co/Comfy-Org/Qwen-Image_ComfyUI) | Apache-2.0 |
 | `models/kandinsky6/text_encoders/clip_l.safetensors` | [openai/clip-vit-large-patch14](https://huggingface.co/openai/clip-vit-large-patch14), as repackaged by [Comfy-Org](https://huggingface.co/Comfy-Org/HunyuanVideo_repackaged) | MIT |
 | `models/kandinsky6/vae/kandinsky6_audio_vae.safetensors` | [MMAudio](https://huggingface.co/hkchengrex/MMAudio)'s 44.1 kHz autoencoder and [BigVGAN v2](https://huggingface.co/nvidia/bigvgan_v2_44khz_128band_512x), as packed by Kandinsky Lab | **CC BY-NC 4.0, non-commercial** (MMAudio); MIT (BigVGAN) |

models/kandinsky6/NOTICE.md

@@ -2,21 +2,49 @@
 
 The files WAS Node Suite runs Kandinsky 6 with, laid out as ComfyUI's own `models` folder:
 `diffusion_models/` for Load Diffusion Model, `text_encoders/` for DualCLIPLoader with type
-`kandinsky5`, `vae/` for Load VAE and Kandinsky 6 Audio VAE Loader. Every file is byte for byte
-the upstream file named beside it; its SHA-256 matches upstream's own record.
+`kandinsky5`, `vae/` for Load VAE and Kandinsky 6 Audio VAE Loader. The text encoders and the audio VAE are byte for byte the upstream file named beside them; the
+transformers are ComfyUI quantisations of the upstream file, described below.
 
 | File | Upstream | Licence |
 |---|---|---|
-| `diffusion_models/kandinsky6_lite_distill_5s.safetensors` | [kandinskylab/Kandinsky-6.0-Lite-distill-5s-Diffusers](https://huggingface.co/kandinskylab/Kandinsky-6.0-Lite-distill-5s-Diffusers) at `efbaeb9961770cc82fece5b9cb2197049d8c983c`, `transformer/diffusion_pytorch_model.safetensors` | MIT, Kandinsky Lab: `diffusion_models/LICENSE-Kandinsky6` |
-| `diffusion_models/kandinsky6_lite_5s.safetensors` | [kandinskylab/Kandinsky-6.0-Lite-5s-Diffusers](https://huggingface.co/kandinskylab/Kandinsky-6.0-Lite-5s-Diffusers) at `5686616f04318cbec7b0a4978b4ff1697395de15`, `transformer/diffusion_pytorch_model.safetensors` | MIT, Kandinsky Lab: `diffusion_models/LICENSE-Kandinsky6` |
+| `diffusion_models/kandinsky6_lite_distill_5s_int8_convrot.safetensors` | int8 ConvRot of [kandinskylab/Kandinsky-6.0-Lite-distill-5s-Diffusers](https://huggingface.co/kandinskylab/Kandinsky-6.0-Lite-distill-5s-Diffusers) at `efbaeb9961770cc82fece5b9cb2197049d8c983c`, `transformer/diffusion_pytorch_model.safetensors` | MIT, Kandinsky Lab: `diffusion_models/LICENSE-Kandinsky6` |
+| `diffusion_models/kandinsky6_lite_distill_5s_w6a8.safetensors` | W6A8 of [kandinskylab/Kandinsky-6.0-Lite-distill-5s-Diffusers](https://huggingface.co/kandinskylab/Kandinsky-6.0-Lite-distill-5s-Diffusers) at `efbaeb9961770cc82fece5b9cb2197049d8c983c`, `transformer/diffusion_pytorch_model.safetensors` | MIT, Kandinsky Lab: `diffusion_models/LICENSE-Kandinsky6` |
+| `diffusion_models/kandinsky6_lite_5s_int8_convrot.safetensors` | int8 ConvRot of [kandinskylab/Kandinsky-6.0-Lite-5s-Diffusers](https://huggingface.co/kandinskylab/Kandinsky-6.0-Lite-5s-Diffusers) at `5686616f04318cbec7b0a4978b4ff1697395de15`, `transformer/diffusion_pytorch_model.safetensors` | MIT, Kandinsky Lab: `diffusion_models/LICENSE-Kandinsky6` |
+| `diffusion_models/kandinsky6_lite_5s_w6a8.safetensors` | W6A8 of [kandinskylab/Kandinsky-6.0-Lite-5s-Diffusers](https://huggingface.co/kandinskylab/Kandinsky-6.0-Lite-5s-Diffusers) at `5686616f04318cbec7b0a4978b4ff1697395de15`, `transformer/diffusion_pytorch_model.safetensors` | MIT, Kandinsky Lab: `diffusion_models/LICENSE-Kandinsky6` |
+| `diffusion_models/kandinsky6_pro_distill_5s_int8_convrot.safetensors` | int8 ConvRot of [kandinskylab/Kandinsky-6.0-Pro-distill-5s-Diffusers](https://huggingface.co/kandinskylab/Kandinsky-6.0-Pro-distill-5s-Diffusers) at `1018d5828716e8bea3de99609f45c924de9c5564`, `transformer/diffusion_pytorch_model.safetensors` | MIT, Kandinsky Lab: `diffusion_models/LICENSE-Kandinsky6` |
+| `diffusion_models/kandinsky6_pro_distill_5s_w6a8.safetensors` | W6A8 of [kandinskylab/Kandinsky-6.0-Pro-distill-5s-Diffusers](https://huggingface.co/kandinskylab/Kandinsky-6.0-Pro-distill-5s-Diffusers) at `1018d5828716e8bea3de99609f45c924de9c5564`, `transformer/diffusion_pytorch_model.safetensors` | MIT, Kandinsky Lab: `diffusion_models/LICENSE-Kandinsky6` |
 | `text_encoders/qwen_2.5_vl_7b_fp8_scaled.safetensors` | [Comfy-Org/Qwen-Image_ComfyUI](https://huggingface.co/Comfy-Org/Qwen-Image_ComfyUI) at `1f12b17be14c89b026c51a91d67c32f84bb047bc`, `split_files/text_encoders/qwen_2.5_vl_7b_fp8_scaled.safetensors`, Comfy-Org's fp8 repackaging of [Qwen/Qwen2.5-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct) | Apache-2.0, Alibaba Cloud: `text_encoders/LICENSE-Qwen2.5-VL` |
 | `text_encoders/clip_l.safetensors` | [Comfy-Org/HunyuanVideo_repackaged](https://huggingface.co/Comfy-Org/HunyuanVideo_repackaged) at `533a43813f01517d9e624f9e824a85bb94094120`, `split_files/text_encoders/clip_l.safetensors`, the text tower of [openai/clip-vit-large-patch14](https://huggingface.co/openai/clip-vit-large-patch14) | MIT, OpenAI: `text_encoders/LICENSE-CLIP` |
 | `vae/kandinsky6_audio_vae.safetensors` | [kandinskylab/Kandinsky-6.0-Lite-distill-5s-Diffusers](https://huggingface.co/kandinskylab/Kandinsky-6.0-Lite-distill-5s-Diffusers) at `efbaeb9961770cc82fece5b9cb2197049d8c983c`, `audio_vae/diffusion_pytorch_model.safetensors` | **CC BY-NC 4.0, non-commercial**, and MIT: see below |
 
+## The transformers are ComfyUI quantisations
+
+The Kandinsky 6 transformers are held here only as ComfyUI quantised checkpoints, made from the upstream file
+with comfy-kitchen's own quantisers and loaded by ComfyUI's Load Diffusion Model as they are.
+
+| Suffix | Format |
+|---|---|
+| `_int8_convrot` | `int8_tensorwise` with ConvRot rotation, group 256, one scale per output row |
+| `_w6a8` | `w6a8_int8`, 6-bit weights in groups of 32 with ConvRot rotation, group 256, run with int8 activations |
+
+Quantised: the query, key, value and output projections and both feed-forward weights of every
+transformer block whose input width is a multiple of 256. Every other tensor is bfloat16. On Lite the
+audio stream is 896 wide, so its block weights stay bfloat16 too.
+
+Source transformers, SHA-256:
+
+    b1b52668561d9138d93bb4da13c9999e615bf569be9f8cf65732d54652447f53  kandinskylab/Kandinsky-6.0-Lite-distill-5s-Diffusers@efbaeb9961 transformer
+    f84ea3266560bbd1f203a874b8cee43e080073820facaf3306cab8fae2798a97  kandinskylab/Kandinsky-6.0-Pro-distill-5s-Diffusers@1018d58287 transformer
+    1550090ca56fed8c75dda0d30f8aa7b46775e8e1ebb086b70ad532e40eb878f9  kandinskylab/Kandinsky-6.0-Lite-5s-Diffusers@5686616f04 transformer
+
 ## SHA-256
 
-    b1b52668561d9138d93bb4da13c9999e615bf569be9f8cf65732d54652447f53  diffusion_models/kandinsky6_lite_distill_5s.safetensors
-    1550090ca56fed8c75dda0d30f8aa7b46775e8e1ebb086b70ad532e40eb878f9  diffusion_models/kandinsky6_lite_5s.safetensors
+    b3fb6f0bd38077e769ef947555927a25d585269903328417bf8c6da8dda277c0  diffusion_models/kandinsky6_lite_distill_5s_int8_convrot.safetensors
+    e936e403761efd07463bc96a4d6608a8014399243c76a8b249d6e3b7be9398e9  diffusion_models/kandinsky6_lite_distill_5s_w6a8.safetensors
+    f6b8a302c99116ee1273a062b8978e10437a6b06f98c71f1c5ba49ecb485aa4c  diffusion_models/kandinsky6_lite_5s_int8_convrot.safetensors
+    375b16bd1f3981806d7f9513978f0c1e0a460d074d080fe1f267c189cbb895f4  diffusion_models/kandinsky6_lite_5s_w6a8.safetensors
+    8b96fc841b9ecd1d96218b70ecffd846ba8b620fa24d087fc99a547fbdc4edf9  diffusion_models/kandinsky6_pro_distill_5s_int8_convrot.safetensors
+    d26c82be04d5e2d97f4bcf184f0af09768f3af92ade3b7fff52ee181f0a0b968  diffusion_models/kandinsky6_pro_distill_5s_w6a8.safetensors
     cb5636d852a0ea6a9075ab1bef496c0db7aef13c02350571e388aea959c5c0b4  text_encoders/qwen_2.5_vl_7b_fp8_scaled.safetensors
     660c6f5b1abae9dc498ac2d21e1347d2abdb0cf6c0c0c8576cd796491d9a6cdd  text_encoders/clip_l.safetensors
     dfccdf9d0d8d3de270a23b097400f21adb91f4ec683bdc26369cb65b08a5af87  vae/kandinsky6_audio_vae.safetensors
@@ -42,4 +70,4 @@ non-commercial use, unchanged from Kandinsky Lab's file.
 | File | Where it is | Why |
 |---|---|---|
 | `hunyuan_video_vae_bf16.safetensors`, the video VAE | [Comfy-Org/HunyuanVideo_repackaged](https://huggingface.co/Comfy-Org/HunyuanVideo_repackaged), `split_files/vae/` | Tencent Hunyuan Community License |
-| Kandinsky 6 Pro, distilled and not | [kandinskylab](https://huggingface.co/kandinskylab) | Not yet mirrored |
+| Kandinsky 6 Pro, not distilled | [kandinskylab/Kandinsky-6.0-Pro-5s-Diffusers](https://huggingface.co/kandinskylab/Kandinsky-6.0-Pro-5s-Diffusers) | Gated upstream |

models/kandinsky6/diffusion_models/{kandinsky6_lite_5s.safetensors → kandinsky6_lite_5s_int8_convrot.safetensors}

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1550090ca56fed8c75dda0d30f8aa7b46775e8e1ebb086b70ad532e40eb878f9
-size 7476528464
+oid sha256:f6b8a302c99116ee1273a062b8978e10437a6b06f98c71f1c5ba49ecb485aa4c
+size 4219006432

models/kandinsky6/diffusion_models/{kandinsky6_lite_distill_5s.safetensors → kandinsky6_lite_5s_w6a8.safetensors}

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b1b52668561d9138d93bb4da13c9999e615bf569be9f8cf65732d54652447f53
-size 6381599096
+oid sha256:375b16bd1f3981806d7f9513978f0c1e0a460d074d080fe1f267c189cbb895f4
+size 3751252584

models/kandinsky6/diffusion_models/kandinsky6_lite_distill_5s_int8_convrot.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b3fb6f0bd38077e769ef947555927a25d585269903328417bf8c6da8dda277c0
+size 4221717816

models/kandinsky6/diffusion_models/kandinsky6_lite_distill_5s_w6a8.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e936e403761efd07463bc96a4d6608a8014399243c76a8b249d6e3b7be9398e9
+size 3753963984

models/kandinsky6/diffusion_models/kandinsky6_pro_distill_5s_int8_convrot.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b96fc841b9ecd1d96218b70ecffd846ba8b620fa24d087fc99a547fbdc4edf9
+size 34905129600

models/kandinsky6/diffusion_models/kandinsky6_pro_distill_5s_w6a8.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d26c82be04d5e2d97f4bcf184f0af09768f3af92ade3b7fff52ee181f0a0b968
+size 29349022544