| --- |
| license: apache-2.0 |
| base_model: |
| - Qwen/Qwen3-32B |
| pipeline_tag: text-generation |
| tags: |
| - comfyui |
| - qwen3 |
| - int8 |
| - w4a8 |
| - w6a8 |
| - convrot |
| - text-generation |
| --- |
| |
| # Language models for ComfyUI |
|
|
| Language models converted to one ComfyUI file each, quantized in ComfyUI's own formats, for |
| writing text inside a graph: prompt writing, chat and retrieval. Load them with the **Language |
| Model Loader** node from [WAS Node Suite](https://github.com/WASasquatch/was-node-suite-comfyui), |
| which hands a CLIP to core **Generate Text**, **Fast Generate Text** and **LLM From CLIP**. |
|
|
| ## Files |
|
|
| Put them in `ComfyUI/models/text_encoders`. |
|
|
| | Model | File | Size | Weights | Writing speed, RTX 4090 24 GB | |
| |---|---|--:|---|--:| |
| | [Qwen3 32B](https://huggingface.co/Qwen/Qwen3-32B) | [`Qwen3-32B/qwen3_32b_w4a8.safetensors`](Qwen3-32B/qwen3_32b_w4a8.safetensors) | 19.1 GB | 4-bit weights in groups of 16, 8-bit activations | 27.6 tokens/s | |
| | | [`Qwen3-32B/qwen3_32b_w6a8.safetensors`](Qwen3-32B/qwen3_32b_w6a8.safetensors) | 26.0 GB | 6-bit weights in groups of 32, 8-bit activations | 3.3 tokens/s | |
| | | [`Qwen3-32B/qwen3_32b_int8_convrot.safetensors`](Qwen3-32B/qwen3_32b_int8_convrot.safetensors) | 32.8 GB | 8-bit weights | 1.5 tokens/s | |
|
|
| Speeds are Fast Generate Text's graph decode, greedy, with ComfyUI's `--reserve-vram 2`. A file |
| that fits on the graphics card runs from it; a larger one streams part of its weights from system |
| memory for every token, which is why w6a8 and int8 are slow on a 24 GB card. On a 32 GB card the |
| w6a8 file fits. Licence for all three: [Apache-2.0](Qwen3-32B/LICENSE), |
| [notice](Qwen3-32B/NOTICE.md). |
|
|
| | | w4a8 | w6a8 | int8 convrot | |
| |---|--:|--:|--:| |
| | Mean weight error of the layer linears | 7.3% | 2.3% | 0.91% | |
|
|
| ## Qwen3 32B |
|
|
| Qwen's dense 32B model with its own output layer, all 64 layers, 40,960 tokens of context, and |
| the hybrid thinking switch: Generate Text's `thinking` input turns reasoning on, and off it answers |
| straight away. All three files gave the same greedy answer to the same question. |
|
|
| - Every layer's linears are quantized, each row rotated by a 256-wide Hadamard block before |
| rounding (ConvRot, as Comfy-Org quantizes its Qwen text encoders). The embedding and the output |
| layer are int8 in every file. Norms stay bfloat16. |
| - Tensor names are Qwen's own (`model.layers.0.self_attn.q_proj.weight`, `lm_head.weight`), with |
| a `comfy_quant` tag beside each quantized layer. |
| - In the w4a8 and w6a8 files each layer's `mlp.down_proj`, 25,600 inputs wide, is stored as two |
| 12,800-wide column halves, `mlp.down_proj.parts.0` and `mlp.down_proj.parts.1`, whose outputs |
| add up to the layer's. ComfyUI's 4-bit and 6-bit kernels take a layer up to 17,408 inputs wide. |
| Language Model Loader adds the halves back up. |
| - ComfyUI's **Load CLIP** does not read a 32B Qwen3: it takes a 5120-wide Qwen3 for Mistral 24B. |
| **Language Model Loader** builds it on ComfyUI's own Qwen3 model. |
| - w4a8 and w6a8 need an RTX 30 series card or newer; int8 convrot an RTX 20 series or newer. |
| - It reads text only. MiniMax H3's `qwen3vl_32b` text encoder is a different file: the first 50 |
| layers of Qwen3-VL 32B with no output layer, which conditions H3 and cannot write text. |
|
|
| Tested with ComfyUI 0.42. |
|
|
| ## License |
|
|
| Qwen3 32B is by the Qwen team at Alibaba Cloud and is licensed under the Apache License 2.0, a copy |
| of which is in [`Qwen3-32B/LICENSE`](Qwen3-32B/LICENSE). The files here are modified from |
| [Qwen/Qwen3-32B](https://huggingface.co/Qwen/Qwen3-32B) by WAS: what changed is listed in |
| [`Qwen3-32B/NOTICE.md`](Qwen3-32B/NOTICE.md). |
|
|
