Blame for README.md

---
license: apache-2.0
base_model:
- Qwen/Qwen3-32B
pipeline_tag: text-generation
tags:
- comfyui
- qwen3
- int8
- w4a8
- w6a8
- convrot
- text-generation
---

# Language models for ComfyUI

Language models converted to one ComfyUI file each, quantized in ComfyUI's own formats, for
writing text inside a graph: prompt writing, chat and retrieval. Load them with the **Language
Model Loader** node from [WAS Node Suite](https://github.com/WASasquatch/was-node-suite-comfyui),
which hands a CLIP to core **Generate Text**, **Fast Generate Text** and **LLM From CLIP**.

## Files

Put them in `ComfyUI/models/text_encoders`.

| Model | File | Size | Weights | Writing speed, RTX 4090 24 GB |
|---|---|--:|---|--:|
| [Qwen3 32B](https://huggingface.co/Qwen/Qwen3-32B) | [`Qwen3-32B/qwen3_32b_w4a8.safetensors`](Qwen3-32B/qwen3_32b_w4a8.safetensors) | 19.1 GB | 4-bit weights in groups of 16, 8-bit activations | 27.6 tokens/s |
| | [`Qwen3-32B/qwen3_32b_w6a8.safetensors`](Qwen3-32B/qwen3_32b_w6a8.safetensors) | 26.0 GB | 6-bit weights in groups of 32, 8-bit activations | 3.3 tokens/s |
| | [`Qwen3-32B/qwen3_32b_int8_convrot.safetensors`](Qwen3-32B/qwen3_32b_int8_convrot.safetensors) | 32.8 GB | 8-bit weights | 1.5 tokens/s |

Speeds are Fast Generate Text's graph decode, greedy, with ComfyUI's `--reserve-vram 2`. A file
that fits on the graphics card runs from it; a larger one streams part of its weights from system
memory for every token, which is why w6a8 and int8 are slow on a 24 GB card. On a 32 GB card the
w6a8 file fits. Licence for all three: [Apache-2.0](Qwen3-32B/LICENSE),
[notice](Qwen3-32B/NOTICE.md).

| | w4a8 | w6a8 | int8 convrot |
|---|--:|--:|--:|
| Mean weight error of the layer linears | 7.3% | 2.3% | 0.91% |

## Qwen3 32B

Qwen's dense 32B model with its own output layer, all 64 layers, 40,960 tokens of context, and
the hybrid thinking switch: Generate Text's `thinking` input turns reasoning on, and off it answers
straight away. All three files gave the same greedy answer to the same question.

- Every layer's linears are quantized, each row rotated by a 256-wide Hadamard block before
  rounding (ConvRot, as Comfy-Org quantizes its Qwen text encoders). The embedding and the output
  layer are int8 in every file. Norms stay bfloat16.
- Tensor names are Qwen's own (`model.layers.0.self_attn.q_proj.weight`, `lm_head.weight`), with
  a `comfy_quant` tag beside each quantized layer.
- In the w4a8 and w6a8 files each layer's `mlp.down_proj`, 25,600 inputs wide, is stored as two
  12,800-wide column halves, `mlp.down_proj.parts.0` and `mlp.down_proj.parts.1`, whose outputs
  add up to the layer's. ComfyUI's 4-bit and 6-bit kernels take a layer up to 17,408 inputs wide.
  Language Model Loader adds the halves back up.
- ComfyUI's **Load CLIP** does not read a 32B Qwen3: it takes a 5120-wide Qwen3 for Mistral 24B.
  **Language Model Loader** builds it on ComfyUI's own Qwen3 model.
- w4a8 and w6a8 need an RTX 30 series card or newer; int8 convrot an RTX 20 series or newer.
- It reads text only. MiniMax H3's `qwen3vl_32b` text encoder is a different file: the first 50
  layers of Qwen3-VL 32B with no output layer, which conditions H3 and cannot write text.

Tested with ComfyUI 0.42.

## License

Qwen3 32B is by the Qwen team at Alibaba Cloud and is licensed under the Apache License 2.0, a copy
of which is in [`Qwen3-32B/LICENSE`](Qwen3-32B/LICENSE). The files here are modified from
[Qwen/Qwen3-32B](https://huggingface.co/Qwen/Qwen3-32B) by WAS: what changed is listed in
[`Qwen3-32B/NOTICE.md`](Qwen3-32B/NOTICE.md).