---
license: apache-2.0
base_model:
- Qwen/Qwen3-VL-8B-Instruct
base_model_relation: quantized
pipeline_tag: image-text-to-text
tags:
- fp8
- compressed-tensors
- qwen-image
- text-encoder
- qwen3-vl
---

# Qwen-Image-2.1 Text Encoder: FP8

FP8 build of the text encoder shipped in [Qwen/Qwen-Image-2.1](https://huggingface.co/Qwen/Qwen-Image-2.1)
(`text_encoder/`), 9.9 GB instead of 16.3 GB. That encoder is stock
[Qwen/Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct): all 750 tensors match, and every sampled
tensor is byte-identical.

## Format

[compressed-tensors](https://github.com/neuralmagic/compressed-tensors) `float-quantized`, scheme `FP8_DYNAMIC`, made
with llm-compressor 0.13.0 (no calibration needed):

| Part | Precision |
|---|---|
| Every Linear layer of the language model (attention q/k/v/o, MLP gate/up/down) | FP8 E4M3 weights (`.weight`) + one bf16 scale per output channel (`.weight_scale`, shape `[out, 1]`); activations dynamic per-token FP8 when run as W8A8 |
| Embeddings, `lm_head`, norms, the whole vision tower | bf16, unchanged |

For a weight-only loader, the dequantized weight is simply `weight.to(bf16) * weight_scale`.

## How it was tested

For a diffusion text encoder, what matters is what the image model reads: the **last decoder layer's hidden
states before the final norm**, from the pipeline's own prompt templates. This was measured through diffusers'
`QwenImage21Pipeline.encode_prompt` on held-out prompts (12 text-to-image, 8 image-edit on 2 held-out images),
token by token against the bf16 original:

| How it runs | Mean token cosine vs bf16 | Relative L2 error |
|---|---|---|
| Weight-only (FP8 weights dequantized, bf16 compute) | 0.9746 | 0.168 |
| W8A8 (transformers + compressed-tensors, FP8 activations) | 0.9572 | 0.195 |

Same-seed 1024px images kept the layout of the bf16 ones, and rendered text stayed correct. For reference, GGUF
Q8_0 measures 0.9938 / 0.070 on the same test: E4M3's 3-bit mantissa is coarser than 8-bit integers with a scale
per 32 weights. If you need the closest match to bf16, use the Q8_0 in
[darrellbest/Qwen-Image-2.1-Text-Encoder-GGUF](https://huggingface.co/darrellbest/Qwen-Image-2.1-Text-Encoder-GGUF).

## Use

```python
from transformers import Qwen3VLForConditionalGeneration
te = Qwen3VLForConditionalGeneration.from_pretrained("darrellbest/Qwen-Image-2.1-Text-Encoder-FP8")
```

transformers decompresses the weights to bf16 on load (needs `compressed-tensors`), so this saves disk and
download, not VRAM. A loader that keeps the FP8 weights (see Format) is what saves VRAM.

License: Apache-2.0, the license of Qwen3-VL-8B-Instruct. (The rest of Qwen-Image-2.1 is under the Qwen Research
License; this repository contains only the encoder.)