Blame for README.md

---
base_model: black-forest-labs/FLUX.2-dev
pipeline_tag: text-to-image
library_name: diffusers
license: other
license_name: flux-2-dev-non-commercial-license
license_link: https://huggingface.co/black-forest-labs/FLUX.2-dev/blob/main/LICENSE.md
tags:
- auto-round
- mxfp8
- rtn
- multimodal
---

# FLUX.2-dev-MXFP8-RTN-FullEncoder-AutoRound

## Model Details

This is an **MXFP8 W8A8** quantization of
[black-forest-labs/FLUX.2-dev](https://huggingface.co/black-forest-labs/FLUX.2-dev),
generated with [AutoRound](https://github.com/intel/auto-round) using zero-shot
RTN. Please follow the license of the original model.

Unlike the transformer-only `FLUX.2-dev-MXFP8-RTN-AutoRound`, this variant
quantizes both the diffusion transformer blocks and the full multimodal encoder.

- **Quantized:** diffusion transformer blocks (192 Linear layers), encoder
  language layers (280), vision tower layers (168), and multimodal projector (3).
- **Kept in BF16:** VAE, embeddings, norms, encoder lm_head, vision patch
  convolution, and non-block transformer projections, including modulation/output.
- **Settings:** group size 32, dynamic activation quantization, `iters=0`,
  `model_free=True`, `disable_opt_rtn=True`. No calibration or smoothing.
- **AutoRound commit:** `8869842f9741538ea729849d5c23592bdb605e61`.

## How to Use

Run the following commands from the parent of the downloaded model directory.
Name the downloaded model directory `FLUX.2-dev-MXFP8-RTN-FullEncoder-AutoRound`. The bundled
scripts handle component quantization configuration and checkpoint index layout.

### Text-to-Image — vLLM-Omni

Evaluated environment: vLLM 0.32.0, vLLM-Omni 0.32.0, Transformers 5.19,
Diffusers 0.40, PyTorch 2.13+cu130 on NVIDIA B200.

```bash
CUDA_VISIBLE_DEVICES=0 python FLUX.2-dev-MXFP8-RTN-FullEncoder-AutoRound/examples/text_to_image.py \
    --model FLUX.2-dev-MXFP8-RTN-FullEncoder-AutoRound \
    --prompt "A red colored car." \
    --output output.png
```

Defaults: 1024 × 1024, 50 steps, guidance 4.0, seed 42. The script sets
`CUDA_LAUNCH_BLOCKING=1`, uses component-specific configs, and hides stale
duplicate index files in a temporary runtime view.

### Image Question Answering — Transformers + AutoRound

Evaluated environment: Transformers 5.14.1, PyTorch 2.13 and the AutoRound commit
above, with the `auto_round:torch_mxfp8` backend.

```bash
CUDA_VISIBLE_DEVICES=0 python FLUX.2-dev-MXFP8-RTN-FullEncoder-AutoRound/examples/scienceqa.py \
    --model FLUX.2-dev-MXFP8-RTN-FullEncoder-AutoRound \
    --index 0
```

Loads one question from
[ScienceQA-IMG-Test-2017](https://huggingface.co/datasets/changwangss/ScienceQA-IMG-Test-2017)
and selects an answer using next-token logits restricted to the valid option
letters. This is forced-choice inference, not free-form generation.

**Backend note:** ScienceQA was evaluated with Transformers, not vLLM-Omni.
The verified Omni T2I encoder skips the vision tower and projector; a full
vLLM-Omni multimodal question-answering recipe has not been validated here.
The Transformers script explicitly enables all 451 quantized encoder Linear
layers, including the projector omitted by the serialized block list.

## Evaluation

### Text-to-Image — DrawBench (200 Images)

| Model | CLIP | CLIP-IQA | ImageReward |
|---|---:|---:|---:|
| BF16 | 32.5236 | 71.7570 | 1.1716 |
| Upstream transformer-only MXFP8 | 32.4859 | 72.0392 | 1.1779 |
| Transformer + language-only encoder MXFP8 | 32.4436 | 71.9781 | 1.1547 |
| **Full encoder MXFP8 (this model)** | **32.4436** | **71.9781** | **1.1547** |

Same prompts, 1024 × 1024, 50 steps, guidance 4.0, seed 42 + image ID.
Metrics follow the neural-compressor PR2625 example: CLIP ViT-B/32 image-text
logits, three-pair CLIP-IQA (×100), and ImageReward-v1.0.

Compared with BF16, CLIP decreases by 0.25%, CLIP-IQA increases by 0.31%, and
ImageReward decreases by 1.44%. The full/language-only encoder scores match at
four decimal places; pixel-level equality was not checked. Quantizing the visual
modules does not accelerate the current Omni T2I path, which does not use them.

### Multimodal Encoder — ScienceQA (2,017 Image Questions)

| Encoder | Correct / Total | Accuracy |
|---|---:|---:|
| BF16 | 1,841 / 2,017 | 91.2742% |
| **Full encoder MXFP8** | **1,842 / 2,017** | **91.3237%** |

The full image-only test split was evaluated with identical image, question,
options and hint inputs. Answers, lectures and solutions were not model inputs.
All 171 visual/projector Linear layers executed for every question with finite
outputs, and all 451 quantized encoder layers loaded without key mismatches.

Relative accuracy loss is **−0.0543%**, meeting the **loss < 1%** criterion on
this task. One additional correct answer is not a significant improvement or
evidence of universal losslessness. ImageReward does not meet a per-metric 1%
loss threshold. An earlier 200-question subset gave 92.5% BF16 versus 90.5% MXFP8;
the complete split is the primary result.

Data: [changwangss/ScienceQA-IMG-Test-2017](https://huggingface.co/datasets/changwangss/ScienceQA-IMG-Test-2017),
revision `d21962594f7ae2cd2819fe48e95903d8a3aef5a5`.
Source: `lmms-lab-encoder/ScienceQA`, ScienceQA-IMG/test,
revision `69dd4d6b67373d38f96a5badd5d24d0eb5bcdc50`. No sampling or filtering.

## Publication Notes

All pipeline assets are included as regular files, together with the two tested
example scripts. No external filesystem symlinks are required.

## Limitations and License

Results apply to the stated backends and evaluation protocols. Free-form VQA,
image editing and other backends have not been validated. Generated content may
be inaccurate; downstream applications require appropriate quality and safety testing.

Follow the [original FLUX.2-dev license](https://huggingface.co/black-forest-labs/FLUX.2-dev/blob/main/LICENSE.md).
Quantization grants no additional rights. ScienceQA is separately distributed
under [CC BY-NC-SA 4.0](https://scienceqa.github.io/).