FLUX.2-dev-MXFP8-RTN-FullEncoder-AutoRound

Model Details

This is an MXFP8 W8A8 quantization of black-forest-labs/FLUX.2-dev, generated with AutoRound using zero-shot RTN. Please follow the license of the original model.

Unlike the transformer-only FLUX.2-dev-MXFP8-RTN-AutoRound, this variant quantizes both the diffusion transformer blocks and the full multimodal encoder.

• Quantized: diffusion transformer blocks (192 Linear layers), encoder language layers (280), vision tower layers (168), and multimodal projector (3).
• Kept in BF16: VAE, embeddings, norms, encoder lm_head, vision patch convolution, and non-block transformer projections, including modulation/output.
• Settings: group size 32, dynamic activation quantization, iters=0, model_free=True, disable_opt_rtn=True. No calibration or smoothing.
• AutoRound commit: 8869842f9741538ea729849d5c23592bdb605e61.

How to Use

Run the following commands from the parent of the downloaded model directory. Name the downloaded model directory FLUX.2-dev-MXFP8-RTN-FullEncoder-AutoRound. The bundled scripts handle component quantization configuration and checkpoint index layout.

Text-to-Image — vLLM-Omni

Evaluated environment: vLLM 0.32.0, vLLM-Omni 0.32.0, Transformers 5.19, Diffusers 0.40, PyTorch 2.13+cu130 on NVIDIA B200.

CUDA_VISIBLE_DEVICES=0 python FLUX.2-dev-MXFP8-RTN-FullEncoder-AutoRound/examples/text_to_image.py \
    --model FLUX.2-dev-MXFP8-RTN-FullEncoder-AutoRound \
    --prompt "A red colored car." \
    --output output.png

Defaults: 1024 × 1024, 50 steps, guidance 4.0, seed 42. The script sets CUDA_LAUNCH_BLOCKING=1, uses component-specific configs, and hides stale duplicate index files in a temporary runtime view.

Image Question Answering — Transformers + AutoRound

Evaluated environment: Transformers 5.14.1, PyTorch 2.13 and the AutoRound commit above, with the auto_round:torch_mxfp8 backend.

CUDA_VISIBLE_DEVICES=0 python FLUX.2-dev-MXFP8-RTN-FullEncoder-AutoRound/examples/scienceqa.py \
    --model FLUX.2-dev-MXFP8-RTN-FullEncoder-AutoRound \
    --index 0

Loads one question from ScienceQA-IMG-Test-2017 and selects an answer using next-token logits restricted to the valid option letters. This is forced-choice inference, not free-form generation.

Backend note: ScienceQA was evaluated with Transformers, not vLLM-Omni. The verified Omni T2I encoder skips the vision tower and projector; a full vLLM-Omni multimodal question-answering recipe has not been validated here. The Transformers script explicitly enables all 451 quantized encoder Linear layers, including the projector omitted by the serialized block list.

Evaluation

Text-to-Image — DrawBench (200 Images)

Model	CLIP	CLIP-IQA	ImageReward
BF16	32.5236	71.7570	1.1716
Upstream transformer-only MXFP8	32.4859	72.0392	1.1779
Transformer + language-only encoder MXFP8	32.4436	71.9781	1.1547
Full encoder MXFP8 (this model)	32.4436	71.9781	1.1547

Same prompts, 1024 × 1024, 50 steps, guidance 4.0, seed 42 + image ID. Metrics follow the neural-compressor PR2625 example: CLIP ViT-B/32 image-text logits, three-pair CLIP-IQA (×100), and ImageReward-v1.0.

Compared with BF16, CLIP decreases by 0.25%, CLIP-IQA increases by 0.31%, and ImageReward decreases by 1.44%. The full/language-only encoder scores match at four decimal places; pixel-level equality was not checked. Quantizing the visual modules does not accelerate the current Omni T2I path, which does not use them.

Multimodal Encoder — ScienceQA (2,017 Image Questions)

Encoder	Correct / Total	Accuracy
BF16	1,841 / 2,017	91.2742%
Full encoder MXFP8	1,842 / 2,017	91.3237%

The full image-only test split was evaluated with identical image, question, options and hint inputs. Answers, lectures and solutions were not model inputs. All 171 visual/projector Linear layers executed for every question with finite outputs, and all 451 quantized encoder layers loaded without key mismatches.

Relative accuracy loss is −0.0543%, meeting the loss < 1% criterion on this task. One additional correct answer is not a significant improvement or evidence of universal losslessness. ImageReward does not meet a per-metric 1% loss threshold. An earlier 200-question subset gave 92.5% BF16 versus 90.5% MXFP8; the complete split is the primary result.

Data: changwangss/ScienceQA-IMG-Test-2017, revision d21962594f7ae2cd2819fe48e95903d8a3aef5a5. Source: lmms-lab-encoder/ScienceQA, ScienceQA-IMG/test, revision 69dd4d6b67373d38f96a5badd5d24d0eb5bcdc50. No sampling or filtering.

Publication Notes

All pipeline assets are included as regular files, together with the two tested example scripts. No external filesystem symlinks are required.

Limitations and License

Results apply to the stated backends and evaluation protocols. Free-form VQA, image editing and other backends have not been validated. Generated content may be inaccurate; downstream applications require appropriate quality and safety testing.

Follow the original FLUX.2-dev license. Quantization grants no additional rights. ScienceQA is separately distributed under CC BY-NC-SA 4.0.