moonbox-dev/FLUX.2-klein-4B-ONNX

FLUX.2 Klein 4B ONNX

ONNX builds of FLUX.2 Klein 4B, made by Moonbox for on-device use with ONNX Runtime (CUDA, DirectML, CPU). Each precision is a folder with one graph per part of the pipeline; large weights sit next to each graph as ONNX external data (*.onnx.data), so keep a folder's files together.

Files

The last column is how far each graph's output is from the original PyTorch model run in fp32 on the same inputs (relative RMS error). fp32 is the yardstick because the original's own bf16 run is not exact either. Image-quality scores are added as each precision passes the image eval.

Precision	File	Size	sha256	Checked against fp32
fp16	fp16/text_encoder/model.onnx	0.00 GB	ea01d40937ab	0.29 %
fp16	fp16/text_encoder/model.onnx.data	6.23 GB	a87ba8455940	(weights of the row above)
fp16	fp16/transformer/model.onnx	0.01 GB	6fa7ef9b7909	0.87 %
fp16	fp16/transformer/model.onnx.data	7.75 GB	f17c1c03ae93	(weights of the row above)
fp16	fp16/vae_decoder/model.onnx	0.00 GB	620711cf8e15	0.11 %
fp16	fp16/vae_decoder/model.onnx.data	0.10 GB	bd3bade0e4f9	(weights of the row above)
fp16	fp16/vae_encoder/model.onnx	0.00 GB	14537e5b607c	0.10 %
fp16	fp16/vae_encoder/model.onnx.data	0.07 GB	89b409a0956f	(weights of the row above)
bf16	bf16/text_encoder/model.onnx	0.00 GB	f518c2b09e8c	1.9 %
bf16	bf16/text_encoder/model.onnx.data	6.23 GB	b79d1069f154	(weights of the row above)
bf16	bf16/transformer/model.onnx	0.00 GB	f211d2e5a4f7	4.2 %
bf16	bf16/transformer/model.onnx.data	7.75 GB	d4c2cf7e07dc	(weights of the row above)
bf16	bf16/vae_decoder/model.onnx	0.00 GB	620711cf8e15	0.11 % (fp16 graph)
bf16	bf16/vae_decoder/model.onnx.data	0.10 GB	bd3bade0e4f9	(weights of the row above)
bf16	bf16/vae_encoder/model.onnx	0.00 GB	14537e5b607c	0.10 % (fp16 graph)
bf16	bf16/vae_encoder/model.onnx.data	0.07 GB	89b409a0956f	(weights of the row above)
int8	int8/text_encoder/model.onnx	0.00 GB	9105098d8a45	1.8 %
int8	int8/text_encoder/model.onnx.data	3.67 GB	f08c2364a203	(weights of the row above)
int8	int8/transformer/model.onnx	0.00 GB	2782451f5d71	2.5 %
int8	int8/transformer/model.onnx.data	4.28 GB	094e10544e8e	(weights of the row above)
int8	int8/vae_decoder/model.onnx	0.00 GB	620711cf8e15	0.11 % (fp16 graph)
int8	int8/vae_decoder/model.onnx.data	0.10 GB	bd3bade0e4f9	(weights of the row above)
int8	int8/vae_encoder/model.onnx	0.00 GB	14537e5b607c	0.10 % (fp16 graph)
int8	int8/vae_encoder/model.onnx.data	0.07 GB	89b409a0956f	(weights of the row above)
int4	int4/text_encoder/model.onnx	0.00 GB	86d5368cc146	13 %
int4	int4/text_encoder/model.onnx.data	2.35 GB	e7e2c5ebec18	(weights of the row above)
int4	int4/transformer/model.onnx	0.00 GB	63c197f6719a	30 %
int4	int4/transformer/model.onnx.data	2.48 GB	f85c0804458d	(weights of the row above)
int4	int4/vae_decoder/model.onnx	0.00 GB	620711cf8e15	0.11 % (fp16 graph)
int4	int4/vae_decoder/model.onnx.data	0.10 GB	bd3bade0e4f9	(weights of the row above)
int4	int4/vae_encoder/model.onnx	0.00 GB	14537e5b607c	0.10 % (fp16 graph)
int4	int4/vae_encoder/model.onnx.data	0.07 GB	89b409a0956f	(weights of the row above)

How to run

Each precision folder is the whole FLUX.2 [klein] 4B pipeline as four ONNX graphs; the tokenizer and the scheduler settings are the original ones from black-forest-labs/FLUX.2-klein-4B.

Graph	Inputs	Output
text_encoder/model.onnx	input_ids, attention_mask [B, S] int64 (the Qwen3 chat template, padded to 512)	prompt_embeds [B, S, 7680]
transformer/model.onnx	hidden_states [B, N, 128], encoder_hidden_states [B, S, 7680], timestep [B] (t / 1000), img_ids [B, N, 4], txt_ids [B, S, 4]	noise_pred [B, N, 128]
vae_encoder/model.onnx	image [B, 3, H, W] in [-1, 1]	latents [B, 32, H/8, W/8]
vae_decoder/model.onnx	latents [B, 32, h, w]	image [B, 3, 8h, 8w] in [-1, 1]

The graphs take exactly what diffusers' Flux2KleinPipeline passes to its text encoder, transformer and VAE, so the pipeline's own tokenizing, latent packing, position ids and flow-matching schedule drive them unchanged. The text encoder graph holds Qwen3's first 27 layers: the pipeline stacks the hidden states before layers 9, 18 and 27, so the later layers never matter.

import onnxruntime as ort
providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
transformer = ort.InferenceSession("fp16/transformer/model.onnx", providers=providers)
noise_pred = transformer.run(None, {"hidden_states": ..., "encoder_hidden_states": ..., "timestep": ...,
                                    "img_ids": ..., "txt_ids": ...})[0]

Which folder. Whole pictures, the full pipeline per folder against the original pipeline run in fp32 (512x512, 4 steps, 3 prompts, same seeds; PSNR, higher is closer):

Build	PSNR vs fp32
fp16	31.7 dB	closest to the original; start here on NVIDIA
original pipeline in bf16 (PyTorch)	27.3 dB	how the model is usually run, for scale
int8	27.2 dB	as close as the usual bf16 run, half the size
bf16	24.8 dB	bf16 arithmetic, for providers that prefer it
int4	19.1 dB	sharp, same scene and subject, but the sample drifts (pose, details)
another seed (fp32)	12.7 dB	a different picture altogether

int8 and int4 quantize the text encoder's and transformer's matrix weights (ONNX Runtime MatMulNBits, weight-only: int8 symmetric blocks of 32, int4 asymmetric blocks of 32) and keep the VAE in fp16; bf16 also uses the fp16 VAE (ONNX has no bf16 convolution). Every 16-bit softmax and SiLU in the graphs runs in fp32 inside the graph, as PyTorch does: ONNX Runtime's CUDA QuickGelu fusion of SiLU is otherwise not accurate enough for the Qwen3 text encoder in fp16. Pass inputs as contiguous arrays (bf16 inputs as raw bits in an OrtValue).

How these were made

From FLUX.2 Klein 4B at revision e7b7dc27f9: each part exported with torch.onnx, then converted or quantized with ONNX Runtime's tools (fp16 / bf16 casts, int8 and int4 weight quantization). Every graph was checked against the original before upload; the sha256 column above is what the files must hash to.

License

Apache-2.0. These files are a derivative of the original model and carry its license: it is copied in this repository, and it governs your use of these files. The original model card follows unchanged.

---

Original model card

The FLUX.2 [klein] model family are our fastest image models to date. FLUX.2 [klein] unifies generation and editing in a single compact architecture, delivering state-of-the-art quality with end-to-end inference in as low as under a second. Built for applications that require real-time image generation without sacrificing quality, and runs on consumer hardware, with as little as 13GB VRAM.

FLUX.2 [klein] 4B is a 4 billion parameter rectified flow transformer capable of generating images from text descriptions and supports multi-reference editing capabilities.

Fully open under Apache 2.0. Our most accessible model runs on consumer GPUs like the RTX 3090/4070. Compact but capable: supports text-to-image, image editing, and multi-reference at quality that punches above its size. Built for local development, edge deployment, and production use.

For more information, please read our blog post.

Key Features

1. Our fastest distilled model for sub-second image generation.
2. Best suited for interactive workflows, production deployments, and latency-critical applications.
3. Text-to-image and image-to-image multi-reference editing in a single unified model.
4. Runs on consumer GPUs (~13GB VRAM).
5. Open weights available for commercial use under the Apache 2.0 license.

Usage

We provide a reference implementation of FLUX.2 [klein] 4B, as well as sampling code, in a dedicated GitHub repository. Developers and creatives looking to build on top of FLUX.2 [klein] 4B are encouraged to use this as a starting point.

API Endpoints

The FLUX.2 [klein] 4B model is available via the BFL API:

• bfl.ai

FLUX.2 [klein] 4B is also available in both ComfyUI and Diffusers.

Using with Diffusers 🧨

To use FLUX.2 [klein] 4B with the 🧨 Diffusers python library, first install or upgrade diffusers:

pip install git+https://github.com/huggingface/diffusers.git

Then you can use Flux2KleinPipeline to run the model:

import torch
from diffusers import Flux2KleinPipeline

device = "cuda"
dtype = torch.bfloat16

pipe = Flux2KleinPipeline.from_pretrained("black-forest-labs/FLUX.2-klein-4B", torch_dtype=dtype)
pipe.enable_model_cpu_offload()  # save some VRAM by offloading the model to CPU

prompt = "A cat holding a sign that says hello world"
image = pipe(
    prompt=prompt,
    height=1024,
    width=1024,
    guidance_scale=1.0,
    num_inference_steps=4,
    generator=torch.Generator(device=device).manual_seed(0)
).images[0]
image.save("flux-klein.png")

---

Limitations

• This model is not intended or able to provide factual information.
• While the model can output text, text rendered may be inaccurate or subject to distortion.
• As a statistical model, this checkpoint may represent or amplify biases observed in the training data.
• The model may fail to generate output that matches the prompts.
• Prompt following is heavily influenced by the prompting style.

Out-of-Scope Use

The model and its derivatives may not be used:

• In any way that violates applicable law.
• For the purpose of exploiting, harming or attempting to exploit or harm minors in any way; including but not limited to the solicitation, creation, acquisition, or dissemination of child exploitative content.
• To generate or disseminate deceptive, fraudulent, misleading or otherwise harmful content.
• To generate or disseminate personal identifiable information that can be used to harm an individual.
• To harass, abuse, threaten, stalk, or bully individuals or groups of individuals.
• To create non-consensual intimate imagery or illegal pornographic content.
• For fully automated decision making or high risk applications that adversely impact an individual's legal rights or otherwise create or modify a binding, enforceable obligation.

Nothing contained in this Model Card should be interpreted as or deemed a restriction or modification to the license the model is released under.

Hardware

The FLUX.2 [klein] 4B model fits in ~13GB VRAM and is accessible on NVIDIA RTX 3090/4070 and above.

---

Responsible AI Development

Black Forest Labs is committed to the responsible development and deployment of our models. Prior to releasing the FLUX.2 family of models, we evaluated and mitigated a number of risks in our model checkpoints and hosted services, including the generation of unlawful content, including child sexual abuse material (CSAM) and nonconsensual intimate imagery (NCII). We implemented a series of pre-release mitigations to help prevent misuse by third parties, with additional post-release mitigations to help address residual risks:

1. Pre-training mitigation. We filtered pre-training data for multiple categories of "not safe for work" (NSFW) and known child sexual abuse material (CSAM) to help prevent a user generating unlawful content in response to text prompts or uploaded images. We have partnered with the https://www.iwf.org.uk/, an independent nonprofit organization dedicated to preventing online abuse, to filter known CSAM from the training data.
2. Post-training mitigation. Subsequently, we undertook multiple rounds of targeted fine-tuning to provide additional mitigation against potential abuse, including both text-to-image (T2I) and image-to-image (I2I) attacks. By inhibiting certain behaviors and suppressing certain concepts in the trained model, these techniques can help to prevent a user generating synthetic CSAM or NCII from a text prompt, or transforming an uploaded image into synthetic CSAM or NCII.
3. Ongoing evaluation. Throughout this process, we conducted multiple internal and external third-party evaluations of model checkpoints to identify further opportunities for mitigation. External third-party evaluations focused on eliciting CSAM and NCII through adversarial testing with (i) text-only prompts, (ii) a single uploaded reference image with text prompts, and (iii) multiple uploaded reference images with text prompts. Based on this feedback, we conducted further safety fine-tuning to produce our open-weight FLUX.2 [klein] models.
4. Release decision. After safety fine-tuning and prior to release, we conducted a final third-party evaluation of the proposed release checkpoints, focused on T2I and I2I generation of synthetic CSAM and NCII, including a comparison with other open-weight T2I and I2I models. The final FLUX.2 [klein] checkpoints demonstrated high resilience against violative inputs in complex generation and editing tasks, and demonstrated higher resilience than leading open-weight models across these risk categories. Based on these findings, we approved the release of the open-weight FLUX.2 [klein] 4B models under an Apache 2.0 license and the release of the FLUX.2 [klein] 9B models under a non-commercial license to support third-party research and development.
5. Inference filters. The repository for the FLUX.2 [klein] models includes filters for NSFW and protected content in inputs and outputs. Filters or manual review must be used with the FLUX.2 [klein] 9B models under the terms of the FLUX Non-Commercial License, and we encourage deployers to implement these mitigations when using the FLUX.2 [klein] 4B models. Where we implement these features on our own hosted services, we may apply multiple filters to intercept text prompts, uploaded images, and output images. We utilize both in-house and third-party filters to mitigate against harmful outputs, such as CSAM and NCII outputs, including filters provided by https://thehive.ai/ and https://www.microsoft.com/.
6. Content provenance. Content provenance features can help users and platforms better identify, label, and interpret AI-generated content online. The inference code for FLUX.2 [klein] implements an example of pixel-layer watermarking. Additionally, this repository includes links to the https://c2pa.org/ standard for metadata. The API for FLUX.2 [klein] applies cryptographically-signed C2PA metadata to downloaded output content to indicate that images were produced with our model.
7. Policies. Acceptable use of our models and access to our API are governed by policies set out in applicable documentation, including FLUX Non-Commercial License (for our non-commercial open-weight users); Developer Terms of Service, Self-Hosted Commercial License Terms, and Usage Policy (for our commercial open-weight model users); and Developer Terms of Service, FLUX API Service Terms, and Usage Policy (for our API users). These prohibit the generation of unlawful content or the use of generated content for unlawful, defamatory, or abusive purposes.
8. Safety. Black Forest Labs takes model safety seriously. We provide a dedicated email address (safety@blackforestlabs.ai) to solicit feedback from the community. We maintain a reporting relationship with organizations such as the https://www.iwf.org.uk/ and the https://www.missingkids.org/, and welcome ongoing engagement with authorities, developers, and researchers to share intelligence about emerging risks and develop effective mitigations.

---

License

This model is licensed under the https://www.apache.org/licenses/LICENSE-2.0.

Trademarks & IP

This project may contain trademarks or logos for projects, products, or services. Use of Black Forest Labs and FLUX trademarks or logos in modified versions of this project must not cause confusion or imply sponsorship or endorsement. Any use of third-party trademarks, intellectual property or logos are subject to those third-party's policies.