Papers
arxiv:2611.05648

NitroFusion2: Bringing Adversarial Loss to the Text Stream of Multimodal Diffusion Transformers

Published on Nov 3
· Submitted by
Chen Dar-Yen
on Nov 5
Authors:

Abstract

Multimodal diffusion transformers use text-stream adversarial distillation and aspect-conditioned discrimination to achieve high-quality one- to two-step image generation.

Adversarial diffusion distillation has conventionally relied on image features for discrimination. We revisit this choice for multimodal diffusion transformers (MMDiTs), whose text stream becomes image-dependent through joint attention. We introduce Text-Stream Adversarial Distillation (TSAD), which exploits intermediate text-token states as a new adversarial representation. Because the text stream has already read the image while remaining tied to the text condition, changing that condition changes what information it extracts from the image. Building on this property, we introduce Aspect-Cued Discrimination (ACD), which conditions the discriminator on different aspects of the sample, such as typography, objects, or style, to steer which properties are emphasized during adversarial training. Building on the expert-discriminator design of NitroFusion, we integrate TSAD and ACD in NitroFusion2. NitroFusion2 brings few-step distillation from the conventional 4-8-step regime down to just 1-2 steps at comparable quality. At one step, it produces sharp rendered text and rich fine-grained details, making one-step generation practical rather than merely a stress test for distillation. At two steps, it remains competitive across the evaluated benchmarks. These results establish the MMDiT text stream as a powerful and steerable representation for adversarial diffusion distillation.

Community

Paper author Paper submitter

Hi, first author here.

In an MMDiT, text and image tokens share every attention layer, so by mid-network the text tokens have already read the image. We use these intermediate text-token states of a frozen MMDiT as the discriminator's representation (Text-Stream Adversarial Distillation). In controlled ablations, text-stream discrimination outperforms conventional image-stream discrimination, and combining both streams is best on most metrics.

Because the discriminator reads the text stream, its text condition also steers what it judges: swapping the cue (typography, objects, style, …) turns it into an expert on that aspect, with aspect-specific gains (Aspect-Cued Discrimination).

Built on Z-Image-Turbo, NitroFusion2 brings few-step distillation from 4–8 steps down to 1–2 steps.

Questions and feedback welcome!
If you find NitroFusion2 interesting or useful, an upvote here or a star on GitHub helps more people find it. Thank you🙏

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2611.05648
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 2

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.05648 in a dataset README.md to link it from this page.

Spaces citing this paper 2

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.