| --- |
| language: |
| - en |
| tags: |
| - image-generation |
| - diffusion |
| - latent-alignment |
| - model-composition |
| pipeline_tag: text-to-image |
| --- |
| |
| <p align="center"> |
| <img src="fig/instantfusion-banner.png" alt="InstantFusion — A Shared Latent Interface for Image Generators" width="100%"> |
| </p> |
|
|
| <p align="center"> |
| <strong>Composing diffusion models through a shared latent space</strong> |
| </p> |
|
|
| <p align="center"> |
| <a href="https://github.com/InternLM/InstantFusion">GitHub</a> · |
| <a href="https://internlm.github.io/InstantFusion/">Project Page</a> · |
| <a href="https://huggingface.co/papers/2610.13944">Paper</a> |
| </p> |
|
|
| ## Overview |
|
|
| InstantFusion connects heterogeneous image generators through a learned shared latent interface. Model-specific Latent AutoEncoders (LAEs) align intermediate states while preserving the information needed for reconstruction and continued denoising. The base generators remain frozen during LAE training. |
|
|
| The shared interface supports generation acceleration, multi-preference composition, and cross-model on-policy distillation (OPD). |
|
|
| <p align="center"> |
| <img src="fig/method.png" alt="InstantFusion training and inference overview" width="100%"> |
| </p> |
|
|
| ## Released checkpoints |
|
|
| This repository provides trained LAE interface weights. The base generators must be obtained separately. |
|
|
| | Checkpoint | Connected generators | |
| | --- | --- | |
| | [sd3_qwen_lae.bin](sd3_qwen_lae.bin) | SD3 and Qwen-Image | |
| | [flux1_dev_qwen_lae.bin](flux1_dev_qwen_lae.bin) | FLUX.1-dev and Qwen-Image | |
| | [flux1_schnell_qwen_lae.bin](flux1_schnell_qwen_lae.bin) | FLUX.1-schnell and Qwen-Image | |
| | [flux1_sd3_qwen_lae.bin](flux1_sd3_qwen_lae.bin) | FLUX.1, SD3, and Qwen-Image | |
| | [flux2_dev_qwen_lae.bin](flux2_dev_qwen_lae.bin) | FLUX.2-dev and Qwen-Image | |
| | [flux2_klein_qwen_lae.bin](flux2_klein_qwen_lae.bin) | FLUX.2-Klein and Qwen-Image | |
| | [zimage_turbo_qwen_lae.bin](zimage_turbo_qwen_lae.bin) | Z-Image-Turbo and Qwen-Image | |
|
|
| ## Visual examples |
|
|
| ### Latent reconstruction |
|
|
| LAE reconstruction examples for Qwen-Image, FLUX.1, and SD3. Mean SSIM is reported over 100 images for each model. |
|
|
| <p align="center"> |
| <img src="fig/reconstruct.png" alt="LAE reconstruction examples and mean SSIM for Qwen-Image, FLUX.1, and SD3" width="100%"> |
| </p> |
|
|
| ### Cross-model denoising |
|
|
| Intermediate states can be transferred through the shared space so that another generator can continue denoising. The examples below show cross-model generation with SD3, FLUX.1, and Qwen-Image. |
|
|
| <p align="center"> |
| <img src="fig/trans.png" alt="Cross-model handoff and continued denoising examples" width="100%"> |
| </p> |
|
|
| For implementation details, supported inference configurations, and training instructions, visit the [InstantFusion GitHub repository](https://github.com/InternLM/InstantFusion). |
