iris-3b / README.md

Jinstudio

lisperidlabs

Duplicate from speridlabs/iris-3b

10805d0 9 days ago

preview code

Raw History Blame Contribute Delete

15.7 kB

---
license: apache-2.0
library_name: pytorch
pipeline_tag: text-to-image
tags:
  - text-to-image
  - diffusion
  - pixel-space
  - flow-matching
  - general-vision-learner
  - vision-foundation-model
  - depth-estimation
  - image-restoration
  - super-resolution
  - arxiv:2610.09450
---
<h1 align="center">Iris-3B</h1>

<p align="center"><b>Pixel-space generation &amp; general vision learner.</b><br>A 3-billion-parameter model that paints every pixel directly — no VAE, no latent space — and, fine-tuned, estimates depth and restores and upscales images.</p>

<p align="center">
  <a href="https://arxiv.org/abs/2610.09450"><img alt="Paper" src="https://img.shields.io/badge/arXiv-2610.09450-b31b1b?style=for-the-badge"></a>
  <a href="https://speridlabs.com/research/iris"><img alt="Project Page" src="https://img.shields.io/badge/%F0%9F%8C%90%20Project%20Page-1f6feb?style=for-the-badge"></a>
  <a href="https://github.com/speridlabs/iris-3b"><img alt="Code" src="https://img.shields.io/badge/Code-181717?style=for-the-badge&logo=github"></a>
  <a href="https://huggingface.co/spaces/speridlabs/iris-3b"><img alt="Demo" src="https://img.shields.io/badge/%F0%9F%A4%97%20Demo-ffcc4d?style=for-the-badge"></a>
  <a href="https://huggingface.co/spaces/speridlabs/iris-3b-turbo"><img alt="Turbo demo" src="https://img.shields.io/badge/%E2%9A%A1%20Turbo%20Demo-ffcc4d?style=for-the-badge"></a>
  <a href="https://www.apache.org/licenses/LICENSE-2.0"><img alt="License" src="https://img.shields.io/badge/Apache%202.0-3c873a?style=for-the-badge"></a>
</p>

Generative priors are a promising foundation for downstream vision tasks. In this project we
explore pixel-space generative models as an alternative to vision foundation models such as DINOv2.

Most image generators work in a compressed "latent" space and rely on a separate
decoder to turn that into an image. Iris-3B skips that step: the network itself
outputs every pixel, so nothing is lost to a lossy, texture-biased latent. You type a
prompt, and you get a 1024-pixel image.

Iris-3B is not only an image generator. The same pixel-space prior, fine-tuned with no
architectural change, is a general vision learner for dense tasks where detail matters:
monocular depth estimation and image restoration and upscaling ship in this repo (see
*A general vision learner* below).

[Iris-3B Turbo](#iris-3b-turbo), in the `turbo/` folder, is a 4-step distilled version of Iris-3B: a
1024×1024 image in about 0.7 s on one H100.

<div align="center">
  <video src="https://huggingface.co/speridlabs/iris-3b/resolve/main/assets/reel.mp4" width="720" autoplay loop muted playsinline controls></video>
</div>

## Examples

Generated at native aspect ratios of about one megapixel with the default settings (CFG 3, 100 steps).

<table>
<tr>
<td width="33%" valign="top"><img src="assets/gallery/0020.jpg"><br><sub>Studio portrait of a Maasai elder wearing vibrant beaded jewelry, deep red cloth, dark backdrop, Rembrandt lighting, ultra detailed skin texture</sub></td>
<td width="33%" valign="top"><img src="assets/gallery/0021.jpg"><br><sub>Close-up portrait of a young woman with silver glitter freckles and iridescent makeup, soft pastel background, high fashion beauty photography</sub></td>
<td width="33%" valign="top"><img src="assets/gallery/1166.jpg"><br><sub>A glass sculpture of a heart filled with flowers, caustics and reflections, 3D render</sub></td>
</tr>
</table>

<table>
<tr>
<td width="33%" valign="top"><img src="assets/gallery/0029.jpg"><br><sub>Black and white portrait of a fisherman with a thick grey beard and deep wrinkles, piercing eyes, overcast light, fine grain film photograph</sub></td>
<td width="33%" valign="top"><img src="assets/gallery/0115.jpg"><br><sub>A Byzantine-style mosaic of a peacock made of tiny gold and turquoise tiles, shimmering texture</sub></td>
<td width="33%" valign="top"><img src="assets/gallery/1153.jpg"><br><sub>A bronze sculpture of a horse in motion, patina, dramatic museum spotlight</sub></td>
</tr>
</table>

<table>
<tr>
<td width="33%" valign="top"><img src="assets/gallery/1088.jpg"><br><sub>The ancient city of Petra with the Treasury carved into pink sandstone, morning light</sub></td>
<td width="33%" valign="top"><img src="assets/gallery/1196.jpg"><br><sub>A tree with lightbulbs instead of fruits glowing at dusk, surreal concept art</sub></td>
<td width="33%" valign="top"><img src="assets/gallery/1214.jpg"><br><sub>Thousands of sky lanterns rising into the night sky at Yi Peng festival in Chiang Mai</sub></td>
</tr>
</table>

<table>
<tr>
<td width="50%" valign="top"><img src="assets/gallery/0005.jpg"><br><sub>The aurora borealis swirling green and violet over a snowy Lofoten fishing village with red wooden cabins, reflections in a calm fjord, night photograph</sub></td>
<td width="50%" valign="top"><img src="assets/gallery/1001.jpg"><br><sub>Volcanic eruption at night in Iceland, rivers of glowing lava flowing across black fields, plumes of steam lit orange, long exposure</sub></td>
</tr>
</table>

## Get started

**1. Install the code**

```bash
git clone https://github.com/speridlabs/iris-3b.git
cd iris-3b
pip install -e .          # Python 3.11+, PyTorch 2.7.1+
```

**2. Download the weights** (about 12 GB for text-to-image; the `depth/` and `upscaler/` folders add about 12 GB each, `turbo/` about 6 GB)

```bash
hf download speridlabs/iris-3b --local-dir iris-3b --exclude "depth/*" "upscaler/*" "turbo/*"
```

**3. Generate an image**

```bash
python scripts/sample.py --checkpoint iris-3b \
    --prompt "a red fox sleeping in fresh snow, golden hour"
```

The text encoder (Qwen3-VL-4B-Instruct) downloads automatically on the first run.
You need an NVIDIA GPU with CUDA.

## Tips

| Setting | Default | What it does |
|---|---|---|
| `--cfg-scale` | 3 | How strictly the image follows the prompt. Higher values follow the prompt more literally but can look harsher. |
| `--steps` | 100 | Number of denoising steps. Fewer steps are faster but lose some detail. |
| `--seed` | — | Fix it to get the same image again. |
| `--negative-prompt` | — | Things you don't want in the image. |
| `--txt-file` | — | A text file with one prompt per line, for batches. |

Default output is 1024×1024. Write prompts as plain descriptive English sentences.

## Iris-3B Turbo

<p align="center"><img src="assets/turbo.webp" width="512" alt="Iris-3B Turbo sample: a shiba inu in aviator sunglasses driving a supercar under the words IRIS TURBO"></p>

Iris-3B Turbo, in the `turbo/` folder of this repo, is a 4-step distilled version of Iris-3B. Classifier-free guidance
is baked into the weights, so each step is a single forward pass; a 1024×1024 image takes about 0.7 s on one H100
(batch 4). It is a full fine-tune of all 3B parameters, not a LoRA. Try it in the [Turbo demo](https://huggingface.co/spaces/speridlabs/iris-3b-turbo).

With the code installed (step 1 above):

```bash
hf download speridlabs/iris-3b --include "turbo/*" --local-dir iris-3b    # about 6 GB
python scripts/sample.py --checkpoint iris-3b/turbo \
    --prompt "a red fox sleeping in fresh snow, golden hour"
```

The script reads the 4-step schedule from `turbo/distill.yaml` (σ = 1, 0.938, 0.833, 0.625, deterministic flow-Euler
steps) and samples without CFG. The schedule is part of the model: `--steps`, `--cfg-scale` and `--negative-prompt`
do not apply, while the prompt, `--height`, `--width` and `--seed` do. Turbo was distilled at about one megapixel in 7
aspect buckets: 1024×1024, 752×1344, 896×1344, 896×1200 and their transposes. The image above was generated on an H100 with `--seed 6`
and the prompt *Colorful animated cartoon poster: a smiling shiba inu in aviator sunglasses behind the wheel of a low futuristic
supercar, dynamic angle, neon sunset sky, with big bold vector lettering "IRIS TURBO" in the sky*.

### Benchmarks

| Model | GenEval | DPG-Bench | LongText-Bench (EN) | OneIG-Bench (EN) |
|---|---|---|---|---|
| Iris-3B | 0.798 | 86.52 | 0.857 | 0.540 |
| Iris-3B Turbo | 0.820 | 86.68 | 0.804 | 0.516 |
| [iris-3b-turbo-lora](https://huggingface.co/pentacoxian-dev/iris-3b-turbo-lora) r2-1000 | 0.760 | 85.05 | 0.733 | 0.486 |

| OneIG-Bench (EN) | Alignment | Text | Reasoning | Style | Diversity |
|---|---|---|---|---|---|
| Iris-3B | 0.863 | 0.889 | 0.279 | 0.449 | 0.220 |
| Iris-3B Turbo | 0.862 | 0.848 | 0.283 | 0.433 | 0.154 |
| iris-3b-turbo-lora r2-1000 | 0.851 | 0.721 | 0.258 | 0.445 | 0.153 |

All models at 1024×1024 with the official scorers and the same prompts and seeds; Iris-3B with its default sampling
settings. The community adapter was run with its recommended settings through its own code. Turbo is slightly ahead of
the base model on prompt following (GenEval, DPG-Bench) and behind it on long rendered text and on OneIG-Bench, where
the gap is mostly text and diversity.

### Training recipe

Turbo is distilled from Iris-3B with [Trajectory Distribution Matching](https://arxiv.org/abs/2503.06674) (TDM),
data-free, as a full fine-tune of all 3B parameters in two stages. The trainer and both configs are in the
[code repository](https://github.com/speridlabs/iris-3b#training-recipe) (`scripts/distill.py`).

| | Stage 1 | Stage 2 (released) |
|---|---|---|
| Steps (σ) | 6 (1, 0.926, 0.833, 0.714, 0.556, 0.333) | 4 (1, 0.938, 0.833, 0.625) |
| Initialization | Iris-3B | student: stage-1 EMA; fake network: stage 1 |
| Student updates | 1,600 | 500 |
| Global batch | 64 | 16 |
| Student / fake-network LR | 1e-6 / 5e-6 | 2.5e-7 / 2.5e-6 |
| EMA decay | 0.999 | 0.9995 |
| Prompts | ~981K | ~1.19M |

- **Networks**: frozen Iris-3B teacher, the student and a "fake" score network trained online on the student's
  outputs; deterministic flow-Euler step transitions.
- **Guidance**: CFG 3.5 (empty negative prompt) baked into the teacher target; the student samples without CFG.
- **Optimizer**: AdamW, betas (0, 0.999), weight decay 0, gradient clipping 1.0, constant LR, one fake update per
  student update.
- **Precision**: FP32 weights under BF16 autocast.
- **Resolution**: about 1 MP in 7 aspect buckets, one per global batch: 1024×1024, 752×1344, 896×1344, 896×1200 and
  their transposes.
- **Prompts**: text only, no images. Stage 1: fine-grained text-to-image prompts, MONET captions, rendered-text prompts
  and SFT captions, about 12% with quoted text. Stage 2: MONET captions, captions of text-rendering sources and other
  SFT captions. The prompt sets are not released.
- **Release**: the stage-2 EMA weights, exported in bf16.

### Thanks to the community turbo LoRA

The first few-step Iris came from the community: [pentacoxian-dev](https://huggingface.co/pentacoxian-dev) built
[iris-3b-turbo-lora](https://huggingface.co/pentacoxian-dev/iris-3b-turbo-lora)
([code](https://github.com/pentaCoxian/iris-3b-turbo), [demo](https://huggingface.co/spaces/pentacoxian-dev/iris-3b-turbo)),
a 4-step DMD2 distillation packaged as a rank-64 LoRA. It is a lightweight 460 MB adapter trained on a single consumer
GPU, far easier to download, share and build on than a full 6 GB checkpoint. Thank you, it is a joy to see Iris picked
up by the community this quickly.

### Limitations of Turbo

- Dense or long rendered text is less reliable than with the base model.
- Less variety across seeds: different seeds for one prompt give more similar images.
- Faint grain or speckle can appear on flat, vector-style backgrounds.
- Sampling is fixed at 4 steps, with no CFG or negative prompt.

## A general vision learner: depth, image restoration and upscaling

The same backbone, fine-tuned per task: Iris-3B estimates monocular depth and restores and upscales images. Both run the
whole model in a single forward pass with the empty prompt, so no text encoder is needed. The weights download on
first use.

**Monocular depth** — photo → relative depth:

<table>
<tr>
<td width="25%" valign="top"><img src="assets/downstream/depth-1-photo.webp"><br><sub>Photo</sub></td>
<td width="25%" valign="top"><img src="assets/downstream/depth-1-depth.webp"><br><sub>Iris-3B depth</sub></td>
<td width="25%" valign="top"><img src="assets/downstream/depth-2-photo.webp"><br><sub>Photo</sub></td>
<td width="25%" valign="top"><img src="assets/downstream/depth-2-depth.webp"><br><sub>Iris-3B depth</sub></td>
</tr>
</table>

**Image restoration and upscaling** — degraded input → restored image:

<table>
<tr>
<td width="25%" valign="top"><img src="assets/downstream/restore-1-input.webp"><br><sub>Input</sub></td>
<td width="25%" valign="top"><img src="assets/downstream/restore-1-iris.webp"><br><sub>Iris-3B</sub></td>
<td width="25%" valign="top"><img src="assets/downstream/restore-2-input.webp"><br><sub>Input</sub></td>
<td width="25%" valign="top"><img src="assets/downstream/restore-2-iris.webp"><br><sub>Iris-3B</sub></td>
</tr>
</table>

Try both interactively in the [demo](https://huggingface.co/spaces/speridlabs/iris-3b), or from the command line:

```bash
python scripts/depth.py photo.jpg --out depth_out      # <name>.npy + colorized <name>.png
python scripts/upscale.py photo.jpg --out upscaled     # <name>_x4.png
```

| Task | Training | Result |
|---|---|---|
| Depth | Direct regression of relative log depth, 10K steps | AbsRel 0.071, δ1 0.946 (mean of NYUv2, KITTI, ETH3D, ScanNet, DIODE; Marigold V2 protocol, zero-shot) |
| Image restoration and upscaling | One-step adversarial restoration (HYPIR recipe), 10K steps, EMA weights | LPIPS 0.292, PSNR 20.68 (DIV2K validation, 4×) |

Depth is affine-invariant, not metric: fit a scale and shift in log space to compare with ground truth.
Restoration tiles large outputs into overlapping 1024×1024 windows.

## What's in this repo

| File | Contents |
|---|---|
| `model.safetensors` | Text-to-image weights (EMA, float32; run with bfloat16 autocast) |
| `config.yaml` | Architecture and sampling settings read by `scripts/sample.py` |
| `depth/` | Depth model: `model.safetensors`, `empty_prompt.safetensors`, `config.yaml` |
| `upscaler/` | Image restoration and upscaling model: same three files |
| `turbo/` | Iris-3B Turbo, the 4-step distilled model: `model.safetensors` (EMA, bfloat16; upcast to float32 and run with bfloat16 autocast), `config.yaml`, `distill.yaml` (its sampling schedule) |

## About the model

| | |
|---|---|
| Size | 3B parameters (plus a frozen 4B text encoder) |
| Architecture | Diffusion transformer: 8 dual-stream + 16 single-stream blocks, then a 4-block pixel head that turns each 16×16 patch back into pixels |
| Text encoder | Qwen3-VL-4B-Instruct (frozen) |
| Training | Rectified flow, trained from scratch at 256 → 512 → 1024 px, then supervised fine-tuning at 1024 px (665K steps total) |

The full training recipe is in the [code repository](https://github.com/speridlabs/iris-3b).

## Limitations

- Like every image generator, Iris-3B can produce inaccurate, biased or unsafe content. Review outputs before using them.
- Text rendering inside images and exact object counts are not always reliable.
- Images are generated at about one megapixel; for larger prints, use an upscaler.

## License

Released under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).
The text encoder, Qwen3-VL-4B-Instruct, is downloaded from its publisher and stays under its own license (Apache 2.0).

## Citation

```bibtex
@article{licai2026iris,
  title   = {Iris-3B: Going Beyond the Latent with Pixel-Space
             Diffusion Training, Conversion and Fine-Tuning},
  author  = {Li Cai, Hanqiu and Garabito, Chema},
  journal = {arXiv preprint arXiv:2610.09450},
  year    = {2026},
  url     = {https://arxiv.org/abs/2610.09450}
}
```

**Made with ❤️ by [speridlabs.com](https://speridlabs.com)**

	---
	license: apache-2.0
	library_name: pytorch
	pipeline_tag: text-to-image
	tags:
	- text-to-image
	- diffusion
	- pixel-space
	- flow-matching
	- general-vision-learner
	- vision-foundation-model
	- depth-estimation
	- image-restoration
	- super-resolution
	- arxiv:2610.09450
	---

	<h1 align="center">Iris-3B</h1>

	<p align="center"><b>Pixel-space generation & general vision learner.</b><br>A 3-billion-parameter model that paints every pixel directly — no VAE, no latent space — and, fine-tuned, estimates depth and restores and upscales images.</p>

	<p align="center">
	<a href="https://arxiv.org/abs/2610.09450"><img alt="Paper" src="https://img.shields.io/badge/arXiv-2610.09450-b31b1b?style=for-the-badge"></a>
	<a href="https://speridlabs.com/research/iris"><img alt="Project Page" src="https://img.shields.io/badge/%F0%9F%8C%90%20Project%20Page-1f6feb?style=for-the-badge"></a>
	<a href="https://github.com/speridlabs/iris-3b"><img alt="Code" src="https://img.shields.io/badge/Code-181717?style=for-the-badge&logo=github"></a>
	<a href="https://huggingface.co/spaces/speridlabs/iris-3b"><img alt="Demo" src="https://img.shields.io/badge/%F0%9F%A4%97%20Demo-ffcc4d?style=for-the-badge"></a>
	<a href="https://huggingface.co/spaces/speridlabs/iris-3b-turbo"><img alt="Turbo demo" src="https://img.shields.io/badge/%E2%9A%A1%20Turbo%20Demo-ffcc4d?style=for-the-badge"></a>
	<a href="https://www.apache.org/licenses/LICENSE-2.0"><img alt="License" src="https://img.shields.io/badge/Apache%202.0-3c873a?style=for-the-badge"></a>
	</p>

	Generative priors are a promising foundation for downstream vision tasks. In this project we
	explore pixel-space generative models as an alternative to vision foundation models such as DINOv2.

	Most image generators work in a compressed "latent" space and rely on a separate
	decoder to turn that into an image. Iris-3B skips that step: the network itself
	outputs every pixel, so nothing is lost to a lossy, texture-biased latent. You type a
	prompt, and you get a 1024-pixel image.

	Iris-3B is not only an image generator. The same pixel-space prior, fine-tuned with no
	architectural change, is a general vision learner for dense tasks where detail matters:
	monocular depth estimation and image restoration and upscaling ship in this repo (see
	A general vision learner below).

	[Iris-3B Turbo](#iris-3b-turbo), in the `turbo/` folder, is a 4-step distilled version of Iris-3B: a
	1024×1024 image in about 0.7 s on one H100.

	<div align="center">
	<video src="https://huggingface.co/speridlabs/iris-3b/resolve/main/assets/reel.mp4" width="720" autoplay loop muted playsinline controls></video>
	</div>

	## Examples

	Generated at native aspect ratios of about one megapixel with the default settings (CFG 3, 100 steps).

	<table>
	<tr>
	<td width="33%" valign="top"><img src="assets/gallery/0020.jpg"><br><sub>Studio portrait of a Maasai elder wearing vibrant beaded jewelry, deep red cloth, dark backdrop, Rembrandt lighting, ultra detailed skin texture</sub></td>
	<td width="33%" valign="top"><img src="assets/gallery/0021.jpg"><br><sub>Close-up portrait of a young woman with silver glitter freckles and iridescent makeup, soft pastel background, high fashion beauty photography</sub></td>
	<td width="33%" valign="top"><img src="assets/gallery/1166.jpg"><br><sub>A glass sculpture of a heart filled with flowers, caustics and reflections, 3D render</sub></td>
	</tr>
	</table>

	<table>
	<tr>
	<td width="33%" valign="top"><img src="assets/gallery/0029.jpg"><br><sub>Black and white portrait of a fisherman with a thick grey beard and deep wrinkles, piercing eyes, overcast light, fine grain film photograph</sub></td>
	<td width="33%" valign="top"><img src="assets/gallery/0115.jpg"><br><sub>A Byzantine-style mosaic of a peacock made of tiny gold and turquoise tiles, shimmering texture</sub></td>
	<td width="33%" valign="top"><img src="assets/gallery/1153.jpg"><br><sub>A bronze sculpture of a horse in motion, patina, dramatic museum spotlight</sub></td>
	</tr>
	</table>

	<table>
	<tr>
	<td width="33%" valign="top"><img src="assets/gallery/1088.jpg"><br><sub>The ancient city of Petra with the Treasury carved into pink sandstone, morning light</sub></td>
	<td width="33%" valign="top"><img src="assets/gallery/1196.jpg"><br><sub>A tree with lightbulbs instead of fruits glowing at dusk, surreal concept art</sub></td>
	<td width="33%" valign="top"><img src="assets/gallery/1214.jpg"><br><sub>Thousands of sky lanterns rising into the night sky at Yi Peng festival in Chiang Mai</sub></td>
	</tr>
	</table>

	<table>
	<tr>
	<td width="50%" valign="top"><img src="assets/gallery/0005.jpg"><br><sub>The aurora borealis swirling green and violet over a snowy Lofoten fishing village with red wooden cabins, reflections in a calm fjord, night photograph</sub></td>
	<td width="50%" valign="top"><img src="assets/gallery/1001.jpg"><br><sub>Volcanic eruption at night in Iceland, rivers of glowing lava flowing across black fields, plumes of steam lit orange, long exposure</sub></td>
	</tr>
	</table>

	## Get started

	1. Install the code

	```bash
	git clone https://github.com/speridlabs/iris-3b.git
	cd iris-3b
	pip install -e . # Python 3.11+, PyTorch 2.7.1+
	```

	2. Download the weights (about 12 GB for text-to-image; the `depth/` and `upscaler/` folders add about 12 GB each, `turbo/` about 6 GB)

	```bash
	hf download speridlabs/iris-3b --local-dir iris-3b --exclude "depth/" "upscaler/" "turbo/*"
	```

	3. Generate an image

	```bash
	python scripts/sample.py --checkpoint iris-3b \
	--prompt "a red fox sleeping in fresh snow, golden hour"
	```

	The text encoder (Qwen3-VL-4B-Instruct) downloads automatically on the first run.
	You need an NVIDIA GPU with CUDA.

	## Tips

	\| Setting \| Default \| What it does \|
	\|---\|---\|---\|
	\| `--cfg-scale` \| 3 \| How strictly the image follows the prompt. Higher values follow the prompt more literally but can look harsher. \|
	\| `--steps` \| 100 \| Number of denoising steps. Fewer steps are faster but lose some detail. \|
	\| `--seed` \| — \| Fix it to get the same image again. \|
	\| `--negative-prompt` \| — \| Things you don't want in the image. \|
	\| `--txt-file` \| — \| A text file with one prompt per line, for batches. \|

	Default output is 1024×1024. Write prompts as plain descriptive English sentences.

	## Iris-3B Turbo

	<p align="center"><img src="assets/turbo.webp" width="512" alt="Iris-3B Turbo sample: a shiba inu in aviator sunglasses driving a supercar under the words IRIS TURBO"></p>

	Iris-3B Turbo, in the `turbo/` folder of this repo, is a 4-step distilled version of Iris-3B. Classifier-free guidance
	is baked into the weights, so each step is a single forward pass; a 1024×1024 image takes about 0.7 s on one H100
	(batch 4). It is a full fine-tune of all 3B parameters, not a LoRA. Try it in the [Turbo demo](https://huggingface.co/spaces/speridlabs/iris-3b-turbo).

	With the code installed (step 1 above):

	```bash
	hf download speridlabs/iris-3b --include "turbo/*" --local-dir iris-3b # about 6 GB
	python scripts/sample.py --checkpoint iris-3b/turbo \
	--prompt "a red fox sleeping in fresh snow, golden hour"
	```

	The script reads the 4-step schedule from `turbo/distill.yaml` (σ = 1, 0.938, 0.833, 0.625, deterministic flow-Euler
	steps) and samples without CFG. The schedule is part of the model: `--steps`, `--cfg-scale` and `--negative-prompt`
	do not apply, while the prompt, `--height`, `--width` and `--seed` do. Turbo was distilled at about one megapixel in 7
	aspect buckets: 1024×1024, 752×1344, 896×1344, 896×1200 and their transposes. The image above was generated on an H100 with `--seed 6`
	and the prompt *Colorful animated cartoon poster: a smiling shiba inu in aviator sunglasses behind the wheel of a low futuristic
	supercar, dynamic angle, neon sunset sky, with big bold vector lettering "IRIS TURBO" in the sky*.

	### Benchmarks

	\| Model \| GenEval \| DPG-Bench \| LongText-Bench (EN) \| OneIG-Bench (EN) \|
	\|---\|---\|---\|---\|---\|
	\| Iris-3B \| 0.798 \| 86.52 \| 0.857 \| 0.540 \|
	\| Iris-3B Turbo \| 0.820 \| 86.68 \| 0.804 \| 0.516 \|
	\| [iris-3b-turbo-lora](https://huggingface.co/pentacoxian-dev/iris-3b-turbo-lora) r2-1000 \| 0.760 \| 85.05 \| 0.733 \| 0.486 \|

	\| OneIG-Bench (EN) \| Alignment \| Text \| Reasoning \| Style \| Diversity \|
	\|---\|---\|---\|---\|---\|---\|
	\| Iris-3B \| 0.863 \| 0.889 \| 0.279 \| 0.449 \| 0.220 \|
	\| Iris-3B Turbo \| 0.862 \| 0.848 \| 0.283 \| 0.433 \| 0.154 \|
	\| iris-3b-turbo-lora r2-1000 \| 0.851 \| 0.721 \| 0.258 \| 0.445 \| 0.153 \|

	All models at 1024×1024 with the official scorers and the same prompts and seeds; Iris-3B with its default sampling
	settings. The community adapter was run with its recommended settings through its own code. Turbo is slightly ahead of
	the base model on prompt following (GenEval, DPG-Bench) and behind it on long rendered text and on OneIG-Bench, where
	the gap is mostly text and diversity.

	### Training recipe

	Turbo is distilled from Iris-3B with [Trajectory Distribution Matching](https://arxiv.org/abs/2503.06674) (TDM),
	data-free, as a full fine-tune of all 3B parameters in two stages. The trainer and both configs are in the
	[code repository](https://github.com/speridlabs/iris-3b#training-recipe) (`scripts/distill.py`).

	\| \| Stage 1 \| Stage 2 (released) \|
	\|---\|---\|---\|
	\| Steps (σ) \| 6 (1, 0.926, 0.833, 0.714, 0.556, 0.333) \| 4 (1, 0.938, 0.833, 0.625) \|
	\| Initialization \| Iris-3B \| student: stage-1 EMA; fake network: stage 1 \|
	\| Student updates \| 1,600 \| 500 \|
	\| Global batch \| 64 \| 16 \|
	\| Student / fake-network LR \| 1e-6 / 5e-6 \| 2.5e-7 / 2.5e-6 \|
	\| EMA decay \| 0.999 \| 0.9995 \|
	\| Prompts \| ~981K \| ~1.19M \|

	- Networks: frozen Iris-3B teacher, the student and a "fake" score network trained online on the student's
	outputs; deterministic flow-Euler step transitions.
	- Guidance: CFG 3.5 (empty negative prompt) baked into the teacher target; the student samples without CFG.
	- Optimizer: AdamW, betas (0, 0.999), weight decay 0, gradient clipping 1.0, constant LR, one fake update per
	student update.
	- Precision: FP32 weights under BF16 autocast.
	- Resolution: about 1 MP in 7 aspect buckets, one per global batch: 1024×1024, 752×1344, 896×1344, 896×1200 and
	their transposes.
	- Prompts: text only, no images. Stage 1: fine-grained text-to-image prompts, MONET captions, rendered-text prompts
	and SFT captions, about 12% with quoted text. Stage 2: MONET captions, captions of text-rendering sources and other
	SFT captions. The prompt sets are not released.
	- Release: the stage-2 EMA weights, exported in bf16.

	### Thanks to the community turbo LoRA

	The first few-step Iris came from the community: [pentacoxian-dev](https://huggingface.co/pentacoxian-dev) built
	[iris-3b-turbo-lora](https://huggingface.co/pentacoxian-dev/iris-3b-turbo-lora)
	([code](https://github.com/pentaCoxian/iris-3b-turbo), [demo](https://huggingface.co/spaces/pentacoxian-dev/iris-3b-turbo)),
	a 4-step DMD2 distillation packaged as a rank-64 LoRA. It is a lightweight 460 MB adapter trained on a single consumer
	GPU, far easier to download, share and build on than a full 6 GB checkpoint. Thank you, it is a joy to see Iris picked
	up by the community this quickly.

	### Limitations of Turbo

	- Dense or long rendered text is less reliable than with the base model.
	- Less variety across seeds: different seeds for one prompt give more similar images.
	- Faint grain or speckle can appear on flat, vector-style backgrounds.
	- Sampling is fixed at 4 steps, with no CFG or negative prompt.

	## A general vision learner: depth, image restoration and upscaling

	The same backbone, fine-tuned per task: Iris-3B estimates monocular depth and restores and upscales images. Both run the
	whole model in a single forward pass with the empty prompt, so no text encoder is needed. The weights download on
	first use.

	Monocular depth — photo → relative depth:

	<table>
	<tr>
	<td width="25%" valign="top"><img src="assets/downstream/depth-1-photo.webp"><br><sub>Photo</sub></td>
	<td width="25%" valign="top"><img src="assets/downstream/depth-1-depth.webp"><br><sub>Iris-3B depth</sub></td>
	<td width="25%" valign="top"><img src="assets/downstream/depth-2-photo.webp"><br><sub>Photo</sub></td>
	<td width="25%" valign="top"><img src="assets/downstream/depth-2-depth.webp"><br><sub>Iris-3B depth</sub></td>
	</tr>
	</table>

	Image restoration and upscaling — degraded input → restored image:

	<table>
	<tr>
	<td width="25%" valign="top"><img src="assets/downstream/restore-1-input.webp"><br><sub>Input</sub></td>
	<td width="25%" valign="top"><img src="assets/downstream/restore-1-iris.webp"><br><sub>Iris-3B</sub></td>
	<td width="25%" valign="top"><img src="assets/downstream/restore-2-input.webp"><br><sub>Input</sub></td>
	<td width="25%" valign="top"><img src="assets/downstream/restore-2-iris.webp"><br><sub>Iris-3B</sub></td>
	</tr>
	</table>

	Try both interactively in the [demo](https://huggingface.co/spaces/speridlabs/iris-3b), or from the command line:

	```bash
	python scripts/depth.py photo.jpg --out depth_out # <name>.npy + colorized <name>.png
	python scripts/upscale.py photo.jpg --out upscaled # <name>_x4.png
	```

	\| Task \| Training \| Result \|
	\|---\|---\|---\|
	\| Depth \| Direct regression of relative log depth, 10K steps \| AbsRel 0.071, δ1 0.946 (mean of NYUv2, KITTI, ETH3D, ScanNet, DIODE; Marigold V2 protocol, zero-shot) \|
	\| Image restoration and upscaling \| One-step adversarial restoration (HYPIR recipe), 10K steps, EMA weights \| LPIPS 0.292, PSNR 20.68 (DIV2K validation, 4×) \|

	Depth is affine-invariant, not metric: fit a scale and shift in log space to compare with ground truth.
	Restoration tiles large outputs into overlapping 1024×1024 windows.

	## What's in this repo

	\| File \| Contents \|
	\|---\|---\|
	\| `model.safetensors` \| Text-to-image weights (EMA, float32; run with bfloat16 autocast) \|
	\| `config.yaml` \| Architecture and sampling settings read by `scripts/sample.py` \|
	\| `depth/` \| Depth model: `model.safetensors`, `empty_prompt.safetensors`, `config.yaml` \|
	\| `upscaler/` \| Image restoration and upscaling model: same three files \|
	\| `turbo/` \| Iris-3B Turbo, the 4-step distilled model: `model.safetensors` (EMA, bfloat16; upcast to float32 and run with bfloat16 autocast), `config.yaml`, `distill.yaml` (its sampling schedule) \|

	## About the model

	\| \| \|
	\|---\|---\|
	\| Size \| 3B parameters (plus a frozen 4B text encoder) \|
	\| Architecture \| Diffusion transformer: 8 dual-stream + 16 single-stream blocks, then a 4-block pixel head that turns each 16×16 patch back into pixels \|
	\| Text encoder \| Qwen3-VL-4B-Instruct (frozen) \|
	\| Training \| Rectified flow, trained from scratch at 256 → 512 → 1024 px, then supervised fine-tuning at 1024 px (665K steps total) \|

	The full training recipe is in the [code repository](https://github.com/speridlabs/iris-3b).

	## Limitations

	- Like every image generator, Iris-3B can produce inaccurate, biased or unsafe content. Review outputs before using them.
	- Text rendering inside images and exact object counts are not always reliable.
	- Images are generated at about one megapixel; for larger prints, use an upscaler.

	## License

	Released under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).
	The text encoder, Qwen3-VL-4B-Instruct, is downloaded from its publisher and stays under its own license (Apache 2.0).

	## Citation

	```bibtex
	@article{licai2026iris,
	title = {Iris-3B: Going Beyond the Latent with Pixel-Space
	Diffusion Training, Conversion and Fine-Tuning},
	author = {Li Cai, Hanqiu and Garabito, Chema},
	journal = {arXiv preprint arXiv:2610.09450},
	year = {2026},
	url = {https://arxiv.org/abs/2610.09450}
	}
	```

	Made with ❤️ by [speridlabs.com](https://speridlabs.com)