README.md

---
license: apache-2.0
base_model: OpenMOSS-Team/MOSS-Image-SFT
base_model_relation: quantized
library_name: mlx
pipeline_tag: text-to-image
language:
- en
- zh
tags:
- mlx
- apple-silicon
- moss-image
- text-to-image
- text-rendering
- layout
- quantized
- 8-bit
---

# MOSS-Image-SFT for Apple Silicon (MLX, compact)

An MLX build of [OpenMOSS-Team/MOSS-Image-SFT](https://huggingface.co/OpenMOSS-Team/MOSS-Image-SFT), OpenMOSS's 5B text-to-image DiT for long bilingual text rendering and JSON layout control, for Macs with Apple Silicon. Runs with [moss-image-mlx](https://github.com/felix-ab/moss-image-mlx), a port verified against the official PyTorch pipeline.

- **7.42 GB peak memory** for a 1024² image with a short prompt (about +1 GB with a 650-token prompt), against a 27 GB download and a 16.6 GB peak for the original bf16 weights.
- **all eval text still rendered correctly** (14-image evaluation below).
- **Official seeds reproduce official images.**
- **Download:** 12.1 GB. Transformer 8-bit; text encoder (Qwen3.5-9B language path) 6-bit; VAE decoder fp32.

<p align="center"><img src="assets/en_sign.jpg" width="32%"> <img src="assets/zh_poster.jpg" width="32%"> <img src="assets/layout_jazz.jpg" width="32%"></p>
<p align="center"><i>Renders from this tier at 1024², 50 steps, CFG 3, seed 42. More in assets/.</i></p>

## Quickstart

```bash
pip install moss-image-mlx
moss-mlx "A cozy bakery at dusk. A wooden sign reads \"THE GOLDEN CRUST BAKERY\"." --model VisualInference/MOSS-Image-SFT-MLX-compact -o bakery.png
```

```python
from moss_image_mlx import MOSSImagePipeline, Request, save
from huggingface_hub import snapshot_download

pipe = MOSSImagePipeline(snapshot_download("VisualInference/MOSS-Image-SFT-MLX-compact"))
req = Request(prompt="A neon sign that reads \"OPEN LATE\" above a ramen bar", width=1024, height=1024, seed=7)
save(pipe(req), req, "ramen.png")
```

Layout prompts (JSON), batch rendering, the prompt cache and every option are documented in the [GitHub README](https://github.com/felix-ab/moss-image-mlx).

## Which tier?

| Tier | Download | Peak at 1024² | Pick it if |
|---|---|---|---|
| [8bit](https://huggingface.co/VisualInference/MOSS-Image-SFT-MLX-8bit) | 14.1 GB | 9.3 GB | you have 24 GB or more; reference quality |
| [compact](https://huggingface.co/VisualInference/MOSS-Image-SFT-MLX-compact) | 12.1 GB | 7.4 GB | you have 16 GB |
| original bf16 (`--model OpenMOSS-Team/MOSS-Image-SFT`) | 27.2 GB | 16.6 GB | you have 32 GB or more; fastest (~11.5 min per image), no quantization |

Measured on a Mac mini M4 Pro (48 GB), MLX 0.32.4, one 1024² image at the official defaults (50 steps, CFG 3, shift 5, block 22 skipped in the negative branch). Time: **12.4 min** per image. The model is compute-bound, so quantization saves memory, not time.

## Quality evidence

The same 14 prompts were rendered with the bf16 weights and with this tier, then scored:
- **OCR:** Apple Vision, order-free character recall against the requested text.
- **Layout:** IoU between each requested text box and where the text actually appears.

| Prompt (1024², seed 42) | bf16 reference | This tier |
|---|---|---|
| Bakery sign + chalkboard (EN): OCR recall | 0.98 | 0.94 |
| Book cover, 3 lines (EN): OCR recall | 1.00 | 1.00 |
| Typed 4-line paragraph (EN): OCR recall | 1.00 | 1.00 |
| 3-step infographic (EN): OCR recall | 0.99 | 0.99 |
| Handwritten fridge note (ZH): OCR recall | 1.00 | 0.96 |
| Tea-house calligraphy sign (ZH) ¹: OCR recall | 0.30 | 0.40 |
| Event poster, 3 lines (ZH): OCR recall | 1.00 | 1.00 |
| Printed notice, 4 lines (ZH): OCR recall | 1.00 | 1.00 |
| Layout JSON: EXPLORE poster: box IoU per text | 0.96 | 0.98 |
| Layout JSON: jazz poster, 3 text boxes: box IoU per text | 0.88, 0.93, 0.66 | 0.86, 0.93, 0.80 |

**Summary:**
- OCR mean 0.910 (bf16: 0.908).
- Layout IoU 0.892 (bf16: 0.858).
- Mean PSNR vs the bf16 render 27.5 dB. With this model, bf16 rounding alone moves an image about 23 dB.

¹ Apple's OCR cannot read the vertical calligraphy in this prompt for *any* variant, bf16 included; the render itself is correct.

The remaining misses were checked by eye and are OCR misreads of correct renders (for example 顺 read as 順). More: [how the port was verified](https://github.com/felix-ab/moss-image-mlx#how-it-was-verified).

**What is quantized:**
- **Transformer:** the 42 transformer blocks' linear layers use MLX affine 8-bit weights (group size 64). The patch embedder, timestep MLP, shared modulation, head modulation, final projection and text projection stay bf16 (~1% of weights).
- **Text encoder:** linear layers and embeddings use 6-bit, group size 64. Its RMSNorm weights are kept in fp32 to match the reference.
- **VAE:** the decoder only, fp32.

## Limitations

- Dense, very small text can garble, as in the original model.
- The SFT checkpoint shows faint vertical banding on some skies and walls (upstream behaviour).
- ~12.4 min per 1024² image on an M4 Pro at the official 50 steps. Few-step support is planned.

## 中文说明

本仓库是 OpenMOSS **MOSS-Image-SFT** 的 Apple Silicon（MLX）量化版本，配合 [moss-image-mlx](https://github.com/felix-ab/moss-image-mlx) 使用。

- **显存与下载：** 生成一张 1024² 图像的峰值内存约 **7.42 GB**，下载 12.1 GB。
- **正确性：** 已与官方 PyTorch 流程逐项对齐（采样日程逐位一致，fp32 端到端 69.7 dB）。
- **种子复现：** 使用官方种子可复现官方示例图。
- **文字渲染：** 在 14 张图的评测中，中英文文字渲染与 bf16 原版均正确。

安装：`pip install moss-image-mlx`，然后运行 `moss-mlx "提示词" --model VisualInference/MOSS-Image-SFT-MLX-compact`。

## License and attribution

Apache-2.0, the same as the original weights. See [NOTICE](NOTICE).
- **MOSS-Image:** © OpenMOSS.
- **Text encoder:** Qwen3.5-9B (Apache-2.0).
- **VAE:** FLUX.2 with the FLUX.2 small decoder by Black Forest Labs (Apache-2.0).

This is an independent conversion, not affiliated with or endorsed by OpenMOSS.