VisualInference/MOSS-Image-SFT-MLX-compact

MOSS-Image-SFT for Apple Silicon (MLX, compact)

An MLX build of OpenMOSS-Team/MOSS-Image-SFT, OpenMOSS's 5B text-to-image DiT for long bilingual text rendering and JSON layout control, for Macs with Apple Silicon. Runs with moss-image-mlx, a port verified against the official PyTorch pipeline.

• 7.42 GB peak memory for a 1024² image with a short prompt (about +1 GB with a 650-token prompt), against a 27 GB download and a 16.6 GB peak for the original bf16 weights.
• all eval text still rendered correctly (14-image evaluation below).
• Official seeds reproduce official images.
• Download: 12.1 GB. Transformer 8-bit; text encoder (Qwen3.5-9B language path) 6-bit; VAE decoder fp32.

Renders from this tier at 1024², 50 steps, CFG 3, seed 42. More in assets/.

Quickstart

pip install git+https://github.com/felix-ab/moss-image-mlx
moss-mlx "A cozy bakery at dusk. A wooden sign reads \"THE GOLDEN CRUST BAKERY\"." --model VisualInference/MOSS-Image-SFT-MLX-compact -o bakery.png

from moss_image_mlx import MOSSImagePipeline, Request, save
from huggingface_hub import snapshot_download

pipe = MOSSImagePipeline(snapshot_download("VisualInference/MOSS-Image-SFT-MLX-compact"))
req = Request(prompt="A neon sign that reads \"OPEN LATE\" above a ramen bar", width=1024, height=1024, seed=7)
save(pipe(req), req, "ramen.png")

Layout prompts (JSON), batch rendering, the prompt cache and every option are documented in the GitHub README.

Which tier?

Tier	Download	Peak at 1024²	Pick it if
8bit	14.1 GB	9.3 GB	you have 24 GB or more; reference quality
compact	12.1 GB	7.4 GB	you have 16 GB
original bf16 (--model OpenMOSS-Team/MOSS-Image-SFT)	27.2 GB	16.6 GB	you have 32 GB or more; fastest (~11.5 min per image), no quantization

Measured on a Mac mini M4 Pro (48 GB), MLX 0.32.4, one 1024² image at the official defaults (50 steps, CFG 3, shift 5, block 22 skipped in the negative branch). Time: 12.4 min per image. The model is compute-bound, so quantization saves memory, not time.

Quality evidence

The same 14 prompts were rendered with the bf16 weights and with this tier, then scored:

• OCR: Apple Vision, order-free character recall against the requested text.
• Layout: IoU between each requested text box and where the text actually appears.

Prompt (1024², seed 42)	bf16 reference	This tier
Bakery sign + chalkboard (EN): OCR recall	0.98	0.94
Book cover, 3 lines (EN): OCR recall	1.00	1.00
Typed 4-line paragraph (EN): OCR recall	1.00	1.00
3-step infographic (EN): OCR recall	0.99	0.99
Handwritten fridge note (ZH): OCR recall	1.00	0.96
Tea-house calligraphy sign (ZH) ¹: OCR recall	0.30	0.40
Event poster, 3 lines (ZH): OCR recall	1.00	1.00
Printed notice, 4 lines (ZH): OCR recall	1.00	1.00
Layout JSON: EXPLORE poster: box IoU per text	0.96	0.98
Layout JSON: jazz poster, 3 text boxes: box IoU per text	0.88, 0.93, 0.66	0.86, 0.93, 0.80

Summary:

• OCR mean 0.910 (bf16: 0.908).
• Layout IoU 0.892 (bf16: 0.858).
• Mean PSNR vs the bf16 render 27.5 dB. With this model, bf16 rounding alone moves an image about 23 dB.

¹ Apple's OCR cannot read the vertical calligraphy in this prompt for any variant, bf16 included; the render itself is correct.

The remaining misses were checked by eye and are OCR misreads of correct renders (for example 顺 read as 順). More: how the port was verified.

What is quantized:

• Transformer: the 42 transformer blocks' linear layers use MLX affine 8-bit weights (group size 64). The patch embedder, timestep MLP, shared modulation, head modulation, final projection and text projection stay bf16 (~1% of weights).
• Text encoder: linear layers and embeddings use 6-bit, group size 64. Its RMSNorm weights are kept in fp32 to match the reference.
• VAE: the decoder only, fp32.

Limitations

• Dense, very small text can garble, as in the original model.
• The SFT checkpoint shows faint vertical banding on some skies and walls (upstream behaviour).
• ~12.4 min per 1024² image on an M4 Pro at the official 50 steps. Few-step support is planned.

中文说明

本仓库是 OpenMOSS MOSS-Image-SFT 的 Apple Silicon（MLX）量化版本，配合 moss-image-mlx 使用。

• 显存与下载： 生成一张 1024² 图像的峰值内存约 7.42 GB，下载 12.1 GB。
• 正确性： 已与官方 PyTorch 流程逐项对齐（采样日程逐位一致，fp32 端到端 69.7 dB）。
• 种子复现： 使用官方种子可复现官方示例图。
• 文字渲染： 在 14 张图的评测中，中英文文字渲染与 bf16 原版均正确。

安装：pip install git+https://github.com/felix-ab/moss-image-mlx，然后运行 moss-mlx "提示词" --model VisualInference/MOSS-Image-SFT-MLX-compact。

License and attribution

Apache-2.0, the same as the original weights. See NOTICE.

• MOSS-Image: © OpenMOSS.
• Text encoder: Qwen3.5-9B (Apache-2.0).
• VAE: FLUX.2 with the FLUX.2 small decoder by Black Forest Labs (Apache-2.0).

This is an independent conversion, not affiliated with or endorsed by OpenMOSS.