mlx-community/MOSS-Image-SFT-bf16

MLX (bf16) conversion of OpenMOSS-Team/MOSS-Image-SFT (Apache-2.0) for Apple Silicon. MOSS-Image-SFT is the supervised fine-tune of MOSS-Image-Base: same architecture and prompts, cleaner text grouping and composition. It is the recommended checkpoint.

Model

MOSS-Image pairs a 5B single-stream DiT with the Qwen3.5-9B text encoder and the FLUX.2 VAE with BFL's small decoder. Its DiT design:

42 blocks with one shared adaLN modulation
SiTU-GLU FFN
4-axis RoPE

What it does well

Long text, in English and Chinese. Line breaks in the prompt are kept.
JSON layout prompts that place text and objects by bounding box: summary.caption plus items, with bbox_format: "yxyx" and bbox_coord_space: "abs_centralized" (pixels, centred on the canvas, same size as the request).

The snapshot is a standard diffusers tree (transformer/ text_encoder/ tokenizer/ vae/ scheduler/):

The transformer and text encoder are bf16, the VAE fp32. The files are byte-identical to upstream revision fc5de0d5.
The PyTorch custom pipeline (pipeline_moss_image.py) stays upstream.

Sampling

Setting	Value
Sampler	Euler flow matching
Steps	50
Shift	5 (snapped to the training grid)
CFG	3, with block 22 skipped in the negative branch
Negative prompt	empty

Parity (Swift/MLX port vs the PyTorch reference, fp32, TF32 off)

Component	Result
Qwen3.5-9B encoder, final hidden state	relL2 4.8e-6
DiT, one forward (12 timestep / prompt / skip-layer cases)	relL2 1.1e-6 – 5.3e-6
Scheduler sigmas (4 – 50 steps)	bit-exact
FLUX.2 small decoder	relL2 1.2e-6
bf16 end-to-end (8 steps) vs fp32	inside PyTorch's own bf16 drift

Memory and speed (M5 Max, 1024², 50 steps)

DiT tier	Resident	Peak	Denoise
bf16	10.3 GB	26.2 GB	113 s (2.27 s/step)
int8 (quantized at load)	5.6 GB	21.5 GB	about the same

Every step runs two transformer forwards (CFG).
The peak is set by the bf16 text encoder (16.8 GB). The port loads it per request and releases it before denoising.
Sizes from 256² to 1456² by area work; width and height must be multiples of 16.

Use (Swift / MLXEngine)

import MLXMossImage
import MLXToolKit

let package = MossImageSFTPackage(configuration: MossImageConfiguration(quant: .bf16))  // weights materialize on load
try await package.load()
let r = try await package.run(T2IRequest(prompt: "A hand-painted sign that reads OPEN LATE", width: 1024, height: 1024, seed: 42)) as! T2IResponse

Code: https://github.com/xocialize/moss-image-swift (Swift/MLX port, MIT).

Training data (from the upstream technical report)

Sources: about 3.2 billion image-text records from 16 public datasets. About 98% come from ReLAION, OmniCorpus and DataComp.
Filtering: AI-generated images were filtered out.
Captions: written by Gemma 4 and a Qwen model.
Rights: the report makes no rights statement for the data.

Review the technical report before you use the model.