---
license: apache-2.0
language:
- en
base_model: OpenMOSS-Team/MOSS-Image-Base
library_name: diffusers
pipeline_tag: text-to-image
tags:
- moss-image
- diffusers
- text-to-image
- 16gb
- structured-pruning
- alpha
---

_MOSS-Image pruned to fit a 16 GB card: a 30-block DiT instead of 42, a 0.8B text encoder with a 158M adapter instead of the 15.6 GB 9B one._ Text-to-image at 1024 px, **12.4 GB peak**, one consumer GPU.

![four samples: anime streetwear, armoured figure, kimono cat girl, mouse knight](samples/hero.jpg)

**This is an alpha.** It was assembled in two days on a single **RTX 4080 (16 GB)** and it is **undertrained**: see [Status](#status). Treat it as a working proof that the base model survives a 26.7% cut and a 16× smaller encoder, not as a finished model.

**transformer** — 30 blocks, **3.723 B** parameters, bf16 (upstream: 42 blocks, 5.082 B). Twelve blocks were fused into their neighbours by learned rank-32 merges and then baked into the weights.

**text encoder** — **Qwen3.5-0.8B**, 1.7 GB, frozen, plus a 158M adapter head (upstream: Qwen3.5-VL-9B, 15.63 GB). Cosine **0.924** against the native encoder's condition on held-out captions.

**VAE** — untouched `AutoencoderKLFlux2`, fp32.

**scheduler** — `MOSSImageScheduler`, flow-matching Euler, static shift 5.0.

**resolution** — 1024 by default, any positive multiple of 16; 3:4 and 4:3 work (the samples below are 864x1152 and 1152x1152).

**peak VRAM** — **12.4 GB** at 1024 px / 30 steps, one forward pair per step, no offload. 9.7 GB at 768x1152 when the encoder is freed before the DiT runs; ~1.15 it/s on a 4080.

## What changed

**Block fusion.** Middle blocks of a DiT move the stream little and partly cancel each other, so three adjacent blocks become one: block `s` stays and gets a LoRA, blocks `s+1, s+2` are deleted, and the LoRA is trained to make block `s` reproduce the *original* output of block `s+2`. Seven windows were trained at 1024 px, six went into the artifact — 42 − 12 = **30 blocks**. The merge is baked into the weights (`W' = W + B·A·scale`), so inference is not slowed down and nothing extra has to be loaded.

**Encoder replacement.** The 15.63 GiB 9B encoder is replaced by Qwen3.5-0.8B plus an adapter that imitates what the 9B encoder emitted, wired in as the `text_encoder` component (`MOSSAdapterTextEncoder`). It returns the same `(B, L, 4096)` states, so `encode_prompt`, the CFG negative branch and every `__call__` argument keep working.

**Block skipping is off by default in this folder.** The stock 42-block recipe skips layer 22 in the unconditional branch; in a 30-block DiT that index points at a different block and visibly wrecks the image (acid colours, red skin) — measured, not guessed. Pass `uncond_skip_layers` only with indices you measure yourself.

Both earlier models (the 42-block DiT and the 9B encoder) are **not** in this folder. They are unmodified files from [`OpenMOSS-Team/MOSS-Image-Base`](https://huggingface.co/OpenMOSS-Team/MOSS-Image-Base): re-download that repo into a scratch directory to rebuild the compression with `src/run_hi.sh`.

## Why "Mossling"

A mossling is a small moss. The base model is MOSS-Image, this is the same plant, 26.7% smaller, growing on a 16 GB card. It is an **unofficial derivative**, not a MOSS-Image release and not affiliated with OpenMOSS / MOSI.AI.

## Examples

Every image below comes from this folder: 50 steps, CFG 3, seed 7, each at the resolution of its source row. All four prompts are rows of the fine-tune set, copied verbatim — tag lists included, since that is the text the model was trained on.

| | prompt |
| --- | --- |
| ![sweater](samples/sweater_864x1152.png) | black sweater, brown skirt, closed mouth, ear piercing, earrings, hair behind ear, hair intakes, hairclip, jacket, jewelry, large breasts, long hair, long sleeves, off shoulder, pencil skirt, piercing, skirt, sweater, sweater tucked in, undressing, white jacket |
| ![wolverine](samples/wolverine_armor_1152x1152.png) | This highly detailed CGI rendering captures the iconic character Wolverine from Marvel Comics in his futuristic armor suit. The costume features intricate blue and gold plates, with sharp claws protruding from both hands, emphasizing his signature weapon. The background is dimly lit, highlighting the armored figure's muscular physique and determined expression under a glowing helmet. |
| ![kimono](samples/kimono_1152x1152.png) | animal hands, animal print, brown eyes, cat hair ornament, cat paws, cat print, cowboy shot, dragon tail, fish skeleton print, hair ornament, japanese clothes, kimono, limited palette, long hair, obi, obijime, parted lips, sash, standing, white hair, yukata |
| ![mouse](samples/mouse_knight_864x1152.png) | A cartoon of a grey-furred female mouse with golden eyes, her body retaining all the natural, unmistakable features of a mouse—small, round ears, a long, twitching tail, and paws with delicate fingers. She wears simple beige clothing suited for travel, with small, practical bronze armor pieces tailored perfectly for her size. A long, curved sword rests on her shoulder, the hilt grasped firmly in her paw as her stance remains low to the ground, reflecting her nimbleness. The background is an arid desert, with the ruins of an ancient temple in the distance, weathered by time and partially overtaken by shifting sands. |

The mouse row's caption continues with twelve random words ("Florgnax qwermbly …"); they are dropped here and were kept out of the prompt.

## Usage

```python
import torch
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained(".", custom_pipeline="pipeline_moss_image",
                                         trust_remote_code=True,
                                         dtype={"default": torch.bfloat16, "vae": torch.float32})
pipe.to("cuda")

image = pipe("a red fox walks through a snowy forest at dusk, cinematic, 85mm",
             height=1152, width=768, num_inference_steps=50, guidance_scale=3.0,
             generator=torch.Generator("cuda").manual_seed(7)).images[0]
```

`model_index.json` declares our text encoder as the `text_encoder` component, so `prompt=` works; the adapter head is taken from the sibling `adapter/adapter.safetensors`. Verified on `diffusers 0.41.0.dev0` — the `dtype=` dict form is recent (`model_index.json` records `_diffusers_version: 0.38.0`); older releases expect `torch_dtype=`.

Low-VRAM CLI from the adapter repo — it encodes the condition up front and frees the 0.8B encoder before the DiT runs:

```bash
python3 adapter/src/example.py \
    --prompt "a red fox walks through a snowy forest at dusk, cinematic, 85mm" --out fox.png
```

Training and compression scripts are in `src/` (`src/README.md`): `window.py` + `run_hi.sh` rebuild the block fusion from the original 42-block weights, `make_dataset.py` turns captioned images into cached latents/conditions, `train.py` is a fine-tuning loop that fits the 3.72 B DiT into 16 GB.

## Status

Honest list, this is what "alpha" means here:

* **The fine-tune is one epoch and stops there.** The DiT in this folder is the fused 30-block build after a single pass over 39 576 captioned rows: **4 947 steps** at batch 8 (lr 1e-5 cosine to 1e-6, 500-step warmup, no EMA). The loss flattens early and stays noisy — 0.32 to 0.50 with no trend after the first thousand steps. The previous build of this card stopped at 25 000 of 39 025 steps (batch 1) of an earlier run; the final epoch is not a clear win over it on photo prompts: on the four prompts that were on this card before, the final weights keep fine texture but print structure errors (a faint second profile behind the sitter's hair, fused feet, extra fingers), and the images read less photographic. On dataset-style tag prompts — the four above — output stays clean, which is what the training saw. Expect roughness on hands, complex geometry and anything outside the training distribution.
* **Not a full re-train.** The base model's own training is inherited; only the block merges and the short fine-tune above are new.
* **Text inside the image degrades** — signs and JSON layouts come out garbled, though one large word can land. Two of the fused blocks carried that ability, and one block plus a low-rank patch does not reproduce it.
* **JSON layout prompts are the weakest case** for the adapter as well (cosine 0.756): it was trained on captions, not on layout instructions — the two limitations stack on sign/layout prompts.
* **One seed per prompt in the samples.** Quality varies with seed, and the four images above are one draw each, seed 7, nothing re-rolled. Four random rows were generated, two were kept; the armoured-figure and mouse rows were on the card before and were re-shot on the final weights.
* **English prompts**, as in the dataset behind the adapter.

Quality against the base model, measured on the pruned-but-not-fine-tuned build (not re-measured for the final epoch):

| | stock 42-block + 9B | this folder |
| --- | --- | --- |
| peak VRAM, 1024 px / 30 steps | ~25.4 GiB | **12.4 GiB** |
| CLIP, 1024 px / 30 steps | 29.59 | 29.12 |
| CLIP, 512 px / 20 steps | 29.49 | 29.18 |

What was tried and rejected (refit on self-generated trajectories, FFN narrowing) is written down in `src/README.md` so nobody repeats it.

## Files

```
pipeline_moss_image.py   MOSSImagePipeline, MOSSImageTransformer, MOSSAdapterTextEncoder, scheduler
model_index.json         diffusers wiring: 30-block transformer, adapter encoder, VAE, scheduler
transformer/             DiT config + bf16 weights (3.723 B)
text_encoder/            Qwen3.5-0.8B, frozen
adapter/                 adapter v1: adapter.safetensors (158M head) + its own src/ and samples/
vae/                     untouched AutoencoderKLFlux2, fp32
scheduler/, tokenizer/   untouched
samples/                 the four images above + hero.jpg
src/                     compression and training scripts, plus the method write-up
```

## License and NOTICE

The weights and the bundled inference code are released under the [Apache License, Version 2.0](LICENSE).

This is a derivative of `OpenMOSS-Team/MOSS-Image-Base` (MOSS-Image, MOSI.AI). The 30-block DiT, the 0.8B encoder and the adapter are modifications of / replacements for components of that model. It is not an official MOSS-Image release and is not endorsed by the upstream authors.

We thank the Diffusers, Qwen, FLUX and OpenMOSS teams.

## Contacts

Please contact us if you can provide GPUs or money for training.

* telegram [recoilme](https://t.me/recoilme) *preferred*
* mail at aiartlab.org (slow response)

## Citation

```
@misc{mossling,
  title={Mossling: MOSS-Image pruned to a 16 GB card},
  author={recoilme and AiArtLab Team},
  url={https://huggingface.co/AiArtLab/mossling},
  year={2026}
}
```