paralaif/MOSS-Image-Alpha-LoRA

MOSS-Image Alpha LoRA (proof of concept)

Text-to-image with a real alpha channel for MOSS-Image. Prompt as usual, end with , isolated on a transparent background, decode with the RGBA VAE from trmz/flux2-klein-alpha, and you get a transparent PNG straight out of the sampler: no chroma key, no background-removal model, and partial transparency where it belongs (glass, fairy wings, gems, lattices).

This is a proof of concept. It was trained on only 39 images in about 40 minutes on a single 16 GB consumer GPU. It is not perfect, but it shows that MOSS-Image can learn to emit alpha-carrying latents with very little data.

Every character, creature and the castle above was generated separately with this LoRA and pasted onto a MOSS-generated background. The pixie's wings stay translucent.

Gallery

All images below are the raw RGBA PNG outputs (seed picked from 3), shown as-is with their alpha channel — the page background shows through. Full-resolution files are in examples/.

Logos, text and game assets work well too — MOSS-Image's typography survives the LoRA:

Structures and translucent materials (Eiffel Tower lattice, pyramid, pagoda, castle, pixie wings, water creature):

Partial alpha up close, over a gradient: the background shows through the tower's lattice, the wings and the fins.

Why a LoRA is needed

The RGBA VAE shares FLUX.2's latent space (MOSS-Image uses the same VAE family), so it can decode MOSS latents directly. But MOSS has only ever produced latents of opaque RGB images, so the decoded alpha is garbage. The LoRA teaches MOSS to produce latents that carry the alpha signal:

How to use (ComfyUI)

File	Where	Source
moss_alpha_v1.safetensors (this repo)	models/loras/	this repo
moss_image_sft_int8_convrot.safetensors	models/diffusion_models/	Comfy-Org/MOSS-Image
qwen3.5_9b_int8_convrot.safetensors	models/text_encoders/	Comfy-Org/MOSS-Image
flux2-vae.safetensors (optional, opaque preview)	models/vae/	Comfy-Org/MOSS-Image
vae/diffusion_pytorch_model.safetensors → rename to flux2_klein_alpha_rgba_vae.safetensors	models/vae/	trmz/flux2-klein-alpha (Apache-2.0)
an 8-step DMD LoRA for MOSS-Image SFT (optional, speed)	models/loras/	e.g. lodestones/MOSS-Image-SFT-8-steps-dmd

1. Copy comfyui/ComfyUI-RGBA-VAE into ComfyUI/custom_nodes/. ComfyUI's stock VAE loader assumes 3 channels and fails on this 4-channel VAE; the node Load RGBA VAE (Flux2) loads it with 4 input/output channels.
2. Load comfyui/moss-alpha-lora-t2i.json and pick your files in the loaders.
3. Write your prompt and end it with , isolated on a transparent background.
4. The RGBA PNG save node writes the transparent image; the other one is the same latent through the normal VAE.

Settings that worked

	Value
LoRA strength	1.0 (0.8 is fine; at 0.6 subjects start turning see-through)
With 8-step DMD	8 steps, CFG 1, er_sde / simple
Without DMD	28 steps, CFG 4 (tested, works)
Resolution	1024², 832×1216, 1216×832 tested

• Keep the trigger phrase. Without it, simple subjects still come out transparent, but prompts that imply a setting (e.g. "the Eiffel Tower") get a full scene and the alpha breaks.
• Describe the subject, not a background. "Full body" helps characters stay in frame.

Research notes

How we got here, including what did not work:

1. RGBA VAE alone ≈ chroma key. Decoding plain MOSS latents with the RGBA VAE gives near-random alpha. With a flat dark-purple background (#1E0050) the alpha becomes usable, but measured against a plain chroma key on the RGB decode it matched at IoU 0.95–0.99: the VAE learned "magenta tint = transparent", so it was effectively a chroma key with extra halos. The latent stats (BatchNorm) are identical between the two VAEs; the decoder changed by 1–2 %, the encoder by much more — the encoder writes an "opaque" signal plain FLUX.2 latents lack.
2. Teacher–student data. We built a small RGBA dataset with MOSS as the image source and other models as alpha teachers:
   • opaque subjects: MOSS on a flat purple background → Klein Base 4B + extract_4b from trmz/flux2-klein-alpha (composite + clean plate → RGBA), checked against a chroma key;
   • glass: MOSS generates an empty textured plate, then the object is inpainted over the same latent (denoise ≈ 0.8), so the extractor sees the real background through the glass;
   • effects (fire, smoke, splashes): Qwen Image 2.1, which generates alpha natively, plus a luminance key for emissive effects on black.
3. Automatic filtering. Border alpha, foreground size, agreement with the chroma key, holes, background drift between composite and plate, and "see-through" share for glass. Yield in the pilots: opaque ~93 %, effects ~67 %, glass ~25 %. We trained on what passed: 39 images (26 opaque, 5 glass, 8 effects).
4. Training with a high learning rate (see below). The model generalises well beyond the 39 training subjects — none of the gallery characters were in the dataset.

Training details

Base	MOSS-Image SFT (int8 convrot), trained with ai-toolkit
VAE for targets	trmz/flux2-klein-alpha RGBA VAE (4 channels in/out)
Data	39 RGBA images, captions "<style> of <subject>, isolated on a transparent background", 512/768/1024 buckets
LoRA	rank 32, alpha 32
Optimizer / LR	AdamW 8-bit, 3e-4 constant
Steps	2000 run, step 1000 released (8 epochs); 1500+ overfits (halos, broken fire)
Timesteps	shift
Hardware / time	1× RTX 5060 Ti 16 GB, ~40 min to step 1000

ai-toolkit already supports RGBA datasets (load_rgba). MOSS needed two small changes, included as training/ai-toolkit-moss-rgba.patch: the FLUX.2 VAE loader reads its channel count from the checkpoint, and MOSS honours model_kwargs: {rgba: true}. The config is training/train_moss_alpha_v1.yaml.

Limitations

• Tiny dataset. 39 images; expect occasional halos, faint semi-transparent patches or soft edges on subjects far from the data.
• Some dark clothing turns translucent, e.g. the witch's "starry" cloak stayed partly see-through at every strength and seed we tried.
• Amorphous effects (fire, smoke) are the weakest category; only 8 such images were in the dataset.
• Glass is real but inconsistent: sometimes beautifully see-through, sometimes nearly opaque.
• The LoRA changes the image a bit compared to plain MOSS (pose, costume details), and pushes towards centred, full-subject compositions.
• Checkpoints after step 1000 overfit; we did not tune LR, rank or schedule.

Credits and licences

• MOSS-Image by OpenMOSS — Apache-2.0.
• RGBA VAE and Extract-4B by Xavier Jara, trmz/flux2-klein-alpha (method: AlphaVAE) — Apache-2.0.
• FLUX.2 Klein Base 4B by Black Forest Labs — Apache-2.0 (used as the extraction teacher).
• Qwen Image 2.1 — Qwen Research License. Built with Qwen: 8 of the 39 training images (effects) were generated with Qwen Image 2.1. Because of that licence, this LoRA is released for non-commercial research and evaluation use only.
• 8-step DMD LoRA for MOSS-Image SFT by lodestones — Apache-2.0.
• Chroma key used for filtering: VNCCS Chroma Key (ComfyUI_VNCCS).
• Training: ai-toolkit by Ostris (MIT).

All example characters and creatures are original designs generated for this card; they are not meant to depict any existing franchise character.