Link MATLOWAI's DeBand comparison and edit anatomy pages

138539a verified 10 days ago

10.5 kB

---
tags:
- comfyui
- diffusion-single-file
- lora
base_model:
- OpenMOSS-Team/MOSS-Image-Base
- OpenMOSS-Team/MOSS-Image-SFT
license: apache-2.0
---
# MOSS-Image: SFT Destripe LoRA + split q/k/v checkpoints

This repo has two things for running MOSS-Image in ComfyUI:

- **[SFT Destripe](#sft-destripe-lora)**, a 1.9 MB LoRA that removes the faint vertical color stripes MOSS-Image SFT draws in smooth areas.
- **[Split q/k/v checkpoints](#split-qkv-non-fused-checkpoints)**: the same weights as Comfy-Org's, with attention q, k and v stored as separate tensors.

Original model repositories:

- https://huggingface.co/OpenMOSS-Team/MOSS-Image-Base
- https://huggingface.co/OpenMOSS-Team/MOSS-Image-SFT

## Files

```
📂 ComfyUI/
├── 📂 models/
│   ├── 📂 loras/
│   │   └── moss_sft_destripe_top64_bf16.safetensors
│   ├── 📂 diffusion_models/
│   │   ├── moss_image_base_bf16_split_qkv.safetensors
│   │   └── moss_image_sft_bf16_split_qkv.safetensors
```

| File | Size | SHA-256 |
| --- | --- | --- |
| `loras/moss_sft_destripe_top64_bf16.safetensors` | 1.9 MB | `fcd0d025f6a62dfe3171ab11fe9b3119eac735e22b94913c4efe45d8c69c2da9` |
| `diffusion_models/moss_image_base_bf16_split_qkv.safetensors` | 10.2 GB | `65a698df38fbee8307a16059958f6dd1fef0db69c1e3805fe13ec1498cb0499e` |
| `diffusion_models/moss_image_sft_bf16_split_qkv.safetensors` | 10.2 GB | `9373f43d0fdad7d6d62b7dc2d96b2a048aab47f9c6e7befe11456b5ec26f70fc` |

Get the text encoder (`text_encoders/`) and VAE (`vae/`) from
[Comfy-Org/MOSS-Image](https://huggingface.co/Comfy-Org/MOSS-Image). They are unchanged.

---

## SFT Destripe LoRA

MOSS-Image **SFT** often paints evenly spaced vertical color stripes across smooth areas such as
skies, fog, walls and studio backdrops. The stripes are about 21 px apart at 1024 px wide and 16 px
apart at 768 px wide. They are a shift in color, not brightness, so they are easy to miss at a glance
and obvious once color is boosted.

`moss_sft_destripe_top64_bf16.safetensors` removes them while keeping nearly all of what SFT does.

**Builds on [MATLOWAI's DeBand investigation](https://huggingface.co/MATLOWAI/MOSS-Image-SFT-DeBand).** His block-swap analysis found the layers that carry the stripes, and his bandviz tool is the banding score used below. His fix reverts whole layers. This LoRA removes only the few directions inside those layers that feed the stripes.

![SFT vs SFT + Destripe LoRA, color-only view](assets/deband_colour_view.png)

*Same prompt and seed (MOSS SFT int8, ER-SDE, 25 steps, CFG 2.5, skip-layer guidance on layer 22).
Left: the image. Right: the color-only view from [bandviz](https://huggingface.co/MATLOWAI/MOSS-Image-SFT-DeBand)
(brightness removed, color ×5). The SFT sky (top) has regular green and purple vertical stripes. With
the LoRA (bottom) it is clean.*

### Usage

1. Load the **SFT** model: Comfy-Org's `moss_image_sft_bf16` or `moss_image_sft_int8_convrot`, or
   `moss_image_sft_bf16_split_qkv` from this repo.
2. Add **LoraLoaderModelOnly** with this file.
3. Set the strength by resolution:
   - **1.0 at 768 px**
   - **1.5 at 1024 px**
4. Sample as usual. Everything else in the workflow stays the same.

At 1024 px the stripes are wider, and 1.0 leaves a faint trace in about half of images. 1.5 clears it
(stripe left in 1 of 12 test images instead of 6 of 12) for a tiny cost in similarity to SFT. 2.0 is no
better than 1.5 and moves further from SFT. The LoRA is built from SFT's own changes, so use it on SFT
only, not on Base.

**Bypass mode on fused files:** the LoRA touches q, k and v in a few layers. ComfyUI's normal LoRA
loading handles that on the fused Comfy-Org checkpoints. **Load LoRA (Bypass)** drops q/k/v adapters on
fused files (see [below](#why-this-matters-for-loras-and-lokrs)). For bypass mode, use the split
checkpoints from this repo.

### How well it works

Blind test: 4 prompts with large smooth areas (sand dunes, a plain wall, open water, a portrait on a
blurred backdrop) × 3 seeds, none of them used to build the fix. ER-SDE, 25 steps, CFG 2.5.

**768 px**

| Model | bandviz score (median) | Stripe still the strongest peak | Similarity to SFT (SSIM) | Size |
| --- | --- | --- | --- | --- |
| SFT | 9.8 | 9 / 12 | 1.000 | — |
| SFT, layers 37–41 reverted to Base ([MATLOWAI's fix](https://huggingface.co/MATLOWAI/MOSS-Image-SFT-DeBand)) | 6.5 | 8 / 12 | 0.980 | 1.13 GB |
| **SFT + this LoRA @ 1.0** | **4.2** | **1 / 12** | **0.995** | **1.9 MB** |
| Base | 4.2 | 5 / 12 | 0.876 | — |

**1024 px**

| Model | bandviz score (median) | Stripe still the strongest peak | Similarity to SFT (SSIM) | Size |
| --- | --- | --- | --- | --- |
| SFT | 24.0 | 11 / 12 | 1.000 | — |
| SFT, layers 37–41 reverted to Base (MATLOWAI's fix) | 8.4 | 4 / 12 | 0.983 | 1.13 GB |
| SFT + this LoRA @ 1.0 | 6.1 | 6 / 12 | 0.996 | 1.9 MB |
| **SFT + this LoRA @ 1.5** | **5.9** | **1 / 12** | **0.993** | **1.9 MB** |
| SFT + this LoRA @ 2.0 | 6.0 | 1 / 12 | 0.992 | 1.9 MB |
| Base | 4.0 | 0 / 12 | 0.863 | — |

- **bandviz score:** MATLOWAI's metric. The column color spectrum's strongest peak at 10–48 px, divided
  by the median. Lower means less banding, and about 16 or below looks clean.
- **Stripe still the strongest peak:** the number of images whose spectrum still peaks at the stripe
  width (16 px at 768, 21.3 px at 1024).
- **SSIM:** how close the image stays to plain SFT at the same seed. 1.0 is identical.

The LoRA brings banding down to about Base's level while keeping the image over 99% similar to SFT.
Reverting whole layers changes the image more and leaves the stripe in more images. On the 1024 px sky
image above, at strength 1.0, bandviz scores 23.7 for SFT (stripes every 21.4 px) and 9.3 with the LoRA.

**See it side by side:** MATLOWAI's [DeBand comparison](https://matlowai.github.io/MOSS-DeBand/#cols=sft,deband_lora,nynxz_1.5,strong_full,sft&item=a_anime_girl_school&seed=11)
puts this LoRA next to SFT, his DeBand fixes and other candidates on the same prompts and seeds, with
synced pan and zoom. [Edit anatomy](https://matlowai.github.io/MOSS-DeBand/edit_anatomy.html) shows
where each fix changes the weights, and how much stripe each removes against how much of SFT it undoes.

### How it was made

The LoRA is extracted, not trained. Building it took four steps:

1. **Split SFT's change into directions.** For each of the 252 linear layers, take the change from
   Base to SFT and break it into directions (SVD).
2. **Score each direction's effect on the stripes.** Run SFT on 4 prompts × 2 seeds at the official
   settings. For steps 5–14, where the stripes form, compute the stripe energy of the model's prediction
   in latent space, unpacked to the VAE's 8× grid, where the 48-cycles-per-width stripe is visible. One
   backward pass then scores every direction in every layer: how much the stripes grow per unit of that
   direction.
3. **Remove the top 64.** The 64 strongest stripe-feeding directions are subtracted from SFT. They sit
   in layers 24–41 and are mostly the largest parts of SFT's change to the MLP (`ffn_up_gate`), plus a
   few in `ffn_down`, q, k, v and o. As a LoRA that is 25 small matrices with ranks 1 to 7.
4. **Validate on unseen prompts and seeds** (table above).

The localization matches MATLOWAI's block-swap analysis, which found the stripes are carried by layers
32–41. This LoRA removes only the few directions inside those layers that feed the stripes, instead of
reverting whole layers.

**Limits:**

- Tested at 768 and 1024 px with ER-SDE / 25 steps / CFG 2.5. Other samplers and resolutions are
  untested.
- Seed-to-seed variation in banding is large. The numbers above are medians over 12 images.

Credits: banding analysis and the bandviz metric by
[MATLOWAI](https://huggingface.co/MATLOWAI/MOSS-Image-SFT-DeBand).

---

## Split q/k/v (non-fused) checkpoints

The same MOSS-Image weights as [Comfy-Org/MOSS-Image](https://huggingface.co/Comfy-Org/MOSS-Image),
packed for ComfyUI with each attention layer's **q, k and v kept as separate tensors** instead of
fused into one `qkv` tensor. Use them to check whether a LoRA or LoKr behaves differently on the
fused files, or as the base for LoRAs trained on separate q/k/v projections.

Load these files with **Load Diffusion Model** in place of the Comfy-Org ones. Everything else in the
workflow stays the same.

### How they differ from Comfy-Org/MOSS-Image

|  | Comfy-Org `moss_image_*_bf16` | This repo `moss_image_*_bf16_split_qkv` |
| --- | --- | --- |
| Attention weights per layer | `layers.N.proj.qkv.weight` `[9216, 3072]` | `layers.N.proj.q.weight`, `.k.weight`, `.v.weight`, each `[3072, 3072]` |
| Tensors | 379 | 463 (42 layers × 3 instead of 1) |
| Everything else | bf16 | identical |

The weights are the same numbers in a different layout: for every layer, `cat(q, k, v)` here
equals `qkv` in the Comfy-Org file, bit for bit. Spot-checked on layers 0 and 41, plus the
output projection, for both Base and SFT. ComfyUI loads either layout as is, because the MOSS
model detects fused or split q/k/v from the keys. Fused is one matmul per layer instead of three,
so it can be a little faster. Image quality is the same.

### Why this matters for LoRAs and LoKrs

Trainers such as AI Toolkit train adapters against **separate** `q`, `k` and `v` layers. On the
fused Comfy-Org files, ComfyUI has to map each of those adapters onto a third of the fused `qkv`
weight:

- **Load LoRA (normal patching):** the adapter is applied to its slice of `qkv`. This works, but
  only for adapter keys named `diffusion_model.layers.N.proj.{q,k,v}` or
  `lora_unet_layers_N_proj_{q,k,v}`. Keys under any other name are skipped, with a
  `lora key not loaded` line in the console.
- **Load LoRA (Bypass):** adapters that target part of a fused weight are dropped. The console
  logs `[BypassLoRA] Adapter key not in model state_dict` for each one, so a LoRA or LoKr loses
  all of its attention q/k/v changes. (ComfyUI `fc04da91`: the patch for a slice is stored under
  a `(key, slice)` tuple, which the bypass loader looks up as a plain state-dict key.)

With the split checkpoints every q/k/v adapter targets its own tensor directly, with no slicing,
in either loader. If a LoRA or LoKr looks noticeably different in ComfyUI than in the trainer's
own samples, try it on these files and check the console for the two messages above.

## Licence

Apache 2.0, as the original MOSS-Image release.

	---
	tags:
	- comfyui
	- diffusion-single-file
	- lora
	base_model:
	- OpenMOSS-Team/MOSS-Image-Base
	- OpenMOSS-Team/MOSS-Image-SFT
	license: apache-2.0
	---

	# MOSS-Image: SFT Destripe LoRA + split q/k/v checkpoints

	This repo has two things for running MOSS-Image in ComfyUI:

	- [SFT Destripe](#sft-destripe-lora), a 1.9 MB LoRA that removes the faint vertical color stripes MOSS-Image SFT draws in smooth areas.
	- [Split q/k/v checkpoints](#split-qkv-non-fused-checkpoints): the same weights as Comfy-Org's, with attention q, k and v stored as separate tensors.

	Original model repositories:

	- https://huggingface.co/OpenMOSS-Team/MOSS-Image-Base
	- https://huggingface.co/OpenMOSS-Team/MOSS-Image-SFT

	## Files

	```
	📂 ComfyUI/
	├── 📂 models/
	│ ├── 📂 loras/
	│ │ └── moss_sft_destripe_top64_bf16.safetensors
	│ ├── 📂 diffusion_models/
	│ │ ├── moss_image_base_bf16_split_qkv.safetensors
	│ │ └── moss_image_sft_bf16_split_qkv.safetensors
	```

	\| File \| Size \| SHA-256 \|
	\| --- \| --- \| --- \|
	\| `loras/moss_sft_destripe_top64_bf16.safetensors` \| 1.9 MB \| `fcd0d025f6a62dfe3171ab11fe9b3119eac735e22b94913c4efe45d8c69c2da9` \|
	\| `diffusion_models/moss_image_base_bf16_split_qkv.safetensors` \| 10.2 GB \| `65a698df38fbee8307a16059958f6dd1fef0db69c1e3805fe13ec1498cb0499e` \|
	\| `diffusion_models/moss_image_sft_bf16_split_qkv.safetensors` \| 10.2 GB \| `9373f43d0fdad7d6d62b7dc2d96b2a048aab47f9c6e7befe11456b5ec26f70fc` \|

	Get the text encoder (`text_encoders/`) and VAE (`vae/`) from
	[Comfy-Org/MOSS-Image](https://huggingface.co/Comfy-Org/MOSS-Image). They are unchanged.

	---

	## SFT Destripe LoRA

	MOSS-Image SFT often paints evenly spaced vertical color stripes across smooth areas such as
	skies, fog, walls and studio backdrops. The stripes are about 21 px apart at 1024 px wide and 16 px
	apart at 768 px wide. They are a shift in color, not brightness, so they are easy to miss at a glance
	and obvious once color is boosted.

	`moss_sft_destripe_top64_bf16.safetensors` removes them while keeping nearly all of what SFT does.

	Builds on [MATLOWAI's DeBand investigation](https://huggingface.co/MATLOWAI/MOSS-Image-SFT-DeBand). His block-swap analysis found the layers that carry the stripes, and his bandviz tool is the banding score used below. His fix reverts whole layers. This LoRA removes only the few directions inside those layers that feed the stripes.

	![SFT vs SFT + Destripe LoRA, color-only view](assets/deband_colour_view.png)

	*Same prompt and seed (MOSS SFT int8, ER-SDE, 25 steps, CFG 2.5, skip-layer guidance on layer 22).
	Left: the image. Right: the color-only view from [bandviz](https://huggingface.co/MATLOWAI/MOSS-Image-SFT-DeBand)
	(brightness removed, color ×5). The SFT sky (top) has regular green and purple vertical stripes. With
	the LoRA (bottom) it is clean.*

	### Usage

	1. Load the SFT model: Comfy-Org's `moss_image_sft_bf16` or `moss_image_sft_int8_convrot`, or
	`moss_image_sft_bf16_split_qkv` from this repo.
	2. Add LoraLoaderModelOnly with this file.
	3. Set the strength by resolution:
	- 1.0 at 768 px
	- 1.5 at 1024 px
	4. Sample as usual. Everything else in the workflow stays the same.

	At 1024 px the stripes are wider, and 1.0 leaves a faint trace in about half of images. 1.5 clears it
	(stripe left in 1 of 12 test images instead of 6 of 12) for a tiny cost in similarity to SFT. 2.0 is no
	better than 1.5 and moves further from SFT. The LoRA is built from SFT's own changes, so use it on SFT
	only, not on Base.

	Bypass mode on fused files: the LoRA touches q, k and v in a few layers. ComfyUI's normal LoRA
	loading handles that on the fused Comfy-Org checkpoints. Load LoRA (Bypass) drops q/k/v adapters on
	fused files (see [below](#why-this-matters-for-loras-and-lokrs)). For bypass mode, use the split
	checkpoints from this repo.

	### How well it works

	Blind test: 4 prompts with large smooth areas (sand dunes, a plain wall, open water, a portrait on a
	blurred backdrop) × 3 seeds, none of them used to build the fix. ER-SDE, 25 steps, CFG 2.5.

	768 px

	\| Model \| bandviz score (median) \| Stripe still the strongest peak \| Similarity to SFT (SSIM) \| Size \|
	\| --- \| --- \| --- \| --- \| --- \|
	\| SFT \| 9.8 \| 9 / 12 \| 1.000 \| — \|
	\| SFT, layers 37–41 reverted to Base ([MATLOWAI's fix](https://huggingface.co/MATLOWAI/MOSS-Image-SFT-DeBand)) \| 6.5 \| 8 / 12 \| 0.980 \| 1.13 GB \|
	\| SFT + this LoRA @ 1.0 \| 4.2 \| 1 / 12 \| 0.995 \| 1.9 MB \|
	\| Base \| 4.2 \| 5 / 12 \| 0.876 \| — \|

	1024 px

	\| Model \| bandviz score (median) \| Stripe still the strongest peak \| Similarity to SFT (SSIM) \| Size \|
	\| --- \| --- \| --- \| --- \| --- \|
	\| SFT \| 24.0 \| 11 / 12 \| 1.000 \| — \|
	\| SFT, layers 37–41 reverted to Base (MATLOWAI's fix) \| 8.4 \| 4 / 12 \| 0.983 \| 1.13 GB \|
	\| SFT + this LoRA @ 1.0 \| 6.1 \| 6 / 12 \| 0.996 \| 1.9 MB \|
	\| SFT + this LoRA @ 1.5 \| 5.9 \| 1 / 12 \| 0.993 \| 1.9 MB \|
	\| SFT + this LoRA @ 2.0 \| 6.0 \| 1 / 12 \| 0.992 \| 1.9 MB \|
	\| Base \| 4.0 \| 0 / 12 \| 0.863 \| — \|

	- bandviz score: MATLOWAI's metric. The column color spectrum's strongest peak at 10–48 px, divided
	by the median. Lower means less banding, and about 16 or below looks clean.
	- Stripe still the strongest peak: the number of images whose spectrum still peaks at the stripe
	width (16 px at 768, 21.3 px at 1024).
	- SSIM: how close the image stays to plain SFT at the same seed. 1.0 is identical.

	The LoRA brings banding down to about Base's level while keeping the image over 99% similar to SFT.
	Reverting whole layers changes the image more and leaves the stripe in more images. On the 1024 px sky
	image above, at strength 1.0, bandviz scores 23.7 for SFT (stripes every 21.4 px) and 9.3 with the LoRA.

	See it side by side: MATLOWAI's [DeBand comparison](https://matlowai.github.io/MOSS-DeBand/#cols=sft,deband_lora,nynxz_1.5,strong_full,sft&item=a_anime_girl_school&seed=11)
	puts this LoRA next to SFT, his DeBand fixes and other candidates on the same prompts and seeds, with
	synced pan and zoom. [Edit anatomy](https://matlowai.github.io/MOSS-DeBand/edit_anatomy.html) shows
	where each fix changes the weights, and how much stripe each removes against how much of SFT it undoes.

	### How it was made

	The LoRA is extracted, not trained. Building it took four steps:

	1. Split SFT's change into directions. For each of the 252 linear layers, take the change from
	Base to SFT and break it into directions (SVD).
	2. Score each direction's effect on the stripes. Run SFT on 4 prompts × 2 seeds at the official
	settings. For steps 5–14, where the stripes form, compute the stripe energy of the model's prediction
	in latent space, unpacked to the VAE's 8× grid, where the 48-cycles-per-width stripe is visible. One
	backward pass then scores every direction in every layer: how much the stripes grow per unit of that
	direction.
	3. Remove the top 64. The 64 strongest stripe-feeding directions are subtracted from SFT. They sit
	in layers 24–41 and are mostly the largest parts of SFT's change to the MLP (`ffn_up_gate`), plus a
	few in `ffn_down`, q, k, v and o. As a LoRA that is 25 small matrices with ranks 1 to 7.
	4. Validate on unseen prompts and seeds (table above).

	The localization matches MATLOWAI's block-swap analysis, which found the stripes are carried by layers
	32–41. This LoRA removes only the few directions inside those layers that feed the stripes, instead of
	reverting whole layers.

	Limits:

	- Tested at 768 and 1024 px with ER-SDE / 25 steps / CFG 2.5. Other samplers and resolutions are
	untested.
	- Seed-to-seed variation in banding is large. The numbers above are medians over 12 images.

	Credits: banding analysis and the bandviz metric by
	[MATLOWAI](https://huggingface.co/MATLOWAI/MOSS-Image-SFT-DeBand).

	---

	## Split q/k/v (non-fused) checkpoints

	The same MOSS-Image weights as [Comfy-Org/MOSS-Image](https://huggingface.co/Comfy-Org/MOSS-Image),
	packed for ComfyUI with each attention layer's q, k and v kept as separate tensors instead of
	fused into one `qkv` tensor. Use them to check whether a LoRA or LoKr behaves differently on the
	fused files, or as the base for LoRAs trained on separate q/k/v projections.

	Load these files with Load Diffusion Model in place of the Comfy-Org ones. Everything else in the
	workflow stays the same.

	### How they differ from Comfy-Org/MOSS-Image

	\| \| Comfy-Org `moss_image__bf16` \| This repo `moss_image__bf16_split_qkv` \|
	\| --- \| --- \| --- \|
	\| Attention weights per layer \| `layers.N.proj.qkv.weight` `[9216, 3072]` \| `layers.N.proj.q.weight`, `.k.weight`, `.v.weight`, each `[3072, 3072]` \|
	\| Tensors \| 379 \| 463 (42 layers × 3 instead of 1) \|
	\| Everything else \| bf16 \| identical \|

	The weights are the same numbers in a different layout: for every layer, `cat(q, k, v)` here
	equals `qkv` in the Comfy-Org file, bit for bit. Spot-checked on layers 0 and 41, plus the
	output projection, for both Base and SFT. ComfyUI loads either layout as is, because the MOSS
	model detects fused or split q/k/v from the keys. Fused is one matmul per layer instead of three,
	so it can be a little faster. Image quality is the same.

	### Why this matters for LoRAs and LoKrs

	Trainers such as AI Toolkit train adapters against separate `q`, `k` and `v` layers. On the
	fused Comfy-Org files, ComfyUI has to map each of those adapters onto a third of the fused `qkv`
	weight:

	- Load LoRA (normal patching): the adapter is applied to its slice of `qkv`. This works, but
	only for adapter keys named `diffusion_model.layers.N.proj.{q,k,v}` or
	`lora_unet_layers_N_proj_{q,k,v}`. Keys under any other name are skipped, with a
	`lora key not loaded` line in the console.
	- Load LoRA (Bypass): adapters that target part of a fused weight are dropped. The console
	logs `[BypassLoRA] Adapter key not in model state_dict` for each one, so a LoRA or LoKr loses
	all of its attention q/k/v changes. (ComfyUI `fc04da91`: the patch for a slice is stored under
	a `(key, slice)` tuple, which the bypass loader looks up as a plain state-dict key.)

	With the split checkpoints every q/k/v adapter targets its own tensor directly, with no slicing,
	in either loader. If a LoRA or LoKr looks noticeably different in ComfyUI than in the trainer's
	own samples, try it on these files and check the console for the two messages above.

	## Licence

	Apache 2.0, as the original MOSS-Image release.