---
license: apache-2.0
base_model: OpenMOSS-Team/MOSS-Image-SFT
base_model_relation: adapter
language:
- en
pipeline_tag: image-to-image
tags:
- moss-image
- controlnet
- lora
- comfyui
- scribble
library_name: diffusers
---

# MOSS-Image-SFT — Scribble Control LoRA

## Short description

A scribble Control LoRA for **MOSS-Image-SFT**. Feed loose white strokes on black. The model follows the rough drawing, and the prompt is free to invent texture and style.

This is a ControlNet-style **edit LoRA**, not the official MOSS color/depth concat adapter. Load it with a LoRA loader and pass one control image through **Text Encode MOSS-Image Edit**.

Selected checkpoint: training step **5000**.

## Files

- **Weights:** [moss_image_sft_scribble_control_lora.safetensors](https://huggingface.co/thedeoxen/MOSS-Image-scribble-controlnet/blob/main/moss_image_sft_scribble_control_lora.safetensors)
- **ComfyUI workflow:** [moss_image_scribble_control.json](https://huggingface.co/thedeoxen/MOSS-Image-scribble-controlnet/blob/main/moss_image_scribble_control.json)
- **Repo:** [thedeoxen/MOSS-Image-scribble-controlnet](https://huggingface.co/thedeoxen/MOSS-Image-scribble-controlnet)

## Examples

Each preview is one combined image, left to right: **control**, then three prompts.

| Control → Results |
| --- |
| ![](images/figure.jpg) |
| ![](images/espresso.jpg) |
| ![](images/ramen.jpg) |
| ![](images/subway.jpg) |


## Extended description

Follows a loose scribble: the major strokes and masses, not every fine edge. Appearance, materials, clothing and the world come from the text prompt.

The LoRA is rank 64 on linear layers and rank 16 on convolutions. It was trained as an edit adapter: the control image is the reference, and the prompt describes the picture to generate.

## How to use

1. Load **MOSS-Image-SFT**: UNet `moss_image_sft_int8_convrot.safetensors`, text encoder `qwen3.5_9b_w4a8_gptq.safetensors` with type `moss_image`, VAE `flux2-vae_small.safetensors`.
2. Load this LoRA on the UNet at strength **1.0**.
3. Build the control map and pass it as the single reference image in **Text Encode MOSS-Image Edit**.
4. Set **edit format** to `ai-toolkit` on both the positive and the negative encode. This LoRA was trained with kv cache on.
5. Sample with **30 steps**, **CFG 4**, `euler` + `simple`.

No trigger word. The control image sets the structure. The prompt sets identity, materials, clothing and the world.

## ComfyUI

The workflow in this repo loads a photo, builds the control map, encodes it with the prompt, and samples.

The included workflow uses **HED Soft-Edge Lines** with safe mode enabled. The training scribbles are white strokes on black, thicker and softer than Canny or lineart.

Also uses the built-in **Text Encode MOSS-Image Edit** node. Set the edit format to `ai-toolkit`.

### Base model

Trained on and recommended with **[OpenMOSS-Team/MOSS-Image-SFT](https://huggingface.co/OpenMOSS-Team/MOSS-Image-SFT)**.

Typical settings from the included workflow: **30 steps**, **CFG 4**, `euler` + `simple`, LoRA strength **1.0**.

Negative prompt used in training samples:

```
nsfw, worst quality, bad anatomy, watermark, jpeg artifact
```

### Example prompt

```
Full-body photograph of a circus ringmaster in scarlet tails, gold braid and a tall black hat, standing three-quarter view in front of a painted circus tent. One arm raised high, the other holding a slender black cane down at the side. Confetti, saturated color, entire body in frame.
```

## What it does

- Follows a loose scribble: the major strokes and masses, not every fine edge.
- Lets the prompt define identity, materials, clothing and style.
- Works as a single-image control adapter for MOSS-Image-SFT.

## Tips

- A few confident strokes beat a noisy edge map.
- HED soft edges are closer to the training scribbles than a thin PiDiNet sketch.
- Leave texture, color and small details to the prompt.
- Lower the LoRA strength toward 0.7 if the control is too rigid. Strength 1.0 is the trained value.

## Use cases

- Rough sketches and storyboard strokes.
- Fast composition tests before a tighter canny or lineart pass.
- Drawing on top of a photo when only the big shapes should remain.