| ---
|
| license: apache-2.0
|
| base_model: OpenMOSS-Team/MOSS-Image-SFT
|
| base_model_relation: adapter
|
| language:
|
| - en
|
| pipeline_tag: image-to-image
|
| tags:
|
| - moss-image
|
| - controlnet
|
| - lora
|
| - comfyui
|
| - scribble
|
| library_name: diffusers
|
| ---
|
|
|
| # MOSS-Image-SFT — Scribble Control LoRA
|
|
|
| ## Short description
|
|
|
| A scribble Control LoRA for **MOSS-Image-SFT**. Feed loose white strokes on black. The model follows the rough drawing, and the prompt is free to invent texture and style.
|
|
|
| This is a ControlNet-style **edit LoRA**, not the official MOSS color/depth concat adapter. Load it with a LoRA loader and pass one control image through **Text Encode MOSS-Image Edit**.
|
|
|
| Selected checkpoint: training step **5000**.
|
|
|
| ## Files
|
|
|
| - **Weights:** [moss_image_sft_scribble_control_lora.safetensors](https://huggingface.co/thedeoxen/MOSS-Image-scribble-controlnet/blob/main/moss_image_sft_scribble_control_lora.safetensors)
|
| - **ComfyUI workflow:** [moss_image_scribble_control.json](https://huggingface.co/thedeoxen/MOSS-Image-scribble-controlnet/blob/main/moss_image_scribble_control.json)
|
| - **Repo:** [thedeoxen/MOSS-Image-scribble-controlnet](https://huggingface.co/thedeoxen/MOSS-Image-scribble-controlnet)
|
|
|
| ## Examples
|
|
|
| Each preview is one combined image, left to right: **control**, then three prompts.
|
|
|
| | Control → Results |
|
| | --- |
|
| |  |
|
| |  |
|
| |  |
|
| |  |
|
|
|
|
|
| ## Extended description
|
|
|
| Follows a loose scribble: the major strokes and masses, not every fine edge. Appearance, materials, clothing and the world come from the text prompt.
|
|
|
| The LoRA is rank 64 on linear layers and rank 16 on convolutions. It was trained as an edit adapter: the control image is the reference, and the prompt describes the picture to generate.
|
|
|
| ## How to use
|
|
|
| 1. Load **MOSS-Image-SFT**: UNet `moss_image_sft_int8_convrot.safetensors`, text encoder `qwen3.5_9b_w4a8_gptq.safetensors` with type `moss_image`, VAE `flux2-vae_small.safetensors`.
|
| 2. Load this LoRA on the UNet at strength **1.0**.
|
| 3. Build the control map and pass it as the single reference image in **Text Encode MOSS-Image Edit**.
|
| 4. Set **edit format** to `ai-toolkit` on both the positive and the negative encode. This LoRA was trained with kv cache on.
|
| 5. Sample with **30 steps**, **CFG 4**, `euler` + `simple`.
|
|
|
| No trigger word. The control image sets the structure. The prompt sets identity, materials, clothing and the world.
|
|
|
| ## ComfyUI
|
|
|
| The workflow in this repo loads a photo, builds the control map, encodes it with the prompt, and samples.
|
|
|
| The included workflow uses **HED Soft-Edge Lines** with safe mode enabled. The training scribbles are white strokes on black, thicker and softer than Canny or lineart.
|
|
|
| Also uses the built-in **Text Encode MOSS-Image Edit** node. Set the edit format to `ai-toolkit`.
|
|
|
| ### Base model
|
|
|
| Trained on and recommended with **[OpenMOSS-Team/MOSS-Image-SFT](https://huggingface.co/OpenMOSS-Team/MOSS-Image-SFT)**.
|
|
|
| Typical settings from the included workflow: **30 steps**, **CFG 4**, `euler` + `simple`, LoRA strength **1.0**.
|
|
|
| Negative prompt used in training samples:
|
|
|
| ```
|
| nsfw, worst quality, bad anatomy, watermark, jpeg artifact
|
| ```
|
|
|
| ### Example prompt
|
|
|
| ```
|
| Full-body photograph of a circus ringmaster in scarlet tails, gold braid and a tall black hat, standing three-quarter view in front of a painted circus tent. One arm raised high, the other holding a slender black cane down at the side. Confetti, saturated color, entire body in frame.
|
| ```
|
|
|
| ## What it does
|
|
|
| - Follows a loose scribble: the major strokes and masses, not every fine edge.
|
| - Lets the prompt define identity, materials, clothing and style.
|
| - Works as a single-image control adapter for MOSS-Image-SFT.
|
|
|
| ## Tips
|
|
|
| - A few confident strokes beat a noisy edge map.
|
| - HED soft edges are closer to the training scribbles than a thin PiDiNet sketch.
|
| - Leave texture, color and small details to the prompt.
|
| - Lower the LoRA strength toward 0.7 if the control is too rigid. Strength 1.0 is the trained value.
|
|
|
| ## Use cases
|
|
|
| - Rough sketches and storyboard strokes.
|
| - Fast composition tests before a tighter canny or lineart pass.
|
| - Drawing on top of a photo when only the big shapes should remain.
|
|
|
