Hugging Face's logo

thedeoxen
/
MOSS-Image-normal-controlnet

Image-to-Image
Diffusers
English
moss-image
controlnet
lora
comfyui
normal
Model card Files
xet
 Community
MOSS-Image-normal-controlnet / README.md
thedeoxen's picture
thedeoxen
Add example strips
0d718f4 verified
preview code
|
Raw History Blame Contribute Delete
4.32 kB
metadata 
license: apache-2.0
base_model: OpenMOSS-Team/MOSS-Image-SFT
base_model_relation: adapter
language:
  - en
pipeline_tag: image-to-image
tags:
  - moss-image
  - controlnet
  - lora
  - comfyui
  - normal
library_name: diffusers

MOSS-Image-SFT — Normal Control LoRA

Short description

A normal-map Control LoRA for MOSS-Image-SFT. Feed a color normal map. The model follows surface direction and volume, and the prompt sets the material and the scene.

This is a ControlNet-style edit LoRA, not the official MOSS color/depth concat adapter. Load it with a LoRA loader and pass one control image through Text Encode MOSS-Image Edit.

Selected checkpoint: training step 5000.

Files

Examples

Each preview is one combined image, left to right: control, then three prompts.

Control → Results

Extended description

Follows surface orientation from a normal map, so volume and facing stay put while the material changes. Appearance, materials, clothing and the world come from the text prompt.

The LoRA is rank 64 on linear layers and rank 16 on convolutions. It was trained as an edit adapter: the control image is the reference, and the prompt describes the picture to generate.

How to use

  1. Load MOSS-Image-SFT: UNet moss_image_sft_int8_convrot.safetensors, text encoder qwen3.5_9b_w4a8_gptq.safetensors with type moss_image, VAE flux2-vae_small.safetensors.
  2. Load this LoRA on the UNet at strength 1.0.
  3. Build the control map and pass it as the single reference image in Text Encode MOSS-Image Edit.
  4. Set edit format to ai-toolkit on both the positive and the negative encode. This LoRA was trained with kv cache on.
  5. Sample with 30 steps, CFG 4, euler + simple.

No trigger word. The control image sets the structure. The prompt sets identity, materials, clothing and the world.

ComfyUI

The workflow in this repo loads a photo, builds the control map, encodes it with the prompt, and samples.

The included workflow uses BAE Normal Map. The training maps are the usual purple, cyan and green camera-space normals.

Also uses the built-in Text Encode MOSS-Image Edit node. Set the edit format to ai-toolkit.

Base model

Trained on and recommended with OpenMOSS-Team/MOSS-Image-SFT.

Typical settings from the included workflow: 30 steps, CFG 4, euler + simple, LoRA strength 1.0.

Negative prompt used in training samples:

nsfw, worst quality, bad anatomy, watermark, jpeg artifact

Example prompt 

Full-body photograph of a circus ringmaster in scarlet tails, gold braid and a tall black hat, standing three-quarter view in front of a painted circus tent. One arm raised high, the other holding a slender black cane down at the side. Confetti, saturated color, entire body in frame.

What it does

  • Follows surface orientation from a normal map, so volume and facing stay put while the material changes.
  • Lets the prompt define identity, materials, clothing and style.
  • Works as a single-image control adapter for MOSS-Image-SFT.

Tips

  • Use a camera-space normal map, not a grayscale height map.
  • Smooth normals give cleaner volumes. Noisy normals add unwanted bumps.
  • Good when the form must stay and the surface material should change.
  • Lower the LoRA strength toward 0.7 if the control is too rigid. Strength 1.0 is the trained value.

Use cases

  • Relighting and retexturing a solid form.
  • Product shots where the object volume is fixed.
  • Scenes where depth is right but the surface facing also matters.