thedeoxen/MOSS-Image-pose-controlnet

MOSS-Image-SFT — OpenPose / DWPose Control LoRA

Short description

An OpenPose Control LoRA for MOSS-Image-SFT. Feed a DWPose skeleton on a black background. The model follows the body pose, and the prompt sets the character, clothing and scene.

This is a ControlNet-style edit LoRA, not the official MOSS color/depth concat adapter. Load it with a LoRA loader and pass one control image through Text Encode MOSS-Image Edit.

Selected checkpoint: training step 4500.

Files

• Weights: moss_image_sft_pose_control_lora.safetensors
• ComfyUI workflow: moss_image_pose_control.json
• Repo: thedeoxen/MOSS-Image-pose-controlnet

Examples

Each preview is one combined image, left to right: control, then three prompts.

Control → Results

Extended description

Follows body, hand and face keypoints from an OpenPose / DWPose skeleton. Appearance, materials, clothing and the world come from the text prompt.

The LoRA is rank 64 on linear layers and rank 16 on convolutions. It was trained as an edit adapter: the control image is the reference, and the prompt describes the picture to generate.

How to use

1. Load MOSS-Image-SFT: UNet moss_image_sft_int8_convrot.safetensors, text encoder qwen3.5_9b_w4a8_gptq.safetensors with type moss_image, VAE flux2-vae_small.safetensors.
2. Load this LoRA on the UNet at strength 1.0.
3. Build the control map and pass it as the single reference image in Text Encode MOSS-Image Edit.
4. Set edit format to ai-toolkit on both the positive and the negative encode. This LoRA was trained with kv cache on.
5. Sample with 30 steps, CFG 4, euler + simple.

No trigger word. The control image sets the structure. The prompt sets identity, materials, clothing and the world.

ComfyUI

The workflow in this repo loads a photo, builds the control map, encodes it with the prompt, and samples.

The included workflow extracts the skeleton with DWPose (body, hands and face enabled) via comfyui_controlnet_aux. A colored skeleton on a black background matches the training controls.

Also uses the built-in Text Encode MOSS-Image Edit node. Set the edit format to ai-toolkit.

Base model

Trained on and recommended with OpenMOSS-Team/MOSS-Image-SFT.

Typical settings from the included workflow: 30 steps, CFG 4, euler + simple, LoRA strength 1.0.

Negative prompt used in training samples:

nsfw, worst quality, bad anatomy, watermark, jpeg artifact

Example prompt

Full-body anime illustration of a cyber idol with short white hair and a magenta visor. Upright fighting stance, one arm straight above the head, the other arm extended to the side, one knee lifted to the hip, the standing leg straight. Glass coat, rainy megacity, whole body in frame.

What it does

• Follows body, hand and face keypoints from an OpenPose / DWPose skeleton.
• Lets the prompt define identity, materials, clothing and style.
• Works as a single-image control adapter for MOSS-Image-SFT.

Tips

• Black background, colored bones, face and hands included.
• If a limb is missing in the skeleton, the prompt cannot reliably invent that joint.
• Describe the character and the world. The skeleton already carries the pose, so the prompt does not have to repeat every joint.
• Lower the LoRA strength toward 0.7 if the control is too rigid. Strength 1.0 is the trained value.

Use cases

• Posing a character while changing outfit and style.
• Turning a photo's pose into another person.
• Keyframes and storyboards where the body has to match.