moss-serenity / docs /generating.md
malcolmrey's picture
int8-convrot version
70f37fc
|
Raw History Blame Contribute Delete
7.38 kB
# Generating with MOSS Serenity
Everything needed to run the model in ComfyUI, the settings the samples were made with, and how to
prompt it. Back to the [model card](../README.md).
## Files
| ComfyUI folder | File |
|---|---|
| `models/diffusion_models` | `serenity_base_v1_24000_bf16.safetensors` or `serenity_base_v1_24000_int8_convrot.safetensors` (this repository; the INT8 one is half the size and faster) |
| `models/text_encoders` | `qwen3.5_9b_int8_convrot.safetensors` (what the model trained with), or [`qwen3.5_9b_bf16.safetensors`](https://huggingface.co/Comfy-Org/Qwen3.5/blob/main/text_encoders/qwen3.5_9b_bf16.safetensors) |
| `models/vae` | `flux2_vae.safetensors` (the FLUX.2 VAE) |
| `models/loras` | optional: lodestones' [`MOSS-Image-Base-8-steps-dmd`](https://huggingface.co/lodestones/MOSS-Image-Base-8-steps-dmd) for the turbo workflows; the [extracted LoRAs](../lora-extract) |
Use the **Base** DMD LoRA, not the SFT one: Serenity was finetuned from MOSS-Image Base.
## The workflows
All four are in [`workflow/`](../workflow); drag one into ComfyUI and pick your own paths in the
loaders (the model and LoRA paths are the author's folders).
### Turbo: `base-turbo-no-lora.json`
Load Diffusion Model β†’ Power Lora Loader (empty, for your own LoRAs) β†’ Load LoRA (DMD, 1.0) β†’
SkipLayerGuidanceDiTSimple β†’ KSampler.
| Node | Setting |
|---|---|
| Load CLIP | `qwen3.5_9b_int8_convrot`, type `moss_image` |
| Load LoRA (model only) | `MOSS-Image-Base-8-steps-dmd`, strength 1.00 |
| SkipLayerGuidanceDiTSimple | `single_layers` 22, `double_layers` empty, start 0.0, end 0.5 |
| Negative | `bad ugly blurry` |
| EmptyFlux2LatentImage | 1584 Γ— 1776 |
| KSampler | 8 steps, CFG 4.0, euler, simple, denoise 1.00 |
About 16 s per image on an RTX 5090. This is the setup the portrait samples were made with: not
necessarily the best possible, but very good, and fast.
### Full: `base-no-turbo-no-lora.json`
The same without the DMD LoRA, at 50 steps:
| Node | Setting |
|---|---|
| SkipLayerGuidanceDiTSimple | layer 22, start 0.0, end 0.5 |
| Negative | `bad ugly blurry` |
| EmptyFlux2LatentImage | 1920 Γ— 1088 (Full HD, on the 16 px latent grid) |
| KSampler | 50 steps, CFG 4.0, euler, simple |
On an RTX 5090 at 1920 Γ— 1088: ~62 s at 50 steps, ~25 s at 20 (1.24 s per step either way).
Same seeds at 20 and 50 steps give the same composition; 50 adds fine structure (an insect's head,
lettering on signs, small figures). For landscapes 20 is close.
### Turbo with a person LoRA: `base-turbo-lora.json`
The turbo workflow with a separate person LoRA switched on in the **Power Lora Loader** (strength
1.0), for people Serenity was not trained on. MOSS-Image person LoRAs trained on Base load on top
of Serenity. Prompt them the way they were trained: this example's LoRA uses the class word as its
keyword, so the prompt starts `photo of a woman, ...` with no name trigger.
### Base + the extracted LoRA: `lora-extract-turbo-lora.json`
The original **MOSS-Image Base** (`moss_image_base_bf16.safetensors`) instead of Serenity, with
Serenity added back as a LoRA: Load Diffusion Model β†’ Power Lora Loader (`serenity_v1_lora_r256`,
strength 1.0) β†’ Load LoRA (DMD, 1.0) β†’ SkipLayerGuidanceDiTSimple β†’ KSampler at **12 steps**, CFG
4.0, euler / simple, 1584 Γ— 1776, negative `bad ugly blurry`. For when you keep the base model
around and switch LoRAs. The LoRA keeps only part of the finetune (see
[The extracted LoRAs](#the-extracted-loras)); the full model is the reference.
### Notes
- **Shift:** ComfyUI already runs MOSS-Image at shift 5. To change it, add **ModelSamplingAuraFlow**
before the KSampler. Not ModelSamplingSD3: its timestep scale is 1000Γ— off for MOSS-Image.
- **Size:** keep both sides multiples of 16. Portrait (1584 Γ— 1776) suits people; 1920 Γ— 1088 suits
scenes.
- **"widescreen" / "anamorphic" in a prompt** bake black letterbox bars into the image. Leave the
words out if you don't want bars.
## The extracted LoRAs
To keep one base model and switch LoRAs (the
[`lora-extract-turbo-lora.json`](../workflow/lora-extract-turbo-lora.json) workflow does this at 12 steps): Load Diffusion Model `moss_image_base_bf16` β†’ Power Lora Loader or Load LoRA
(`serenity_v1_lora_r256`, strength 1.0) β†’ (DMD LoRA for turbo) β†’ SkipLayerGuidanceDiTSimple β†’
KSampler, with the settings above. Higher ranks keep more of the finetune; compare with the full
model on the same seed.
## Prompting
### The shape
The captions were `photo of <trigger>, <one sentence>`, so prompts work best the same way:
```
photo of <trigger>, a <woman|man>, <look>, <expression>, <framing>, <clothing>, <setting>, <light>
```
The trigger is one lower-case word, the person's name without spaces (the list is in the
[model card](../README.md#characters)). The class word after it (`a woman`, `a man`) matches how
the model was trained.
### Examples
```
photo of jenniferlawrence, a woman with long dark brown hair in a single side braid, a determined
serious expression, looking slightly off-camera to her left, medium close-up from the chest up,
holding a wooden recurve bow with an arrow drawn, a quiver strapped over her shoulder, wearing a dark
green hooded hunting jacket over a gray shirt, standing in a dense misty pine forest, dappled
morning sunlight through the trees, cool green and amber tones, cinematic film still, shallow depth
of field
```
```
photo of nathanfillion, a man with short dark hair, a wry lopsided smile, standing in the cargo bay
of a worn frontier spaceship, wearing a long brown leather duster coat over suspenders and a red
shirt, a holstered pistol at his hip, warm amber light, medium shot, TV series still
```
```
photo of taylorswift, a woman with long blonde hair and bangs, bold red lipstick, a soft joyful
expression with her eyes looking toward the crowd, close-up of her face and shoulders, holding a
microphone near her chin, wearing a glittering silver sequined outfit, bokeh of thousands of
colorful wristband lights behind her, warm golden spotlight
```
### What works
- **Large faces.** Chest-up or closer. In a full-body stadium shot the face was a few dozen pixels
and came out wrong; the same scene as a close-up was clean.
- **Describe the role, not the title.** The model knows the person, not the film. Costume, hair and
setting make the scene: the side braid, bow and pine forest above read as a well-known role
without naming it.
- **Name the hair when it matters.** A prompt that does not mention someone's signature hair can
lose it (red hair especially).
- **Two people: a man and a woman work; two of the same gender are hit-or-miss.** Name both with their
class word, place them (`on the left ...; on the right ...`) and describe each separately. A
man and a woman keep their own faces; two women (or two men) always keep their costumes, but in some
seeds their faces drift toward each other. Three people follow the same rule: one woman with two
men holds, three women converge on one face. See [Two people in one shot](../README.md#two-people-in-one-shot)
and [Three people in one shot](../README.md#three-people-in-one-shot).
- **No trigger, no person:** `photo of a woman, ...` gives an ordinary person; prior preservation
kept the class words close to the base model.
- **Seeds:** likeness varies a little by seed; try a few.