Keep public model documentation English only

37f3a4b verified 5 days ago

11.7 kB

---
language:
- en
license: other
license_name: upstream-anima-and-source-data-terms
license_link: https://huggingface.co/circlestone-labs/Anima/blob/main/LICENSE.md
base_model:
- circlestone-labs/Anima
datasets:
- toyxyz/booruregion_10k
- toyxyz/booru3dpose_10k
- toyxyz/boorupose_outline_10k
pipeline_tag: image-to-image
tags:
- anima
- controlnet
- lllite
- comfyui
- region-control
- 3d-pose
- pose-conditioning
- outline-conditioning
- instance-segmentation
- safetensors
---

# Anima ControlNet-LLLite

Lightweight ControlNet adapters trained on **[Anima Base 1.0](https://huggingface.co/circlestone-labs/Anima/blob/main/split_files/diffusion_models/anima-base-v1.0.safetensors)** (`anima-base-v1.0.safetensors`). Use Anima Base 1.0 as the base model with these adapters. See the [official Anima repository](https://huggingface.co/circlestone-labs/Anima) for the base model, text encoder and VAE. Choose a model by its control type; each model has a separate document with its training and evaluation details.

## Models and control types

| Model | Control type | Detailed documentation |
| --- | --- | --- |
| [anima_regional_4335.safetensors](anima_regional_4335.safetensors) | **RGB region control** for subject layout, silhouettes, poses, and object placement | [Model details and training parameters](docs/anima_regional_4335.md) |
| [anima_3dpose_6200.safetensors](anima_3dpose_6200.safetensors) | **3D pose control** from shaded RGB body meshes, including fingers | [Model details and training parameters](docs/anima_3dpose_6200.md) |
| [anima_3dpose_outline_7000.safetensors](anima_3dpose_outline_7000.safetensors) | **3D pose with outline** from shaded RGB body meshes plus complementary-color silhouette and overlap boundaries | [Model details and training parameters](docs/anima_3dpose_outline_7000.md) |

`anima_regional_4335` uses a **3-channel RGB instance-color map**: distinct solid colors identify regions, and black marks unassigned pixels. Colors are not fixed semantic labels. This is a region-conditioning adapter, not a black/white-mask inpainting model.

`anima_3dpose_6200` uses **colored, shaded 3D body meshes on black** to guide body layout and pose. The final adapter completed **6,200 updates** (4,335 + 1,865 full-state resumed updates). It uses the same Anima LLLite v2 format; pose and hand accuracy can still vary.

`anima_3dpose_outline_7000` uses **the shaded RGB mesh plus complementary-color outlines and body-overlap boundaries**, on black. It completed **7,000 fresh updates**; it is not the 6,200-step model continued for 800 more steps. Start by comparing the training-like **nominal 4px width at a 768px short-side reference**. The body shading remains part of the input: bare lineart, stick-figure OpenPose, depth maps and binary masks are different conditioning types.

### Combining 3D pose and regional control

**For multi-person scenes, using the 3D pose and regional adapters together can help achieve more stable separation and placement of individual characters.** The pose adapter guides body posture, while the regional adapter defines distinct spatial regions for people. The two forms of guidance complement each other; the combined setup has not been quantitatively evaluated in this release.

## Regional control example

![Regional control example: image on the left and colored region overlay on the right](examples/anima_regional_4335_example.png)

Image (left) and colored region overlay (right). Distinct colors show separate subject and object regions.

### Regional control example workflow and sample input

![Regional control ComfyUI workflow with input region map, node connections and generated preview](example_workflow/regional_example_workflow.png)

Sample RGB region map (768 × 1152):

<img src="example_workflow/regional_example.png" alt="Sample RGB instance-color region map" width="384">

Download the [example workflow JSON](example_workflow/regional_example.json) and [sample region map PNG](example_workflow/regional_example.png), or browse the [example_workflow folder and instructions](example_workflow/README.md). Import the JSON into ComfyUI, upload `regional_example.png` in **Load Image**, and select your installed model files. The example uses the 4,335-step adapter, 30 sampling steps, CFG 4 and strength 1.0.

## 3D pose control example

![3D pose mesh condition on the left and generated illustration on the right](examples/anima_3dpose_6200_example.png)

User-provided ComfyUI example: condition (left), generated image (right). Embedded workflow metadata is removed. See the [3D pose document](docs/anima_3dpose_6200.md) for training and evaluation limits.

### 3D pose example workflow and sample input

![3D pose ComfyUI workflow with input mesh, node connections and generated preview](example_workflow/3dpose_example_workflow.png)

Sample pose condition (768 × 1152):

<img src="example_workflow/3dpose_example.png" alt="Sample shaded RGB 3D body-mesh pose condition" width="384">

Download the [example workflow JSON](example_workflow/3dpose_example.json) and [sample pose PNG](example_workflow/3dpose_example.png), or browse the [example_workflow folder and instructions](example_workflow/README.md). Import the JSON into ComfyUI, upload `3dpose_example.png` in **Load Image**, and select your installed model files. The example uses the 6,200-step adapter, 30 sampling steps, CFG 4 and strength 1.0.

## 3D pose outline control and comparison

![Same pose and camera, shaded-mesh inputs above and six generated outputs below](examples/outline7000/comparison_23.jpg)

Columns: **pose6200 + no outline**, **outline7000 + no outline**, then outline7000 with **4 / 6 / 8 / 12px** nominal normalized strokes. A completed comparison used **50 poses and 300 images**, with the same seed, prompt, camera, framing and output size within each pose. It used the **miaomiaoHarem_anima16 derivative**, not Anima Base 1.0, at 30 sampling steps, CFG 4, er_sde/simple and strength 2.

**Thicker is not consistently better.** Some hand boundaries become clearer; other cases gain an unwanted extra person or lose a hand relationship. The original pose adapter can be better on individual cases. Start with 4px and compare 6px; assess 8/12px per scene. This is a qualitative one-seed experiment, not a quantitative model ranking.

[Full outline model document](docs/anima_3dpose_outline_7000.md) · [Representative successes and failures, inputs and outputs](examples/outline7000/README.md) · [Single 4px workflow](example_workflow/outline7000_single_4px.json) · [Six-way workflow](example_workflow/outline7000_six_way_comparison.json)

## Advanced workflow: 3D pose + regional control

![Advanced ComfyUI workflow combining a 3D pose editor, region painter, two ControlNet-LLLite adapters and regional prompting](example_workflow/advanced_region_3dpose_example_workflow.png)

Download the [advanced workflow JSON](example_workflow/advanced_region_3dpose_example.json). This example uses **[ComfyUI_toyxyz_test_nodes](https://github.com/toyxyz/ComfyUI_toyxyz_test_nodes)** for **3D Pose Editor**, **ComfyCouple Region painter**, and **ComfyCouple Mask**, together with [ComfyUI-Anima-LLLite](https://github.com/kohya-ss/ComfyUI-Anima-LLLite). It combines `anima_3dpose_6200` and `anima_regional_4335` on **Anima Base 1.0**, with per-person prompts and editable region masks.

**Keep the regional ControlNet's `end_percent` low** (the supplied example uses **0.3**) to end its influence earlier in sampling. Alternatively, **paint additional mask areas in ComfyCouple Region painter** to expand or refine each person's region. Adjust these areas to cover the intended character, including clothing beyond the body silhouette. The pose adapter uses `end_percent = 1.0`, and both adapters have `preserve_wrapper = true` so their controls can be combined. These are example settings; adjust them to the scene.

See the [advanced workflow instructions](example_workflow/README.md#advanced-workflow-3d-pose--regional-control) for setup and saved settings.

## How to use

1. Use an Anima-compatible ComfyUI setup with [ComfyUI-Anima-LLLite](https://github.com/kohya-ss/ComfyUI-Anima-LLLite).
2. Download the adapter and place it in the `controlnet` model search path (`ComfyUI/models/controlnet/`, or an external directory configured for `controlnet`).
3. Load Anima Base 1.0, its compatible text encoder and VAE. In **Apply Anima ControlNet-LLLite (sd-scripts)** (`AnimaLLLiteApply_sdscripts`), select the adapter and connect the model and RGB control image.
4. Connect the patched model to the sampler and use your usual positive/negative prompts. Keep the control image's aspect ratio aligned with the output canvas.

For `anima_regional_4335`, start with **strength 1.0**, **start/end 0.0/1.0**, **30 sampling steps**, and **CFG 4**. Use the same seed when comparing results.

For `anima_3dpose_6200`, use a **shaded RGB 3D-mesh condition**. Native evaluation used strength/multiplier **1.0**, **30 sampling steps**, **CFG 4** and **flow shift 3**; see [model-specific usage](docs/anima_3dpose_6200.md#use-in-comfyui).

## Dataset

`anima_regional_4335` was trained on [toyxyz/booruregion_10k](https://huggingface.co/datasets/toyxyz/booruregion_10k): paired target images, RGB region maps, and captions. The training split contains **10,000 pairs** (3,834 Danbooru originals and 6,166 Anima-generated images), with **120 validation** and **120 test** pairs.

Full dataset provenance, measured training time, steps, batch size, architecture, parameters, file hashes, evaluation limits, and source terms are recorded in the [model-specific document](docs/anima_regional_4335.md) and its [training JSON](docs/anima_regional_4335.training.json).

`anima_3dpose_6200` was trained on [toyxyz/booru3dpose_10k](https://huggingface.co/datasets/toyxyz/booru3dpose_10k): **10,080 training pairs** (6,220 Anima-generated and 3,860 Danbooru originals), with **10 validation** and **10 test** pairs, both generated-only. Details, measured timings, full-state continuation, file SHA256 and limitations are in its [document](docs/anima_3dpose_6200.md) and [training JSON](docs/anima_3dpose_6200.training.json).

`anima_3dpose_outline_7000` was trained on [toyxyz/boorupose_outline_10k](https://huggingface.co/datasets/toyxyz/boorupose_outline_10k): **10,080 train / 10 validation / 10 test** pairs. The training targets, captions, splits and dimensions match the 3D pose dataset, while all conditioning images differ. Training used fresh 7,000 updates and matched target-camera meshes with nominal 4px complementary-color strokes. [Settings, pinned dataset revision, hashes and evaluation](docs/anima_3dpose_outline_7000.md).

## Acknowledgements

We sincerely thank [Kohya S.](https://github.com/kohya-ss) for the ControlNet-LLLite training and inference implementations, publicly released weights, and detailed technical documentation. These adapters were trained with [sd-scripts](https://github.com/kohya-ss/sd-scripts) and use [ComfyUI-Anima-LLLite](https://github.com/kohya-ss/ComfyUI-Anima-LLLite) for the shared ComfyUI examples.

We also thank [CircleStone Labs](https://huggingface.co/circlestone-labs) and [Comfy Org](https://github.com/Comfy-Org) for releasing the [Anima](https://huggingface.co/circlestone-labs/Anima) base model.

## License

These weights are derivatives of Anima and are subject to the [CircleStone Labs Non-Commercial License](https://huggingface.co/circlestone-labs/Anima/blob/main/LICENSE.md). Please refer to the linked upstream license for the full terms.

	---
	language:
	- en
	license: other
	license_name: upstream-anima-and-source-data-terms
	license_link: https://huggingface.co/circlestone-labs/Anima/blob/main/LICENSE.md
	base_model:
	- circlestone-labs/Anima
	datasets:
	- toyxyz/booruregion_10k
	- toyxyz/booru3dpose_10k
	- toyxyz/boorupose_outline_10k
	pipeline_tag: image-to-image
	tags:
	- anima
	- controlnet
	- lllite
	- comfyui
	- region-control
	- 3d-pose
	- pose-conditioning
	- outline-conditioning
	- instance-segmentation
	- safetensors
	---

	# Anima ControlNet-LLLite

	Lightweight ControlNet adapters trained on [Anima Base 1.0](https://huggingface.co/circlestone-labs/Anima/blob/main/split_files/diffusion_models/anima-base-v1.0.safetensors) (`anima-base-v1.0.safetensors`). Use Anima Base 1.0 as the base model with these adapters. See the [official Anima repository](https://huggingface.co/circlestone-labs/Anima) for the base model, text encoder and VAE. Choose a model by its control type; each model has a separate document with its training and evaluation details.

	## Models and control types

	\| Model \| Control type \| Detailed documentation \|
	\| --- \| --- \| --- \|
	\| [anima_regional_4335.safetensors](anima_regional_4335.safetensors) \| RGB region control for subject layout, silhouettes, poses, and object placement \| [Model details and training parameters](docs/anima_regional_4335.md) \|
	\| [anima_3dpose_6200.safetensors](anima_3dpose_6200.safetensors) \| 3D pose control from shaded RGB body meshes, including fingers \| [Model details and training parameters](docs/anima_3dpose_6200.md) \|
	\| [anima_3dpose_outline_7000.safetensors](anima_3dpose_outline_7000.safetensors) \| 3D pose with outline from shaded RGB body meshes plus complementary-color silhouette and overlap boundaries \| [Model details and training parameters](docs/anima_3dpose_outline_7000.md) \|

	`anima_regional_4335` uses a 3-channel RGB instance-color map: distinct solid colors identify regions, and black marks unassigned pixels. Colors are not fixed semantic labels. This is a region-conditioning adapter, not a black/white-mask inpainting model.

	`anima_3dpose_6200` uses colored, shaded 3D body meshes on black to guide body layout and pose. The final adapter completed 6,200 updates (4,335 + 1,865 full-state resumed updates). It uses the same Anima LLLite v2 format; pose and hand accuracy can still vary.

	`anima_3dpose_outline_7000` uses the shaded RGB mesh plus complementary-color outlines and body-overlap boundaries, on black. It completed 7,000 fresh updates; it is not the 6,200-step model continued for 800 more steps. Start by comparing the training-like nominal 4px width at a 768px short-side reference. The body shading remains part of the input: bare lineart, stick-figure OpenPose, depth maps and binary masks are different conditioning types.

	### Combining 3D pose and regional control

	For multi-person scenes, using the 3D pose and regional adapters together can help achieve more stable separation and placement of individual characters. The pose adapter guides body posture, while the regional adapter defines distinct spatial regions for people. The two forms of guidance complement each other; the combined setup has not been quantitatively evaluated in this release.

	## Regional control example

	![Regional control example: image on the left and colored region overlay on the right](examples/anima_regional_4335_example.png)

	Image (left) and colored region overlay (right). Distinct colors show separate subject and object regions.

	### Regional control example workflow and sample input

	![Regional control ComfyUI workflow with input region map, node connections and generated preview](example_workflow/regional_example_workflow.png)

	Sample RGB region map (768 × 1152):

	<img src="example_workflow/regional_example.png" alt="Sample RGB instance-color region map" width="384">

	Download the [example workflow JSON](example_workflow/regional_example.json) and [sample region map PNG](example_workflow/regional_example.png), or browse the [example_workflow folder and instructions](example_workflow/README.md). Import the JSON into ComfyUI, upload `regional_example.png` in Load Image, and select your installed model files. The example uses the 4,335-step adapter, 30 sampling steps, CFG 4 and strength 1.0.

	## 3D pose control example

	![3D pose mesh condition on the left and generated illustration on the right](examples/anima_3dpose_6200_example.png)

	User-provided ComfyUI example: condition (left), generated image (right). Embedded workflow metadata is removed. See the [3D pose document](docs/anima_3dpose_6200.md) for training and evaluation limits.

	### 3D pose example workflow and sample input

	![3D pose ComfyUI workflow with input mesh, node connections and generated preview](example_workflow/3dpose_example_workflow.png)

	Sample pose condition (768 × 1152):

	<img src="example_workflow/3dpose_example.png" alt="Sample shaded RGB 3D body-mesh pose condition" width="384">

	Download the [example workflow JSON](example_workflow/3dpose_example.json) and [sample pose PNG](example_workflow/3dpose_example.png), or browse the [example_workflow folder and instructions](example_workflow/README.md). Import the JSON into ComfyUI, upload `3dpose_example.png` in Load Image, and select your installed model files. The example uses the 6,200-step adapter, 30 sampling steps, CFG 4 and strength 1.0.

	## 3D pose outline control and comparison

	![Same pose and camera, shaded-mesh inputs above and six generated outputs below](examples/outline7000/comparison_23.jpg)

	Columns: pose6200 + no outline, outline7000 + no outline, then outline7000 with 4 / 6 / 8 / 12px nominal normalized strokes. A completed comparison used 50 poses and 300 images, with the same seed, prompt, camera, framing and output size within each pose. It used the miaomiaoHarem_anima16 derivative, not Anima Base 1.0, at 30 sampling steps, CFG 4, er_sde/simple and strength 2.

	Thicker is not consistently better. Some hand boundaries become clearer; other cases gain an unwanted extra person or lose a hand relationship. The original pose adapter can be better on individual cases. Start with 4px and compare 6px; assess 8/12px per scene. This is a qualitative one-seed experiment, not a quantitative model ranking.

	[Full outline model document](docs/anima_3dpose_outline_7000.md) · [Representative successes and failures, inputs and outputs](examples/outline7000/README.md) · [Single 4px workflow](example_workflow/outline7000_single_4px.json) · [Six-way workflow](example_workflow/outline7000_six_way_comparison.json)

	## Advanced workflow: 3D pose + regional control

	![Advanced ComfyUI workflow combining a 3D pose editor, region painter, two ControlNet-LLLite adapters and regional prompting](example_workflow/advanced_region_3dpose_example_workflow.png)

	Download the [advanced workflow JSON](example_workflow/advanced_region_3dpose_example.json). This example uses [ComfyUI_toyxyz_test_nodes](https://github.com/toyxyz/ComfyUI_toyxyz_test_nodes) for 3D Pose Editor, ComfyCouple Region painter, and ComfyCouple Mask, together with [ComfyUI-Anima-LLLite](https://github.com/kohya-ss/ComfyUI-Anima-LLLite). It combines `anima_3dpose_6200` and `anima_regional_4335` on Anima Base 1.0, with per-person prompts and editable region masks.

	Keep the regional ControlNet's `end_percent` low (the supplied example uses 0.3) to end its influence earlier in sampling. Alternatively, paint additional mask areas in ComfyCouple Region painter to expand or refine each person's region. Adjust these areas to cover the intended character, including clothing beyond the body silhouette. The pose adapter uses `end_percent = 1.0`, and both adapters have `preserve_wrapper = true` so their controls can be combined. These are example settings; adjust them to the scene.

	See the [advanced workflow instructions](example_workflow/README.md#advanced-workflow-3d-pose--regional-control) for setup and saved settings.

	## How to use

	1. Use an Anima-compatible ComfyUI setup with [ComfyUI-Anima-LLLite](https://github.com/kohya-ss/ComfyUI-Anima-LLLite).
	2. Download the adapter and place it in the `controlnet` model search path (`ComfyUI/models/controlnet/`, or an external directory configured for `controlnet`).
	3. Load Anima Base 1.0, its compatible text encoder and VAE. In Apply Anima ControlNet-LLLite (sd-scripts) (`AnimaLLLiteApply_sdscripts`), select the adapter and connect the model and RGB control image.
	4. Connect the patched model to the sampler and use your usual positive/negative prompts. Keep the control image's aspect ratio aligned with the output canvas.

	For `anima_regional_4335`, start with strength 1.0, start/end 0.0/1.0, 30 sampling steps, and CFG 4. Use the same seed when comparing results.

	For `anima_3dpose_6200`, use a shaded RGB 3D-mesh condition. Native evaluation used strength/multiplier 1.0, 30 sampling steps, CFG 4 and flow shift 3; see [model-specific usage](docs/anima_3dpose_6200.md#use-in-comfyui).

	## Dataset

	`anima_regional_4335` was trained on [toyxyz/booruregion_10k](https://huggingface.co/datasets/toyxyz/booruregion_10k): paired target images, RGB region maps, and captions. The training split contains 10,000 pairs (3,834 Danbooru originals and 6,166 Anima-generated images), with 120 validation and 120 test pairs.

	Full dataset provenance, measured training time, steps, batch size, architecture, parameters, file hashes, evaluation limits, and source terms are recorded in the [model-specific document](docs/anima_regional_4335.md) and its [training JSON](docs/anima_regional_4335.training.json).

	`anima_3dpose_6200` was trained on [toyxyz/booru3dpose_10k](https://huggingface.co/datasets/toyxyz/booru3dpose_10k): 10,080 training pairs (6,220 Anima-generated and 3,860 Danbooru originals), with 10 validation and 10 test pairs, both generated-only. Details, measured timings, full-state continuation, file SHA256 and limitations are in its [document](docs/anima_3dpose_6200.md) and [training JSON](docs/anima_3dpose_6200.training.json).

	`anima_3dpose_outline_7000` was trained on [toyxyz/boorupose_outline_10k](https://huggingface.co/datasets/toyxyz/boorupose_outline_10k): 10,080 train / 10 validation / 10 test pairs. The training targets, captions, splits and dimensions match the 3D pose dataset, while all conditioning images differ. Training used fresh 7,000 updates and matched target-camera meshes with nominal 4px complementary-color strokes. [Settings, pinned dataset revision, hashes and evaluation](docs/anima_3dpose_outline_7000.md).

	## Acknowledgements

	We sincerely thank [Kohya S.](https://github.com/kohya-ss) for the ControlNet-LLLite training and inference implementations, publicly released weights, and detailed technical documentation. These adapters were trained with [sd-scripts](https://github.com/kohya-ss/sd-scripts) and use [ComfyUI-Anima-LLLite](https://github.com/kohya-ss/ComfyUI-Anima-LLLite) for the shared ComfyUI examples.

	We also thank [CircleStone Labs](https://huggingface.co/circlestone-labs) and [Comfy Org](https://github.com/Comfy-Org) for releasing the [Anima](https://huggingface.co/circlestone-labs/Anima) base model.

	## License

	These weights are derivatives of Anima and are subject to the [CircleStone Labs Non-Commercial License](https://huggingface.co/circlestone-labs/Anima/blob/main/LICENSE.md). Please refer to the linked upstream license for the full terms.