ElectronGirl/anima-ref-adapter

Anima Reference Adapter (v2)

Character and style from a reference image. Pose, place and framing from your prompt.

Showcase · Quick start · Guide · Speed · Nodes · How it works · Training code

English · 한국어 · 日本語

Give Anima one reference image and a prompt that describes only the scene. The adapter keeps who is drawn (hair, eyes, outfit, accessories) and much of how it is drawn; pose, background and composition follow the prompt. No character name, artist tag or outfit tags, so it works for original characters too. Two references side by side give two characters in one image.

This is not a plain LoRA. It needs the Anima Ref Adapter node from ArcAnima/ComfyUI-Anima-Ref-Adapter.

Development is paused for lack of funding. Further training is not possible for now, and the planned separate character and style references are not released. The training code, data generation and experiment log are public in anima-ref-adapter-research.

What's new in v2

⚡ About 2x faster than the pilot	2.1x per sampling step on an RTX PRO 6000. On an RTX 3060 Ti an 832x1216 image takes 65 s instead of 105 s: the reference adds 32 s instead of 72 s. The reference is encoded once per image instead of at every step.
🧩 One node	Anima Ref Adapter drops into the standard Anima workflow and works with KSampler and any other sampler. The weights download on first use.
👥 Two characters	Two references side by side, and a prompt that says who is on which side.
📚 More training data	15,600 images of 2,300 characters in 1,952 artist styles (pilot: 3,600 images, 300 characters), rank 64.

Showcase

The one-node workflow with Anima's default settings: er_sde, 30 steps, cfg 4.5, ref_guidance 4.5 with APG, prompt masterpiece, best quality, score_7, safe, 1girl, solo, <scene> and the negative prompt from the Anima model card. No other LoRAs. One seed per image, not cherry-picked, except the lying pose (the best of six).

One reference

_{cowboy shot, sitting, cafe, holding cup, coffee, smile, looking at viewer, indoors, window, day  ·  full body, lying, on side, on bed, head rest, looking at viewer, smile, bedroom, indoors, night, lamp light full body, flying, from below, sky, clouds, sunlight, dynamic pose, outstretched arm  ·  sitting, on floor, hugging own legs, sleepy, hangar, indoors, dim lighting cowboy shot, beach, ocean, sunset, wind, hand on own hat, sun hat, smile, looking back  ·  upper body, sitting, poolside, legs in water, splashing, laughing, day}

Two characters

Put the two references side by side in one image (the core Image Stitch node), write who is on the left and who is on the right with a few words each, use 2girls or 1boy, 1girl instead of solo, and a landscape size. Workflow.

_{2girls, the girl on the left is a policewoman with black hair over one eye, the girl on the right is a mecha girl with aqua hair and mechanical wings, standing side by side, peace sign, smile, city, street, day 2girls, the girl on the left is a policewoman with black hair over one eye, the girl on the right is a girl with short black hair in a blue frilled bikini, sitting together on a bench, eating ice cream, park, day 2girls, the girl on the left is a mecha girl with aqua hair and mechanical wings, the girl on the right is a girl with short black hair in a blue frilled bikini, high five, beach, ocean, day}

Quick start

cd ComfyUI/custom_nodes
git clone https://github.com/ArcAnima/ComfyUI-Anima-Ref-Adapter.git

1. Get Anima base v1.0, the Qwen3-0.6B text encoder and the Qwen-Image VAE from circlestone-labs/Anima, as for Anima itself.
2. Restart ComfyUI and open workflows/anima_ref_adapter.json: the Anima text-to-image workflow plus one node. No other custom nodes are needed.
3. Load a reference image, write a prompt that describes only the scene, and queue. The first run downloads anima_ref_adapter_v2_r64.safetensors (183 MB) from Hugging Face into ComfyUI/models/loras/; to install it by hand, download it into that folder.

In your own workflow: put Anima Ref Adapter between the model (after any other LoRAs) and the sampler, and connect the VAE and the reference image.

Step by step, with prompt tips and troubleshooting: user guide.

Speed and compatibility

RTX 3060 Ti 8 GB (fp16), 832x1216, one reference; the faster of two runs:

setup	time per image
Anima base, er_sde 30 steps, cfg 4.5, no reference	33 s
+ pilot adapter, ref_guidance 4.5	105 s
+ v2 adapter, ref_guidance 4.5	65-67 s
ref_guidance 1 (no extra model call)	50 s
+ SageAttention (KJNodes Patch Sage Attention)	62 s
+ EasyCache (core)	63 s
ref_end 0.6	62 s
turbo LoRA (strength 1), 10 steps, cfg 1, ref_guidance 1	11 s

• SageAttention, EasyCache and ref_end 0.6 each save about 7% on this GPU, with nearly the same image.
• A turbo LoRA for Anima base (tested: anima-turbo-lora-v0.2) goes in front of the adapter. It keeps the characters but pulls the drawing toward its own look, so less of the reference style comes through.

The whole showcase with the turbo LoRA (10 steps, cfg 1)

Not tested: the Anima-Turbo and Anima-Aesthetic full models, SamplerSPEED, torch.compile.

Nodes

node
Anima Ref Adapter	The one to use. MODEL + VAE + reference IMAGE + adapter LoRA -> MODEL for any sampler. The released adapter files are listed before they are downloaded and fetched on first use. The reference is resized to ref_megapixels (1.0) with its aspect ratio. ref_guidance runs as a post-CFG patch, one extra model call per step when it is not 1; ref_end stops that call early.
Anima Ref Adapter Loader, Anima Ref Latent, Anima Ref Guider	The separate nodes, for the pilot adapter (it needs the reference at the output size) and the guider options ref_start and outside. Workflow.

Recommended settings

	recommended	notes
base model	Anima base v1.0	the adapter was trained on it; a turbo LoRA also works
sampler	any; er_sde or Euler, 30 steps, simple	as for Anima without the adapter
cfg	4.5	text guidance; 1 with a turbo LoRA
ref_guidance	4.5	1 = cheaper and weaker; higher = more character detail and style, more saturation
correction	apg	or rescale (phi 0.5)
ref_end	1.0	~0.6 is faster with nearly the same image
prompt	quality + 1girl / 1boy + solo + scene tags	no character name, artist tag or outfit tags; a few appearance tags are fine
other LoRAs	in front of the adapter node

How it works

• Asymmetric attention. Target tokens attend to themselves and the reference; reference tokens attend only to themselves and skip the text cross-attention. The reference therefore depends on nothing that changes during sampling, so its keys and values are computed once per image and reused by every step and both CFG branches. It may have any size.
• The reference sits at shifted RoPE coordinates, past the target grid, so it is read as appearance and not copied pixel-aligned (no pose copying).
• A rank-64 LoRA, active only with the reference, learns to read it. Training pairs are the same character in different scenes, generated by Anima, with the character's name, artist and appearance tags removed from the target prompt.
• Separate text and reference guidance, with adaptive projected guidance (APG) against over-saturation.

832x1216, RTX PRO 6000, CFG	pilot (two-frame)	v2 (asymmetric)	no reference
one sampling step	0.58 s	0.28 s (+ 0.1 s once to encode the reference)	0.24 s

The pilot instead appended the reference as a second latent frame and recomputed it at every step. Its method, data and failure modes: technical report.

Pilot (v1)

The pilot (anima_ref_adapter_pilot_r32.safetensors, rank 32) stays available and works with the advanced workflow.

Pilot gallery and held-out characters

Prompt masterpiece, best quality, score_7, newest, highres, absurdres, safe, <gender>, solo, <scene>; Euler 28 steps, cfg 4.5, reference guidance 4.5 + APG, context detailer LoRA 0.3, base slider LoRA 0.6, Anima Safe PAG. Not cherry-picked.

_{full body, walking, hands in pockets, city, street, night, rain  ·  upper body, sitting, holding cup, drinking, cafe, indoors, window, sunlight  ·  cowboy shot, looking at viewer, smile, beach, ocean, sunset  ·  portrait, from side, looking afar, cherry blossoms, petals  ·  upper body, reading, holding book, library, bookshelf  ·  full body, standing, snow, forest, winter full body, walking, smile, city, street, night, neon lights  ·  upper body, sitting, eating, cake, cafe, indoors  ·  cowboy shot, looking back, smile, beach, ocean, sunset  ·  portrait, closed eyes, smile, cherry blossoms, petals, wind  ·  full body, sitting, on chair, crossed legs, classroom, sunlight  ·  upper body, holding umbrella, rain, street, looking at viewer}

Held-out characters, excluded from training. Each row: the reference, the adapter's result for a new scene, the same scene-only prompt without the adapter, and the base model with the character named and its appearance tags.

character	scene tags
Tohru	full body, looking at viewer, heart hands, crying, simple background, white background
Eternity Larva	full body, looking at viewer, against wall, closed eyes, gradient background, sunlight
Meiko	portrait, from side, writing, crying, beach, ocean, light rays
Crocodile	upper body, from below, drinking, sad, bookshelf, indoors

pilot training
data	3,600 images generated by Anima: 600 groups of one character in one artist style x 6 scenes; 300 characters, 427 artists; 12 characters held out
pairs	17,280 (reference, target) pairs; target prompt keeps scene tags only (70%), + some appearance tags (20%), or nothing (10%)
optimization	flow matching, batch 4, AdamW 1e-4 cosine, bf16, one RTX PRO 6000 Max-Q

Limitations

• Identity transfers much better than style; strongly painterly styles barely transfer.
• Small patterns, scars, earrings and unusual skin tones are often lost; large garments can drift in color.
• The reference character occasionally reappears as an object in the scene.
• Trained only on images generated by Anima; it inherits the base model's biases.

Artist names were used only as generation tags. Please respect artists: do not pass off imitations as their work, and follow the use restrictions in §4 of LICENSE.md.

Roadmap

	status
Single reference: character and style from one image	✅ pilot, v2
Asymmetric attention with a cached reference	✅ v2
Two characters from two references side by side	✅ v2
Separate character and style references	⏸️ paused (experiments only)
Higher-resolution training, better painterly styles and fine details	⏸️ paused

License and attribution

Non-commercial. These weights are a modified derivative of the CircleStone Model (Anima base v1.0), distributed under the CircleStone Non-Commercial License (LICENSE.md, NOTICE.txt). This is an independent project, not an official product of CircleStone Labs and not endorsed by it.

The CircleStone Model is licensed by CircleStone Labs LLC under the CircleStone Non-Commercial License. Copyright CircleStone Labs LLC. IN NO EVENT SHALL CIRCLESTONE LABS LLC BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH USE OF THIS MODEL.

Files

file
anima_ref_adapter_v2_r64.safetensors	v2: rank 64, asymmetric attention (metadata arch: asym), bf16, 840 tensors
anima_ref_adapter_pilot_r32.safetensors	pilot: rank 32, two-frame, bf16, 840 tensors
workflows/	anima_ref_adapter.json (one reference), anima_ref_adapter_two_characters.json, anima_ref_adapter_advanced.json (separate nodes, pilot)
showcase/, figures/	gallery images and figures
report.md	technical report (pilot)
README.ko.md, README.ja.md, guide.*.md	Korean and Japanese model card; user guides in English, Korean and Japanese
LICENSE.md, NOTICE.txt	CircleStone Non-Commercial License and attribution notice