README.md

---
license: other
license_name: circlestone-labs-non-commercial-license
license_link: https://huggingface.co/circlestone-labs/Anima/blob/main/LICENSE.md
library_name: pytorch
pipeline_tag: feature-extraction
base_model: circlestone-labs/Anima
tags:
  - anime
  - illustration
  - style-embedding
  - image-retrieval
  - reference-image
  - multi-reference
  - dinov3
  - anima
  - controllable-generation
---

# Anima Style Embedding v2

A multi-reference shared-axis style encoder for anime and illustration images.

Anima Style Embedding v2 converts one to eight reference images into a normalized 1,024-dimensional embedding and a controllable style memory for Anima. It preserves spatial VAE evidence, adds DINOv3 visual context and Anima-aware descriptors, and represents Line, Color, Texture, and Layout as overlapping memberships on one 256-axis style field.

| Model size | References | Embedding | Shared axes | Reference signals |
|---|---:|---:|---:|---|
| 51,991,809 parameters | 1-8 images | 1,024-D, L2-normalized | 256 | VAE + DINOv3-S+ + Anima |

![Anima Style Embedding v2 overview: 91.2% Top-1 and 97.4% Top-5 for a disjoint four-image Gallery and two-image Probe evaluation.](assets/hero_shared_axis_imagegen_v1.png)

## Highlights

- **Style-disjoint evaluation:** retrieval is measured on 1,000 unseen style identities with separate Gallery and Probe image sets.
- **Native multi-reference reasoning:** the encoder compares one to eight references before producing the style representation.
- **One shared style field:** Line, Color, Texture, and Layout are overlapping memberships, not independent encoders.
- **Controllable routing:** Auto, Assisted, and Manual modes support per-image factor weights plus a separate Face weight.
- **Anima-ready memory:** the same shared axes produce the retrieval embedding and the style memory used by the native transfer path.
- **Position-free Layout evidence:** phase-free spatial-power statistics capture composition-like tendencies without copying absolute object coordinates.

## Performance

### Disjoint set-to-set retrieval

Gallery and Probe embeddings are built from different images of the same style. The primary protocol uses cosine similarity, three fixed seeds, and a 1,000-style validation gallery. The sealed test split was not used for checkpoint selection or for the reported v2 results.

The headline condition uses a four-image Gallery and a two-image Probe:

| Top-1 | Top-5 |
|---:|---:|
| **91.23%** | **97.40%** |

![Top-1 and Top-5 retrieval matrices for Gallery sets of 2, 4, and 8 images and Probe sets of 1, 2, 4, and 8 images.](assets/set_retrieval_matrix_imagegen_v1.png)

Top-1:

| Gallery \ Probe | 1 | 2 | 4 | 8 |
|---|---:|---:|---:|---:|
| 2 references | 67.07% | 81.93% | 89.90% | 93.97% |
| 4 references | 77.17% | 91.23% | 96.60% | 98.47% |
| 8 references | 83.27% | 95.07% | 98.83% | 99.83% |

Top-5:

| Gallery \ Probe | 1 | 2 | 4 | 8 |
|---|---:|---:|---:|---:|
| 2 references | 84.73% | 93.27% | 96.97% | 98.23% |
| 4 references | 91.23% | 97.40% | 99.27% | 99.70% |
| 8 references | 94.03% | 98.60% | 99.80% | 100.00% |

Machine-readable Overall, Synthetic, and Human results are included in [`metrics/evaluation-set-matrix.json`](metrics/evaluation-set-matrix.json).

### Reference count and domain behavior

Human references have greater within-style variation when only one Probe image is available. With a two-image Gallery, the Synthetic/Human Top-1 gap narrows from 6.2 points at Probe-1 to 0.8 point at Probe-8.

![Synthetic and Human Top-1 retrieval improve as the disjoint Probe reference set grows from one to eight images.](assets/domain_reference_count_imagegen_v1.png)

| Probe references | Synthetic Top-1 | Human Top-1 |
|---:|---:|---:|
| 1 | 69.20% | 63.00% |
| 2 | 83.27% | 79.73% |
| 4 | 90.73% | 88.47% |
| 8 | 93.80% | 93.00% |

## Factor control and routing

Factor behavior is evaluated on 4,096 interventions from held-out transform families.

![Held-out intervention accuracy, selective factor response, shared-axis overlap, and multi-reference routing results.](assets/factor_control_imagegen_v1.png)

| Factor | Classification accuracy | Target / non-target response |
|---|---:|---:|
| Line | 88.48% | 1.761x |
| Color | 64.16% | 1.406x |
| Texture | 94.04% | 2.393x |
| Layout | 91.70% | 1.970x |

| Control metric | Result |
|---|---:|
| Balanced factor accuracy | 84.60% |
| Strong > weak ordering | 98.83% of 2,048 pairs |
| Median factor-outlier suppression | 3.643x |
| Auto routing error reduction vs. uniform | 19.80% |
| Axes materially shared by all four factors | 82 |
| Effective rank | 248.35 / 256 |

Attribute reconstruction reduces error by 31.2% relative to a train-mean predictor. Removing a factor membership raises its corresponding attribute error, confirming that the memberships are used by the prediction path rather than serving only as labels.

## Architecture

![Architecture from one to eight full and face references through VAE, DINOv3, and Anima signals into a shared 256-axis field, routing, a 1,024-D embedding, and Anima style memory.](assets/architecture_imagegen_v1.png)

The production encoder has one spatial field and one aligned dictionary:

1. A convolutional stem preserves full-image and face VAE spatial fields.
2. Each spatial token is projected into the same 256-axis dictionary.
3. Per-axis moments and translation-invariant power statistics retain persistent and composition-like style evidence.
4. DINOv3 and Anima projections add visual and generator-aware context to the aligned axes.
5. A relation Transformer compares references before estimating per-axis reliability.
6. Overlapping memberships expose Line, Color, Texture, and Layout views without splitting the representation into branches.
7. The mixed axes produce both a normalized retrieval embedding and style memory for Anima conditioning.

### Routing modes

| Mode | Behavior |
|---|---|
| Auto | Uses learned per-reference, per-axis reliability. |
| Assisted | Multiplies learned reliability by user-provided factor and Face weights. |
| Manual | Uses user weights directly to choose factor suppliers. |

Weights normalize across references, so adding references does not automatically increase style strength. A zero Manual factor weight produces zero output contribution and zero gradient on that reference-factor path. Face controls face-token routing and is not treated as a fifth factor branch.

## Quick start

The v2 checkpoint expects precomputed features. Raw-RGB VAE, DINOv3-S+, and Anima descriptor extraction is not bundled in this release candidate.

```python
import json

import torch
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file

from modeling_anima_style_embedding import SharedAxisStyleEncoder

repo_id = "Baragi-AI/Anima-Style-Embedding"
config_path = hf_hub_download(repo_id, "config.json")
weights_path = hf_hub_download(repo_id, "model.safetensors")

with open(config_path, encoding="utf-8") as file:
    config = json.load(file)

encoder = SharedAxisStyleEncoder(**config["architecture"])
encoder.load_state_dict(load_file(weights_path), strict=True)
encoder.eval()

with torch.inference_mode():
    output = encoder(
        full_latents=full_latents,
        face_latents=face_latents,
        dino=dino_features,
        anima=anima_features,
        reference_valid=reference_valid,
        face_valid=face_valid,
        user_weights=user_weights,
        mode="assisted",
    )

style_embedding = output["embedding"]
style_memory = output["style_memory"]
factor_coordinates = output["factor_coordinates"]

assert style_embedding.shape == (full_latents.shape[0], 1024)
```

### Input contract

| Input | Shape | Notes |
|---|---|---|
| Full VAE latents | `[B, R, 16, H, W]` | Spatial full-image evidence |
| Face VAE latents | `[B, R, 16, Hf, Wf]` | Optional, masked by `face_valid` |
| DINOv3 full+face summary | `[B, R, 6912]` | Cached DINOv3-S+ context |
| Anima descriptors | `[B, R, 3, 4096]` | Three internal Anima blocks |
| Reference validity | `[B, R]` | At least one valid reference per sample |
| User weights | `[B, R, 5]` | Line, Color, Texture, Layout, Face |

## Training data and split policy

The base corpus contains 500,000 images:

- 250,000 Anima-generated images from 5,000 Synthetic style identities;
- 250,000 human-created illustrations from 5,000 Human style identities.

Synthetic and Human identities remain separate even when artist names overlap. The split is identity-disjoint:

| Split | Images | Style identities | Use |
|---|---:|---:|---|
| Train | 400,000 | 8,000 | Optimization and training-only statistics |
| Validation | 50,000 | 1,000 | Checkpoint selection and reported v2 metrics |
| Test | 50,000 | 1,000 | Sealed and unused |

Factor training additionally uses 108,096 controlled intervention images. Training and validation intervention families are disjoint.

## Development findings

The final structure follows several measured corrections:

- Global pooling and repeated low-dimensional compression removed local line and texture evidence. The accepted path preserves the VAE spatial field.
- Independent factor branches duplicated capacity and forced artificial separation. Overlapping memberships on one dictionary retained shared evidence.
- An earlier dense-coordinate implementation broke coordinate identity between training and production gating. Aligned per-axis pooling made coordinate `k` refer to the same axis everywhere.
- FiLM improved retrieval, but updating it during factor tuning reduced retrieval. Retrieval-trained FiLM is kept and frozen during later stages.
- Layout needed phase-free radial and directional power statistics. These improved Layout prediction without encoding absolute object locations.
- Fixed validation panels, atomic checkpoints, domain-balanced episodes, and median per-record routing metrics prevented evaluation artifacts from selecting a model.

See [`RESEARCH_REPORT.md`](RESEARCH_REPORT.md) for the full architecture, training stages, failed approaches, and acceptance evidence.

## Applications

- reference-driven style transfer for Anima;
- open-set style retrieval and clustering;
- multi-reference style prototype construction;
- per-reference Line, Color, Texture, Layout, and Face control;
- reference-set curation and similarity scoring;
- research on shared and controllable style representations.

## Limitations

- Retrieval and intervention metrics measure representation behavior, not generated-image fidelity.
- Raw-RGB feature extraction is not bundled in this release candidate.
- The sealed test split remains unused, so v2 does not claim sealed-test performance.
- Color intervention selectivity is weaker than Line, Texture, and Layout.
- Layout attribute causality is positive but weaker than the other attribute memberships.
- Human single-reference retrieval remains harder than Synthetic retrieval.
- Line, Color, Texture, and Layout are useful controls, not a complete or perfectly disentangled theory of style.
- The model is optimized for anime and illustration images and has not been evaluated as a general photographic style encoder.
- The embedding must not be used as evidence of authorship or identity.

## Files

| File | Contents |
|---|---|
| `model.safetensors` | Encoder weights |
| `config.json` | Architecture and feature contract |
| `modeling_anima_style_embedding.py` | PyTorch model definition |
| `metrics/` | Retrieval, factor, attribute, and routing evaluations |
| `assets/` | Model-card figures |
| `RESEARCH_REPORT.md` | Detailed research and implementation report |
| `SHA256SUMS.json` | Release checksums |
| `LICENSE.md` | CircleStone Labs Non-Commercial License v1.2 |
| `CODE_LICENSE/` | License for original release code |
| `NOTICE` | Required attribution notice |
| `THIRD_PARTY_NOTICES.md` | Third-party licenses and attribution |

## License

The model weights are a derivative of [circlestone-labs/Anima](https://huggingface.co/circlestone-labs/Anima) and are distributed under the [CircleStone Labs Non-Commercial License v1.2](https://huggingface.co/circlestone-labs/Anima/blob/main/LICENSE.md).

The license permits the model and its derivatives only for the non-commercial purposes defined by that license. Commercial or production use requires authorization from CircleStone Labs. The upstream license separately permits use of model outputs, subject to its terms and applicable law.

Original Baragi AI release code is provided under the Apache License 2.0. Third-party components remain under their respective licenses and are listed in `NOTICE` and `THIRD_PARTY_NOTICES.md`.

## Citation

```bibtex
@misc{baragi2026animastyleembeddingv2,
  title        = {Anima Style Embedding v2},
  author       = {{Baragi AI}},
  year         = {2026},
  howpublished = {Hugging Face model repository},
  url          = {https://huggingface.co/Baragi-AI/Anima-Style-Embedding}
}
```