| --- |
| library_name: onnxruntime |
| tags: |
| - animation |
| - character-animation |
| - inverse-kinematics |
| - pose-estimation |
| - onnx |
| license: other |
| --- |
| |
| # Animatica Autoposer (SOMA-30) |
|
|
| A neural autoposer for full-body character rigs. Give it a handful of handles — where a hand |
| should be, where the feet are planted, which way the head looks — and it returns a complete, |
| natural pose for a 30-joint skeleton. It is what makes posing in the |
| [Animatica](https://animatica.ai) Blender add-on feel like moving a character rather than |
| rotating thirty bones. |
|
|
| It is a **poser, not an animator**: one pose per call, a few milliseconds each, no motion model |
| and no temporal state. Ask it once per frame and you have an editable animation; ask it once and |
| you have a pose. |
|
|
| ## What you give it |
|
|
| Any number of effectors, in any combination. Each carries a **tolerance** — metres of slack — so |
| a handle can be an exact pin or a loose suggestion the model may overrule to keep the body |
| plausible. |
|
|
| | Effector | Type id | Value | Meaning | |
| |---|---|---|---| |
| | Position | `0` | xyz | put this joint here | |
| | Rotation | `1` | 6D rotation | orient this joint like this | |
| | Look-at | `2` | xyz + direction | aim this joint at this point | |
|
|
| Plus a **body descriptor**: 29 bone lengths, z-scored. The same effectors solve correctly on a |
| child, a heavyset adult, or a giant, because proportion is an input rather than an assumption. |
|
|
| Three position effectors is the sparsest input it was trained to answer well; one or two will |
| return something, but the body below them is mostly prior. |
|
|
| ## What you get back |
|
|
| - **draft joint positions** — `[B, 90]`, the 30 joints as xyz, root first |
| - **6D rotations** — `[B, 180]`, per joint, for FK on your own skeleton |
|
|
| Run the rotations through SOMA-30 forward kinematics with your body's neutral joints to get world |
| positions. The bundled IK pass (below) then pins the effectors exactly and keeps the feet out of |
| the floor. |
|
|
| ## Use it in Blender |
|
|
| The Animatica add-on downloads this repository on first use and needs no token or manual setup — |
| handles appear on your rig and you drag them. That is the shortest path to using this model, and |
| the one it is tuned for. |
|
|
| ## Use it directly (ONNX) |
|
|
| `onnx/` is the runtime bundle: two graphs and the metadata that describes them. |
|
|
| ```python |
| import json, numpy as np, onnxruntime as ort |
| from huggingface_hub import snapshot_download |
| |
| d = snapshot_download("Animatica-ai/autoposer", allow_patterns=["onnx/*"]) |
| meta = json.load(open(f"{d}/onnx/meta.json")) |
| poser = ort.InferenceSession(f"{d}/onnx/poser.onnx") |
| |
| E = 3 # three handles: hips and both hands |
| val = np.zeros((1, E, 7), np.float32) # 6 value channels + 1 tolerance |
| val[0, 0, :3], val[0, 0, 6] = (0.0, 0.95, 0.0), 0.005 |
| val[0, 1, :3], val[0, 1, 6] = (0.35, 1.20, 0.20), 0.005 |
| val[0, 2, :3], val[0, 2, 6] = (-0.35, 1.20, 0.20), 0.005 |
| |
| draft, rot6d = poser.run(None, { |
| "val": val, |
| "weights": np.ones((1, E), np.float32), # encoder blending weights |
| "joint_id": np.array([[0, 13, 19]], np.int64), # Hips, LeftHand, RightHand |
| "eff_type": np.zeros((1, E), np.int64), # all positions |
| "cond": np.zeros((1, 29), np.float32), # z-scored bone lengths |
| }) |
| ``` |
|
|
| Two details the graph will not tell you, and which decide whether the output is a pose or a heap: |
|
|
| - **Reference frame.** Position and look-at values are expressed relative to the centroid of the |
| position targets. This checkpoint is floor-referenced (`floor_ref: true`), so **y stays |
| absolute** — the ground is a real height, not something relative — and the centroid is zeroed |
| on y before subtracting. |
| - **Conditioning.** `cond` is the 29 bone lengths, z-scored with `blen_mean` / `blen_std` from |
| `meta.json` and clamped to ±4.5. Zeros mean "the average body the statistics describe", which |
| is why the snippet above works at all. |
|
|
| ### The IK pass |
|
|
| `onnx/ik.onnx` is a damped least-squares refinement that runs after the network: it pins |
| effectors to the tolerance you asked for, holds a floor plane, and rolls the toes rather than |
| sinking the feet. Twelve iterations over a damping ladder, accepting per step. The network gets |
| the pose right; this makes the handles exact. |
|
|
| `meta.json` carries a **selftest** — recorded inputs and expected outputs — so a miscompiled |
| graph is detectable. It matters more than it sounds: a bad kernel fusion does not raise, it |
| quietly returns a wrong pose. Run the IK graph on `selftest.inputs` and compare against |
| `selftest.expect`; agreement should be well under half a centimetre. |
|
|
| ## Use it directly (PyTorch) |
|
|
| `model.safetensors` + `config.json` are the training weights, for research use and for anyone who |
| wants to fine-tune. `config.json` records the skeleton (`njoints`, `ux`/`ux_names`), the |
| conditioning width (`cond_dim: 29`), floor referencing, and the training step. |
|
|
| ## The skeleton |
|
|
| 30 joints, root first: |
|
|
| ``` |
| Hips Spine1 Spine2 Chest Neck1 Neck2 Head Jaw LeftEye RightEye |
| LeftShoulder LeftArm LeftForeArm LeftHand LeftHandThumbEnd LeftHandMiddleEnd |
| RightShoulder RightArm RightForeArm RightHand RightHandThumbEnd RightHandMiddleEnd |
| LeftLeg LeftShin LeftFoot LeftToeBase |
| RightLeg RightShin RightFoot RightToeBase |
| ``` |
|
|
| `meta.json` has the parent list, the neutral (rest) joint positions, the four foot joints used by |
| the floor term, and the effector joint ids. |
|
|
| ## How it works |
|
|
| Each effector is embedded from its value, tolerance, joint id and type, and the set is blended by |
| weight into a single pose code — so the model takes any number of handles in any order, rather |
| than a fixed rig of them. Two-stage decoding follows: draft joint positions first, then per-joint |
| 6D rotations, which is what keeps limbs attached to a body of the right proportions instead of |
| merely near the handles. The body descriptor is injected twice, at the encoder alongside each |
| effector and again at the decoder embedding (zero-initialised, so it starts as a no-op and is |
| learned into). |
|
|
| ## Files |
|
|
| | File | Size | What | |
| |---|---|---| |
| | `onnx/poser.onnx` | 142 MB | the network | |
| | `onnx/ik.onnx` | 1.9 MB | the IK refinement pass | |
| | `onnx/meta.json` | 19 KB | skeleton, normalisation stats, checksums, selftest | |
| | `model.safetensors` | — | PyTorch weights | |
| | `config.json` | — | training configuration | |
|
|
| ## Limitations |
|
|
| - **One pose at a time.** There is no temporal model: consecutive frames are consistent because |
| consecutive effectors are, not because the model remembers anything. |
| - **SOMA-30 only.** Other skeletons need retargeting onto these 30 joints first. |
| - **Hands and face are along for the ride.** Fingers are a thumb and middle tip per hand; jaw and |
| eyes are in the skeleton but are not meaningfully posed. |
| - **Fewer than three position effectors** leaves most of the body to the prior. It will answer; |
| it just has little to answer from. |
| - **Floor-referenced.** Feed it absolute heights. A pose built in a different vertical frame will |
| come back crouching or floating. |
|
|
| --- |
|
|
| Made by [Animatica](https://animatica.ai). Found something odd? Tell us — the model and the |
| add-on move together, and a pose that comes out wrong is usually interesting. |
|
|
