WVY-3D-1M

A 1M-parameter model that rigs and animates 3D creatures. Built and trained from scratch: plain numpy with a hand-written autograd engine, no PyTorch or other ML libraries. Inference runs in the browser in plain JavaScript.

▶ Try it: WVY-3D-1M Scene Studio · 🎬 demo video

Scenes built, rigged and animated by WVY-3D-1M

WVY-3D-1M is two small networks that work together:

Part What it does Parameters
RigNet Looks at 512 points on a creature's surface, recognises its body plan, places the skeleton's joints and assigns skin weights 234,194
MotionGPT A small GPT whose "words" are poses: given a prompt like quadruped:run it writes a sentence of poses 754,969
Pose tokenizer 6 codebooks × 256 pose words (k-means), one per body plan 77,400 values
Total ≈1.07M

Body plans and actions

Body plan Joints Actions
biped (characters) 17 idle, walk, run, wave, jump, dance
quadruped 19 idle, walk, run, jump, sit, eat
bird 17 idle, hop, peck, fly, glide
fish 8 idle, swim, dart
snake 10 idle, slither
insect (6 legs) 21 idle, crawl, run

Results

Held-out creatures (50 per body plan, never seen in training), joint error as % of body size:

biped quadruped bird fish snake insect
Body plan recognised 100% 100% 100% 100% 100% 100%
Joint error 1.3% 1.3% 1.5% 0.9% 0.4% 0.8%
Skin points on the same bone as ground truth 87% 82% 83% 91% 96% 93%

In the four demo scenes: 37/37 creatures recognised, mean joint error 0.7% of body size, 90% skin agreement.

MotionGPT: next-pose-token loss on unseen clips fell from 8.36 to 1.07 (train 1.20, so no overfitting), meaning about 3 plausible next poses out of 256 at each step.

Generator skeleton (white) vs RigNet's prediction (orange)

Architecture

RigNet (PointNet-style)

  • Per-point MLP 3→64→128→256 (ReLU), max-pool to a 256-d global shape code
  • Fusion: relu(Linear128→256(f2) + Linear256→256(global)) → Linear 256→128
  • Heads: body-plan classifier (256→6), skin logits (128→92), joint attention (128→92), joint offsets (128→276)
  • Joints by voting: every surface point proposes an offset to each joint; joint_j = Σ_i softmax_i(att_ij) · (p_i + offset_ij)
  • 92 output joints = all six body plans' skeletons; the classifier picks which slice to use

MotionGPT (decoder-only transformer)

  • d_model 96, 4 layers, 4 heads, MLP 384, pre-LayerNorm, GELU (tanh), context 64 tokens (4.3 s at 15 poses/s)
  • Vocabulary 1,561 = 6 × 256 pose tokens + 25 prompt tokens (family:action)
  • Sequence: [prompt, pose₁, pose₂, …]; longer clips are written by sliding the window (prompt + last 31 poses)
  • Decoding: top-k 8, temperature 0.8, restricted to the body plan's 256 tokens; upsampled 15 → 30 fps and smoothed

Pose frame (what one token stands for): Euler angles per joint relative to the rest pose (R = Ry·Rx·Rz), plus 4 root channels: sideways sway, height, forward speed and turn rate, in units of the creature's size.

Weight layout and every tensor shape are in config.json. Linear weights are stored (in, out), so y = x @ W + b.

Training

RigNet MotionGPT
Data 4,200 procedurally sculpted creatures (700 per body plan), 512 surface points each, generator skeleton + skin weights as targets Procedural motion clips for 25 actions, a fresh batch generated every step
Steps 4,000 (batch 18: 3 per body plan) 3,000 (batch 24 × 64 tokens)
Optimiser Adam, lr 2e-3, cosine decay Adam, lr 1e-3, 200-step warm-up, cosine decay, weight decay 1e-4
Loss 100·joint MSE + 10·vote MSE + soft cross-entropy (skin) + cross-entropy (body plan) Next-token cross-entropy
Time ~17 min on 1 CPU core ~26 min on 1 CPU core

Training logs: logs/.

How the full pipeline works

  1. Sculpt. A creature is a genome (numbers like neck length, ears, horns, tail) turned into soft signed-distance "clay" and meshed with surface nets. 80 species presets + random variation, or any invented genome. 35 prop types for scenery. This part is procedural, not learned.
  2. Rig. RigNet reads the mesh surface and returns the body plan, joints and per-vertex skin weights.
  3. Animate. MotionGPT writes pose tokens for each action; the runtime decodes them, follows paths (wander, circle, toward, follow, flee), keeps feet on the ground and drives the skeleton.
  4. Direct (optional). On claude.ai, Claude turns a plain-language scene description into a scene script (JSON) that the engine builds.

Files

File Contents
rignet.safetensors, motiongpt.safetensors, pose_tokenizer.safetensors Weights (float32)
browser_weights_fp16.json Same weights in float16/base64 for the browser runtime
config.json Architecture and tensor layout
code/python/ Autograd engine (tinygrad.py), RigNet and MotionGPT model + training code, exporter
code/engine/ JavaScript runtime: sculptor, species, props, inference + scene runtime
code/tools/ Training-data generator (Node) and scene exporter
assets/ Demo video and stills

Run inference in Node

['sculpt','species','objects','brain'].forEach(f => require('./code/engine/' + f + '.js'));
const B = globalThis.RigBrain, S = globalThis.RigSculpt;
B.loadWeights(JSON.parse(require('fs').readFileSync('browser_weights_fp16.json')));
const fox = S.makeCreature({ family: 'quadruped', species: 'fox', seed: 1 });
const rig = B.learnedRig(fox.verts, fox.faces);          // body plan, joints, skin weights
const poses = B.writeMotion('quadruped', 'run', 60, S.RNG(1)); // 60 pose tokens (4 s)

Limitations

  • Training data is procedural, not motion capture: motion is clean but stylised.
  • Six body plans only. Other shapes are approximated (a dragon is a horned quadruped; spiders get six legs). No wings on quadrupeds, no tentacles.
  • 2–6 actions per body plan; creatures follow paths but don't touch or interact.
  • Shapes come from the procedural genome system; only rigging and motion are learned.
  • RigNet expects meshes in a T-pose or A-pose, facing +z, y up.

Citation

@misc{wvy3d1m2026,
  title  = {WVY-3D-1M: a 1M-parameter auto-rigger and motion language model trained from scratch},
  author = {StarpowerTechnology},
  year   = {2026},
  url    = {https://huggingface.co/StarpowerTechnology/WVY-3D-1M}
}
Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Space using StarpowerTechnology/WVY-3D-1M 1