WVY-3D-1M
A 1M-parameter model that rigs and animates 3D creatures. Built and trained from scratch: plain numpy with a hand-written autograd engine, no PyTorch or other ML libraries. Inference runs in the browser in plain JavaScript.
▶ Try it: WVY-3D-1M Scene Studio · 🎬 demo video
WVY-3D-1M is two small networks that work together:
| Part | What it does | Parameters |
|---|---|---|
| RigNet | Looks at 512 points on a creature's surface, recognises its body plan, places the skeleton's joints and assigns skin weights | 234,194 |
| MotionGPT | A small GPT whose "words" are poses: given a prompt like quadruped:run it writes a sentence of poses |
754,969 |
| Pose tokenizer | 6 codebooks × 256 pose words (k-means), one per body plan | 77,400 values |
| Total | ≈1.07M |
Body plans and actions
| Body plan | Joints | Actions |
|---|---|---|
| biped (characters) | 17 | idle, walk, run, wave, jump, dance |
| quadruped | 19 | idle, walk, run, jump, sit, eat |
| bird | 17 | idle, hop, peck, fly, glide |
| fish | 8 | idle, swim, dart |
| snake | 10 | idle, slither |
| insect (6 legs) | 21 | idle, crawl, run |
Results
Held-out creatures (50 per body plan, never seen in training), joint error as % of body size:
| biped | quadruped | bird | fish | snake | insect | |
|---|---|---|---|---|---|---|
| Body plan recognised | 100% | 100% | 100% | 100% | 100% | 100% |
| Joint error | 1.3% | 1.3% | 1.5% | 0.9% | 0.4% | 0.8% |
| Skin points on the same bone as ground truth | 87% | 82% | 83% | 91% | 96% | 93% |
In the four demo scenes: 37/37 creatures recognised, mean joint error 0.7% of body size, 90% skin agreement.
MotionGPT: next-pose-token loss on unseen clips fell from 8.36 to 1.07 (train 1.20, so no overfitting), meaning about 3 plausible next poses out of 256 at each step.
Architecture
RigNet (PointNet-style)
- Per-point MLP 3→64→128→256 (ReLU), max-pool to a 256-d global shape code
- Fusion:
relu(Linear128→256(f2) + Linear256→256(global))→ Linear 256→128 - Heads: body-plan classifier (256→6), skin logits (128→92), joint attention (128→92), joint offsets (128→276)
- Joints by voting: every surface point proposes an offset to each joint;
joint_j = Σ_i softmax_i(att_ij) · (p_i + offset_ij) - 92 output joints = all six body plans' skeletons; the classifier picks which slice to use
MotionGPT (decoder-only transformer)
- d_model 96, 4 layers, 4 heads, MLP 384, pre-LayerNorm, GELU (tanh), context 64 tokens (4.3 s at 15 poses/s)
- Vocabulary 1,561 = 6 × 256 pose tokens + 25 prompt tokens (
family:action) - Sequence:
[prompt, pose₁, pose₂, …]; longer clips are written by sliding the window (prompt + last 31 poses) - Decoding: top-k 8, temperature 0.8, restricted to the body plan's 256 tokens; upsampled 15 → 30 fps and smoothed
Pose frame (what one token stands for): Euler angles per joint relative to the rest pose (R = Ry·Rx·Rz), plus 4 root channels: sideways sway, height, forward speed and turn rate, in units of the creature's size.
Weight layout and every tensor shape are in config.json. Linear weights are stored (in, out), so y = x @ W + b.
Training
| RigNet | MotionGPT | |
|---|---|---|
| Data | 4,200 procedurally sculpted creatures (700 per body plan), 512 surface points each, generator skeleton + skin weights as targets | Procedural motion clips for 25 actions, a fresh batch generated every step |
| Steps | 4,000 (batch 18: 3 per body plan) | 3,000 (batch 24 × 64 tokens) |
| Optimiser | Adam, lr 2e-3, cosine decay | Adam, lr 1e-3, 200-step warm-up, cosine decay, weight decay 1e-4 |
| Loss | 100·joint MSE + 10·vote MSE + soft cross-entropy (skin) + cross-entropy (body plan) | Next-token cross-entropy |
| Time | ~17 min on 1 CPU core | ~26 min on 1 CPU core |
Training logs: logs/.
How the full pipeline works
- Sculpt. A creature is a genome (numbers like neck length, ears, horns, tail) turned into soft signed-distance "clay" and meshed with surface nets. 80 species presets + random variation, or any invented genome. 35 prop types for scenery. This part is procedural, not learned.
- Rig. RigNet reads the mesh surface and returns the body plan, joints and per-vertex skin weights.
- Animate. MotionGPT writes pose tokens for each action; the runtime decodes them, follows paths (wander, circle, toward, follow, flee), keeps feet on the ground and drives the skeleton.
- Direct (optional). On claude.ai, Claude turns a plain-language scene description into a scene script (JSON) that the engine builds.
Files
| File | Contents |
|---|---|
rignet.safetensors, motiongpt.safetensors, pose_tokenizer.safetensors |
Weights (float32) |
browser_weights_fp16.json |
Same weights in float16/base64 for the browser runtime |
config.json |
Architecture and tensor layout |
code/python/ |
Autograd engine (tinygrad.py), RigNet and MotionGPT model + training code, exporter |
code/engine/ |
JavaScript runtime: sculptor, species, props, inference + scene runtime |
code/tools/ |
Training-data generator (Node) and scene exporter |
assets/ |
Demo video and stills |
Run inference in Node
['sculpt','species','objects','brain'].forEach(f => require('./code/engine/' + f + '.js'));
const B = globalThis.RigBrain, S = globalThis.RigSculpt;
B.loadWeights(JSON.parse(require('fs').readFileSync('browser_weights_fp16.json')));
const fox = S.makeCreature({ family: 'quadruped', species: 'fox', seed: 1 });
const rig = B.learnedRig(fox.verts, fox.faces); // body plan, joints, skin weights
const poses = B.writeMotion('quadruped', 'run', 60, S.RNG(1)); // 60 pose tokens (4 s)
Limitations
- Training data is procedural, not motion capture: motion is clean but stylised.
- Six body plans only. Other shapes are approximated (a dragon is a horned quadruped; spiders get six legs). No wings on quadrupeds, no tentacles.
- 2–6 actions per body plan; creatures follow paths but don't touch or interact.
- Shapes come from the procedural genome system; only rigging and motion are learned.
- RigNet expects meshes in a T-pose or A-pose, facing +z, y up.
Citation
@misc{wvy3d1m2026,
title = {WVY-3D-1M: a 1M-parameter auto-rigger and motion language model trained from scratch},
author = {StarpowerTechnology},
year = {2026},
url = {https://huggingface.co/StarpowerTechnology/WVY-3D-1M}
}
- Downloads last month
- -

