fernandotonon commited on
Commit
643b895
·
verified ·
1 Parent(s): a4c332b

Add model card

Browse files
Files changed (1) hide show
  1. README.md +55 -0
README.md ADDED
@@ -0,0 +1,55 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-4.0
3
+ tags:
4
+ - animation
5
+ - motion-in-betweening
6
+ - onnx
7
+ - 3d
8
+ - qtmesheditor
9
+ library_name: onnx
10
+ pipeline_tag: other
11
+ ---
12
+
13
+ # QtMeshEditor — RMIB Animation In-Betweening
14
+
15
+ A small transformer that fills the gap between two sparse keyframes with smooth,
16
+ plausible intermediate motion — **Robust Motion In-Betweening (RMIB)** style,
17
+ exported to **ONNX** for in-app inference via ONNX Runtime.
18
+
19
+ Built for **[QtMeshEditor](https://github.com/fernandotonon/QtMeshEditor)** (issue #409) — a free,
20
+ open-source 3D mesh & animation editor — and its companion **QtMesh Cloud**
21
+ asset service. The app downloads this model on first use and runs it locally
22
+ (offline) to power the dope-sheet **"AI in-between → Fill gap"** action, the
23
+ `qtmesh anim --in-between` CLI, and the `motion_in_between` MCP tool.
24
+
25
+ ## Model
26
+
27
+ - **Input:** `pose_pair` `float32 [1, 2, 220]` — the start and end pose, each
28
+ 22 core-body joints × 10 DoF `[tx,ty,tz, qx,qy,qz,qw, sx,sy,sz]`.
29
+ - **Output:** `interior` `float32 [1, 8, 220]` — 8 predicted in-between poses.
30
+ - **Architecture:** a 4-layer Transformer encoder (start + end pose tokens +
31
+ learnable interior-frame queries → per-frame pose). ~13 MB.
32
+ - **Skeleton:** the 22 CMU core-body joints (hips / spine / neck / head + both
33
+ arms + both legs). QtMeshEditor maps arbitrary rig bone names (Mixamo,
34
+ generic, CMU) onto these roles at runtime; rigs that don't map fall back to a
35
+ deterministic spline.
36
+
37
+ ## Training data & license
38
+
39
+ Trained **from scratch** on the **[CMU Graphics Lab Motion Capture Database](http://mocap.cs.cmu.edu)**
40
+ — permissively licensed (free to use/modify/redistribute, including in
41
+ commercial products; you may not resell the motion data itself; credit
42
+ mocap.cs.cmu.edu). This is what makes the weights redistributable under
43
+ QtMeshEditor's permissive (MIT) distribution — unlike the field-standard LAFAN1
44
+ (CC-BY-NC-ND), which we deliberately avoided.
45
+
46
+ On a held-out CMU split the model beats spherical-linear interpolation (the
47
+ shipped spline fallback) by **more than 2×** on rotation error.
48
+
49
+ Weights released under **CC-BY-4.0**; please credit *QtMeshEditor* and the CMU
50
+ Graphics Lab Motion Capture Database.
51
+
52
+ ## Reproducing
53
+
54
+ `scripts/export-rmib-onnx.py` in the QtMeshEditor repo (one-time, offline; not
55
+ shipped with the app).