ZeyuLing commited on
Commit
0adfa59
·
verified ·
1 Parent(s): 1fed6af

Update supplied-video FlowHMR mesh and tracking demo

Browse files
.gitattributes CHANGED
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ media/demo_input.mp4 filter=lfs diff=lfs merge=lfs -text
37
+ media/demo_latest_mesh.mp4 filter=lfs diff=lfs merge=lfs -text
38
+ media/demo_latest_mesh_preview.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -9,7 +9,7 @@ tags:
9
  - flow-matching
10
  ---
11
 
12
- <h1 align="center">FlowHMR Model Card</h1>
13
 
14
  <p align="center"><strong>491-D SMPL-H motion capture from monocular RGB video, with native flow-matching training.</strong></p>
15
 
@@ -38,25 +38,33 @@ Official source: [FlowHMR](https://github.com/flowhmr/flowhmr), revision
38
 
39
  ## Visual Results
40
 
41
- <!-- MOTIUS_TASK_DEMOS:START -->
42
 
43
- ### Task Demos
44
 
45
- | Task | Input / condition | Rendered output | More |
46
- | --- | --- | --- | --- |
47
- | Monocular Motion Capture | Monocular RGB video | <video src="https://github.com/user-attachments/assets/791ec4c4-638d-478c-b79d-7ddb09d3582b" controls></video> | [MP4](https://github.com/user-attachments/assets/791ec4c4-638d-478c-b79d-7ddb09d3582b) |
48
 
49
- Every public `infer_*` API is represented by a GitHub-native H.264 video player. **All cases** opens the optional interactive comparison.
 
50
 
51
- <!-- MOTIUS_TASK_DEMOS:END -->
 
 
 
52
 
 
 
 
 
 
 
 
 
53
 
54
- | Base | Latest |
55
- | --- | --- |
56
- | <video src="https://github.com/user-attachments/assets/791ec4c4-638d-478c-b79d-7ddb09d3582b" controls></video> | <video src="https://github.com/user-attachments/assets/ca588fd7-52c8-49a8-af11-e0def808a715" controls></video> |
57
 
58
- Real-video previews are generated with the public Motius pipeline, seed 0,
59
- 20 Euler steps, CFG 1 and the official first-frame floor alignment.
60
 
61
  ## Model Overview
62
 
 
9
  - flow-matching
10
  ---
11
 
12
+ <h1 align="center">FlowHMR Latest Model Card</h1>
13
 
14
  <p align="center"><strong>491-D SMPL-H motion capture from monocular RGB video, with native flow-matching training.</strong></p>
15
 
 
38
 
39
  ## Visual Results
40
 
41
+ ### Supplied-video demo — Latest (PHC+ GRPO)
42
 
43
+ <video src="https://huggingface.co/ZeyuLing/Motius-FlowHMR-Latest/resolve/main/media/demo_latest_mesh.mp4" poster="https://huggingface.co/ZeyuLing/Motius-FlowHMR-Latest/resolve/main/media/demo_latest_mesh_preview.png" controls playsinline preload="metadata" width="100%"></video>
44
 
45
+ [Input video](https://huggingface.co/ZeyuLing/Motius-FlowHMR-Latest/resolve/main/media/demo_input.mp4) · [Download demo](https://huggingface.co/ZeyuLing/Motius-FlowHMR-Latest/resolve/main/media/demo_latest_mesh.mp4) ·
46
+ [Render provenance](https://huggingface.co/ZeyuLing/Motius-FlowHMR-Latest/resolve/main/media/demo_latest_mesh.json)
 
47
 
48
+ **Checkpoint:** [Motius-FlowHMR-Latest](https://huggingface.co/ZeyuLing/Motius-FlowHMR-Latest),
49
+ the official Latest release after PHC+ GRPO post-training.
50
 
51
+ - Official checkpoint SHA256: `b7eb06b119feaf5ad7287bafdb2fff38d42dfc22ea9d256f8cdd2ada26dfc53a`.
52
+ - Motius inference safetensors SHA256: `f4ae4168a6f576ffe08a3a19b245e0fd65c0d304b8e93bc2c6ab0a46a6f86912`.
53
+ - 750 frames, 25 seconds, 30 fps; seed 0, 20 Euler steps, CFG 1.
54
+ - VGGT camera sampling every 100 frames (9 keyframes interpolated to 750).
55
 
56
+ The left panel displays the supplied video with the Motius frontend's tracking
57
+ bbox. The right panel reconstructs SMPL-H vertices from the saved predicted
58
+ rotations, shape and translation, using Motius's bundled implementation.
59
+ Three.js provides smooth normals, depth-tested surfaces, lighting and floor
60
+ shadows. The fixed camera fits the complete sequence; horizontal root motion
61
+ is centered for display, while vertical motion is preserved. This is a world
62
+ view, not a mesh reprojection into the source camera. Original audio is retained.
63
+ No external FlowHMR checkout is needed for inference or mesh reconstruction.
64
 
65
+ Earlier parity-validation clips: [Base, 30 frames](https://github.com/user-attachments/assets/791ec4c4-638d-478c-b79d-7ddb09d3582b) ·
66
+ [Latest, 30 frames](https://github.com/user-attachments/assets/ca588fd7-52c8-49a8-af11-e0def808a715). These use a different input clip.
 
67
 
 
 
68
 
69
  ## Model Overview
70
 
media/demo_input.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8f9e9c1193163940eafb512e9a0621a31fe316406e7a69aa3a357f6deceb17f1
3
+ size 6645208
media/demo_latest_mesh.json ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "renderer": "motius-threejs-smooth-mesh",
3
+ "checkpoint_label": "FlowHMR Latest",
4
+ "checkpoint_sha256": "f4ae4168a6f576ffe08a3a19b245e0fd65c0d304b8e93bc2c6ab0a46a6f86912",
5
+ "source_revision": "f12e6a2d46a63d771a66dbb6b5598a1be65b9eea",
6
+ "frames": 750,
7
+ "fps": 30.0,
8
+ "width": 1600,
9
+ "height": 800,
10
+ "vertices": 6890,
11
+ "faces": 13776,
12
+ "view": "root-centered world; fixed camera; original Y retained",
13
+ "camera": {
14
+ "width": 500,
15
+ "height": 585,
16
+ "frames": 750,
17
+ "vertex_count": 6890,
18
+ "half_height": 1.1323156712833715,
19
+ "camera_target": [
20
+ -0.005707634257960004,
21
+ 0.8669544873149713,
22
+ 0.1286162645327557
23
+ ],
24
+ "camera_position": [
25
+ 1.4343384479539978,
26
+ 1.730982136642146,
27
+ -5.631568064315076
28
+ ]
29
+ },
30
+ "source_sha256": "8f9e9c1193163940eafb512e9a0621a31fe316406e7a69aa3a357f6deceb17f1",
31
+ "video_sha256": "ddfd29ce41206457c9e5150b72bf9d1e7eae52f0725eb049442b7bf21404b60d"
32
+ }
media/demo_latest_mesh.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ddfd29ce41206457c9e5150b72bf9d1e7eae52f0725eb049442b7bf21404b60d
3
+ size 7564846
media/demo_latest_mesh_preview.png ADDED

Git LFS Details

  • SHA256: 77aac93373dea54a9ecda2df14af41389de2da447f11b3ff0fe7d67b80890459
  • Pointer size: 131 Bytes
  • Size of remote file: 637 kB