Commit ·
656cdfd
0
Parent(s):
Publish the SeMoCo main-method weights
Browse files- .gitattributes +35 -0
- README.md +72 -0
- humanml3d/feature_stats.npz +3 -0
- humanml3d/flow.pt +3 -0
- humanml3d/kin_stats.npz +3 -0
- humanml3d/semantic_ar.pt +3 -0
- humanml3d/tokenizer.pt +3 -0
- omega/feature_stats.npz +3 -0
- omega/flow.pt +3 -0
- omega/kin_stats.npz +3 -0
- omega/semantic_ar.pt +3 -0
- omega/tokenizer.pt +3 -0
.gitattributes
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
*.7z filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
+
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
+
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
+
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
+
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
+
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
+
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
+
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
+
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
+
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
+
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 14 |
+
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 15 |
+
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 16 |
+
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 17 |
+
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 18 |
+
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 19 |
+
*.pb filter=lfs diff=lfs merge=lfs -text
|
| 20 |
+
*.pickle filter=lfs diff=lfs merge=lfs -text
|
| 21 |
+
*.pkl filter=lfs diff=lfs merge=lfs -text
|
| 22 |
+
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 23 |
+
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 24 |
+
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 25 |
+
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 26 |
+
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
| 27 |
+
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
+
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 29 |
+
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 30 |
+
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 31 |
+
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 32 |
+
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
+
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
+
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
+
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
README.md
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: pytorch
|
| 4 |
+
tags:
|
| 5 |
+
- human-motion
|
| 6 |
+
- text-to-motion
|
| 7 |
+
- flow-matching
|
| 8 |
+
---
|
| 9 |
+
|
| 10 |
+
# SeMoCo: Semantic Motion Tokens with Body-Structured Flow Matching
|
| 11 |
+
|
| 12 |
+
[](https://github.com/OMEGA-i/SeMoCo)
|
| 13 |
+
[](https://arxiv.org/abs/2608.24334)
|
| 14 |
+
[](https://omega-i.github.io/SeMoCo-Page/)
|
| 15 |
+
|
| 16 |
+
## Introduction
|
| 17 |
+
|
| 18 |
+
SeMoCo couples a semantically supervised motion codec with body-structured flow matching for text-to-motion generation. An autoregressive model predicts a semantic code sequence from text, and a conditional flow generates continuous kinematic features for the root, torso, and limbs. The semantic codes and kinematic features are decoded together into motion.
|
| 19 |
+
|
| 20 |
+
This repository contains the pretrained HumanML3D and Ω-MotionVerse models for the main results. Each directory has the tokenizer, semantic model, six-part flow, and the normalization statistics used with those weights.
|
| 21 |
+
|
| 22 |
+
## Model Files
|
| 23 |
+
|
| 24 |
+
| File | Contents |
|
| 25 |
+
|---|---|
|
| 26 |
+
| `humanml3d/tokenizer.pt` | HumanML3D codec |
|
| 27 |
+
| `humanml3d/semantic_ar.pt` | HumanML3D semantic model |
|
| 28 |
+
| `humanml3d/flow.pt` | HumanML3D six-part flow |
|
| 29 |
+
| `humanml3d/kin_stats.npz`, `humanml3d/feature_stats.npz` | Latent and Guo263 normalization |
|
| 30 |
+
| `omega/tokenizer.pt` | Ω-MotionVerse codec |
|
| 31 |
+
| `omega/semantic_ar.pt` | Ω-MotionVerse semantic model |
|
| 32 |
+
| `omega/flow.pt` | Ω-MotionVerse six-part flow |
|
| 33 |
+
| `omega/kin_stats.npz`, `omega/feature_stats.npz` | Latent and UMR499 normalization |
|
| 34 |
+
|
| 35 |
+
## Evaluation
|
| 36 |
+
|
| 37 |
+
| Dataset | FID | R@1 | R@2 | R@3 | MM-Dist |
|
| 38 |
+
|---|---:|---:|---:|---:|---:|
|
| 39 |
+
| HumanML3D | 0.0327 | 0.5561 | 0.7473 | 0.8365 | 2.750 |
|
| 40 |
+
| Ω-MotionVerse | 0.01238 | 0.7325 | 0.8143 | 0.8446 | 0.963 |
|
| 41 |
+
|
| 42 |
+
HumanML3D reports the mean of 20 repeats with the checkpoint held fixed. Ω-MotionVerse is one evaluation of the full test set.
|
| 43 |
+
|
| 44 |
+
## Usage
|
| 45 |
+
|
| 46 |
+
```bash
|
| 47 |
+
hf download poisonousID/SeMoCo --local-dir checkpoints
|
| 48 |
+
```
|
| 49 |
+
|
| 50 |
+
Then generate with the configs from the [code repository](https://github.com/OMEGA-i/SeMoCo):
|
| 51 |
+
|
| 52 |
+
```bash
|
| 53 |
+
semoco-generate --config configs/humanml3d_inference.yaml \
|
| 54 |
+
--inputs data/prompts.npz --output runs/motion.npz
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
+
Use `configs/omega_inference.yaml` for Ω-MotionVerse.
|
| 58 |
+
|
| 59 |
+
## License
|
| 60 |
+
|
| 61 |
+
Apache-2.0.
|
| 62 |
+
|
| 63 |
+
## Citation
|
| 64 |
+
|
| 65 |
+
```bibtex
|
| 66 |
+
@article{semoco,
|
| 67 |
+
title = {SeMoCo: Semantic Motion Tokens with Body-Structured Flow Matching},
|
| 68 |
+
author = {OMEGA-team},
|
| 69 |
+
journal = {arXiv preprint arXiv:2608.24334},
|
| 70 |
+
year = {2026}
|
| 71 |
+
}
|
| 72 |
+
```
|
humanml3d/feature_stats.npz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b5b1004be5bc1834ae4640ad4a54baf5b679fcd7ffbfd864fc8a946552d5cc18
|
| 3 |
+
size 2604
|
humanml3d/flow.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3924a797a548bb4655a9da2ee991b10574f81130136c716170260b17a4e37b01
|
| 3 |
+
size 1711337197
|
humanml3d/kin_stats.npz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e7b722f2be60c0064f522c955d7fa84140878a492d2822c414660ce0aeb7264b
|
| 3 |
+
size 4846
|
humanml3d/semantic_ar.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f8f7b522972bfd100053ead5c9ffa04c28a12ec78dfb6596578b616dd20d380c
|
| 3 |
+
size 1124484731
|
humanml3d/tokenizer.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:87f0d3b2ea5c050a0d26ffad0db39c2136bdfbedabb0ac379bf80cc5b7bd5c33
|
| 3 |
+
size 100330845
|
omega/feature_stats.npz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:eb95e20bfc0b8fe3d8fe9491bfda3487d9d2d07b7e436dd8e45aba10a4552c4b
|
| 3 |
+
size 4492
|
omega/flow.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a3bcaf4675e061d85c57dd1153be86af6a5c4fac3a40214dfd9ab6d350b98333
|
| 3 |
+
size 1714542829
|
omega/kin_stats.npz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a7448a3613027b730164f9b853e2c2eb59afd257ec9b3fa44eee3b9aaebbcec7
|
| 3 |
+
size 4846
|
omega/semantic_ar.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9b1b10d67f3388c5aa2703444272cb9f3fdaa941b7c3cfa30d4ef0a8f04592cd
|
| 3 |
+
size 1124734587
|
omega/tokenizer.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ef9088e4c139696e5ab06a3e9ea3befa9954ea6aa8610c17c038f3e7f782dee6
|
| 3 |
+
size 74883639
|