Instructions to use Fendercorp/MocapAnything-MLX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use Fendercorp/MocapAnything-MLX with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download Fendercorp/MocapAnything-MLX --local-dir MocapAnything-MLX
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
|
Download README.md from Fendercorp/MocapAnything-MLX: direct link, hf CLI and curl.
- Browser
- Download file 2.15 kB
-
https://huggingface.co/Fendercorp/MocapAnything-MLX/resolve/main/README.md
- Command line
-
hf download hf://Fendercorp/MocapAnything-MLX/README.md
-
curl -L -o README.md https://huggingface.co/Fendercorp/MocapAnything-MLX/resolve/main/README.md
2.15 kB
metadata
license: mit
library_name: mlx
pipeline_tag: other
tags:
- motion-capture
- mocap
- animation
- retargeting
- mlx
- apple-silicon
base_model:
- kehong/MoCapAnythingV2-weights
- facebook/dinov2-large
- google-t5/t5-base
MocapAnything-MLX
MoCapAnything V2 (video2pose2rot) repacked for 3Dify Studio's native MLX path on Apple Silicon.
It turns a monocular video plus a target skeleton into per-joint rotations on that skeleton. No Python
is needed at runtime.
| Folder | Contents | Size |
|---|---|---|
mocap/model.safetensors |
video2pose + pose2rot, F32 (38.5M params) | 154 MB |
dinov2-large-224/model.safetensors |
DINOv2 ViT-L/14 image encoder, F16, position table pre-interpolated to 224 px (257 tokens) | 607 MB |
t5-base-encoder/model.safetensors |
T5-base encoder only, F16, for joint-name embeddings | 219 MB |
t5-base-encoder/vocab.json |
SentencePiece unigram vocab as [piece, score] rows |
1 MB |
Conversion
mocap/is thevideo2pos2rot_epoch60.ptstate dict saved as safetensors. The key names are unchanged.dinov2-large-224/isfacebook/dinov2-largewithembeddings.position_embeddingsreplaced by HF'sinterpolate_pos_encoding(·, 224, 224)output, andmask_tokendropped.t5-base-encoder/holds theencoder.*andshared.weighttensors oft5-base, plustokenizer.json→model.vocab.
Verified (MLX Swift port vs reference)
- T5 joint-name embeddings match the shipped
species_info_dictvectors (min cosine 1.0000). - DINOv2 features match HF
transformerson the same pixels (min token cosine 0.9999). - Jaguar-Run (y0 reference, y60 input video): MPJPE 0.058 in normalised units, against 0.50 for holding the reference pose.
Licences
- MoCapAnything V2 code and weights: MIT, from phongdaot/MocapAnything and kehong/MoCapAnythingV2-weights.
- DINOv2: Apache-2.0, from facebook/dinov2-large.
- T5: Apache-2.0, from google-t5/t5-base.