Instructions to use Fendercorp/MocapAnything-MLX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use Fendercorp/MocapAnything-MLX with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download Fendercorp/MocapAnything-MLX --local-dir MocapAnything-MLX
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
|
Download README.md from Fendercorp/MocapAnything-MLX: direct link, hf CLI and curl.
- Browser
- Download file 2.15 kB
-
https://huggingface.co/Fendercorp/MocapAnything-MLX/resolve/main/README.md
- Command line
-
hf download hf://Fendercorp/MocapAnything-MLX/README.md
-
curl -L -o README.md https://huggingface.co/Fendercorp/MocapAnything-MLX/resolve/main/README.md
2.15 kB
| license: mit | |
| library_name: mlx | |
| pipeline_tag: other | |
| tags: | |
| - motion-capture | |
| - mocap | |
| - animation | |
| - retargeting | |
| - mlx | |
| - apple-silicon | |
| base_model: | |
| - kehong/MoCapAnythingV2-weights | |
| - facebook/dinov2-large | |
| - google-t5/t5-base | |
| # MocapAnything-MLX | |
| MoCapAnything V2 (`video2pose2rot`) repacked for 3Dify Studio's native MLX path on Apple Silicon. | |
| It turns a monocular video plus a target skeleton into per-joint rotations on that skeleton. No Python | |
| is needed at runtime. | |
| | Folder | Contents | Size | | |
| |---|---|---| | |
| | `mocap/model.safetensors` | video2pose + pose2rot, F32 (38.5M params) | 154 MB | | |
| | `dinov2-large-224/model.safetensors` | DINOv2 ViT-L/14 image encoder, F16, position table pre-interpolated to 224 px (257 tokens) | 607 MB | | |
| | `t5-base-encoder/model.safetensors` | T5-base encoder only, F16, for joint-name embeddings | 219 MB | | |
| | `t5-base-encoder/vocab.json` | SentencePiece unigram vocab as `[piece, score]` rows | 1 MB | | |
| ## Conversion | |
| - `mocap/` is the `video2pos2rot_epoch60.pt` state dict saved as safetensors. The key names are unchanged. | |
| - `dinov2-large-224/` is `facebook/dinov2-large` with `embeddings.position_embeddings` replaced by | |
| HF's `interpolate_pos_encoding(·, 224, 224)` output, and `mask_token` dropped. | |
| - `t5-base-encoder/` holds the `encoder.*` and `shared.weight` tensors of `t5-base`, plus | |
| `tokenizer.json` → `model.vocab`. | |
| ## Verified (MLX Swift port vs reference) | |
| - T5 joint-name embeddings match the shipped `species_info_dict` vectors (min cosine 1.0000). | |
| - DINOv2 features match HF `transformers` on the same pixels (min token cosine 0.9999). | |
| - Jaguar-Run (y0 reference, y60 input video): MPJPE 0.058 in normalised units, against 0.50 for | |
| holding the reference pose. | |
| ## Licences | |
| - MoCapAnything V2 code and weights: MIT, from [phongdaot/MocapAnything](https://github.com/phongdaot/MocapAnything) and | |
| [kehong/MoCapAnythingV2-weights](https://huggingface.co/kehong/MoCapAnythingV2-weights). | |
| - DINOv2: Apache-2.0, from [facebook/dinov2-large](https://huggingface.co/facebook/dinov2-large). | |
| - T5: Apache-2.0, from [google-t5/t5-base](https://huggingface.co/google-t5/t5-base). | |