Spaces:
Runtime error
Runtime error
| title: Kimodo cpp + MoMask | |
| emoji: 🕺 | |
| colorFrom: pink | |
| colorTo: purple | |
| sdk: gradio | |
| sdk_version: 6.3.0 | |
| python_version: "3.10" | |
| app_file: app.py | |
| pinned: false | |
| fullWidth: true | |
| license: other | |
| license_name: nvidia-open-model-license | |
| license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/ | |
| short_description: Two CPU text-to-motion models, Kimodo GGML and MoMask ONNX | |
| startup_duration_timeout: 1h | |
| preload_from_hub: | |
| - LocalAI-io/Kimodo-SOMA-RP-v1.1-GGML models/kimodo-soma-rp-v1.1-f32.gguf | |
| - LocalAI-io/Kimodo-SOMA-SEED-v1.1-GGML models/kimodo-soma-seed-v1.1-f32.gguf | |
| - LocalAI-io/Kimodo-G1-RP-v1-GGML models/kimodo-g1-rp-v1-f32.gguf | |
| - LocalAI-io/Kimodo-G1-SEED-v1-GGML models/kimodo-g1-seed-v1-f32.gguf | |
| - LocalAI-io/Llama-3-Kimodo-GGML | |
| # Kimodo cpp + MoMask | |
| Two text-to-motion models on one free CPU Space, a tab each. Both take a plain | |
| English description and return a skeletal animation plus a BVH file for Blender. | |
| This Space merges [WeReCooking2/Kimodo-cpp](https://huggingface.co/spaces/WeReCooking2/Kimodo-cpp) | |
| into what was `Luminia/momask-codes`. | |
| ## Which tab to use | |
| | | Kimodo | MoMask | | |
| | --- | --- | --- | | |
| | Runtime | kimodo.cpp, GGML | ONNX Runtime | | |
| | Rigs | SOMA 30 joint human, Unitree G1 34 joint robot | 22 joint human | | |
| | Outputs | glTF orbit preview, BVH, MP4 | MP4, BVH | | |
| | First run on a new prompt | ~2 min, one LLM2Vec encode | ~10 s | | |
| | Repeat prompt | sampling only | same | | |
| MoMask is the one to reach for first because it is roughly an order of magnitude | |
| cheaper. Kimodo is worth the wait when you want the robot rig, a longer clip, or | |
| its markedly better motion quality. | |
| Kimodo's cost is not the sampling, it is the text encode. It conditions on a | |
| 4096 float LLM2Vec embedding produced by a Llama 3 bundle of 35 files, and | |
| `kmd-encode` reopens all of them per call, so one prompt costs one full pass | |
| regardless of length. Embeddings are memoised on the prompt text, so re-rolling a | |
| seed or changing the frame count afterwards costs nothing extra. MoMask has no | |
| equivalent step because CLIP text encoding is inside its ONNX graph. | |
| ## Keeping two native stacks in one process | |
| The two pipelines do not share anything, and the merge is mostly about making | |
| sure they do not stand on each other. | |
| **Import order is load-bearing.** matplotlib's `ft2font` is a pybind11 extension | |
| built against the system libstdc++ and fails to initialise if a differently | |
| linked libstdc++ is already resident. So `app.py` imports the drawing stack | |
| first, before anything native. | |
| **MoMask's backends are deferred.** torch, onnxruntime and CLIP each bring their | |
| own libstdc++ and libgomp, and Kimodo dlopens `libkimodo.so` together with its | |
| own bundled `libgomp.so.1` into the same process. `momask_core._ensure_backends()` | |
| holds them back until the MoMask tab is actually used, so a visitor who only | |
| generates with Kimodo never loads them at all. | |
| **Each pipeline keeps its own weights.** Kimodo runs GGUF checkpoints, MoMask | |
| runs its own ONNX graphs. Nothing is deduplicated or substituted across them, | |
| which keeps any quality complaint attributable to one model. | |
| Two vCPUs run one job at a time, so a second visitor queues. The browser shows a | |
| queue position; an API caller does not. | |
| ## Models | |
| Kimodo, about 1.13 GB each, all preloaded at build time so a wake costs no | |
| download. One is resident at a time. | |
| | Checkpoint | Rig | Joints | | |
| | --- | --- | --- | | |
| | SOMA RP v1.1 | `soma30` | 30 | | |
| | SOMA SEED v1.1 | `soma30` | 30 | | |
| | G1 RP v1 | `g1skel34` | 34 | | |
| | G1 SEED v1 | `g1skel34` | 34 | | |
| MoMask, about 417 MB of ONNX in this repo: CLIP text encoder 254 MB, mask | |
| transformer 56 MB, residual transformer 55 MB, VQ-VAE decoder 46 MB, length | |
| estimator 0.5 MB. | |
| ## Calling it from the API | |
| Two named endpoints, `/generate` for Kimodo and `/momask_generate` for MoMask. | |
| Read them from `/gradio_api/info` rather than guessing. | |
| The Space is also an MCP server at `/gradio_api/mcp/sse`, so an LLM client can | |
| call either tab as a tool. | |
| Watch out for one thing on the streamed response. `data: null` is the heartbeat | |
| payload, not a result and not a failure. It arrives every 15 seconds for as long | |
| as the job is queued or running, so read the `event:` line and treat only | |
| `complete` and `error` as terminal: | |
| ```python | |
| event = None | |
| for line in stream: | |
| if line.startswith("event: "): | |
| event = line[7:].strip() | |
| elif line.startswith("data: ") and event in ("complete", "error"): | |
| payload = line[6:] | |
| break | |
| ``` | |
| `POST /gradio_api/call/generate` returns 200 with an event id whatever happens | |
| next, so judging success by that status is misleading. The outcome is only on | |
| the stream. | |
| ## Credits | |
| [NVIDIA Kimodo](https://www.nvidia.com/) through | |
| [kimodo.cpp](https://github.com/localai-org/kimodo.cpp). | |
| [MoMask](https://github.com/EricGuo5513/momask-codes) by Chuan Guo et al. | |