| --- |
| license: other |
| license_name: exmr-license |
| license_link: https://github.com/twehrbein/ExMR/blob/main/LICENSE |
| pipeline_tag: other |
| tags: |
| - human-motion-capture |
| - 3d-human-pose-estimation |
| - smpl-x |
| - hand-pose-estimation |
| - motion-capture |
| - video |
| --- |
| |
| # ExMR: Extremity Motion Refinement for Monocular Human Motion Capture |
|
|
| Model weights for [ExMR](https://github.com/twehrbein/ExMR), which reconstructs 3D body, hand and |
| foot motion (SMPL-X) of a person from monocular video: an initial temporal estimator predicts |
| whole-body motion, after which a foot refinement module (FootMR) and a hand refinement module |
| (HandMR) revisit local image evidence to predict residual joint rotations for the extremities. |
|
|
| > **ExMR: Extremity Motion Refinement for Monocular Human Motion Capture** <br> |
| > Tom Wehrbein and Bodo Rosenhahn, Leibniz University Hannover <br> |
| > [arXiv](https://arxiv.org/abs/2610.26213) | [Project page](https://twehrbein.github.io/exmr-website/) | [Code](https://github.com/twehrbein/ExMR) | [Demo](https://huggingface.co/spaces/twehrbein/ExMR) |
|
|
| Try it without installing anything: the [demo Space](https://huggingface.co/spaces/twehrbein/ExMR) |
| runs ExMR on up to 20 seconds of your video and returns the SMPL-X parameters and joints, the |
| motion for Blender or humanoid robots (AMASS file) and for animation software (BVH). |
|
|
| ## Files |
|
|
| | File | Content | SHA256 | |
| |---|---|---| |
| | `exmr.safetensors` | ExMR model (EMA weights, 236 MB) | `8d3947310c30444ec59dae4662574d7a32bf83af30320eeac87146529ff2fda5` | |
| | `rtmw-x_384x288_fp16.onnx` | RTMW-x 2D whole-body keypoint detector, FP16 (optional `--pose_backend rtmw`), see below | `957ddb5ed34576593a918b3e514003ab9cc85d037309b9a23d9e0209530e98ed` | |
| | `wheels/nvdiffrast-0.4.0+pt2.8.0cu128-cp312-cp312-linux_x86_64.whl` | nvdiffrast renderer for torch 2.8.0+cu128, Python 3.12, Linux (installed by ExMR's `docs/INSTALL.md`), see below | `ca26d44f4f198e8fb460f86f355c5fdaeaac6319e15a5551ea31397719c0d563` | |
|
|
| ## Usage |
|
|
| Do not download these files by hand: the ExMR download script fetches them together with the |
| third-party models ExMR needs and checks every file: |
|
|
| ```bash |
| git clone --recursive https://github.com/twehrbein/ExMR.git && cd ExMR |
| # set up the environment as described in docs/INSTALL.md, then |
| python tools/download_models.py --smplx_zip models_smplx_v1_1.zip --mano_zip mano_v1_2.zip |
| python tools/demo.py --video path/to/video.mp4 |
| ``` |
|
|
| See the [GitHub repository](https://github.com/twehrbein/ExMR) for installation, the demo and the |
| output format. |
|
|
| ## Training data |
|
|
| ExMR was trained on [BEDLAM](https://bedlam.is.tue.mpg.de/), |
| [BEDLAM2](https://bedlam2.is.tuebingen.mpg.de/), [Human3.6M](http://vision.imar.ro/human3.6m/), |
| [3DPW](https://virtualhumans.mpi-inf.mpg.de/3DPW/) and [ARCTIC](https://arctic.is.tue.mpg.de/). |
| Its inputs are image features of SAM 3D Body and HaMeR and 2D keypoints of Sapiens2 or RTMW; its |
| outputs are [SMPL-X](https://smpl-x.is.tue.mpg.de/) parameters. |
|
|
| ## License |
|
|
| `exmr.safetensors` is released under the same terms as the ExMR code |
| ([LICENSE](https://github.com/twehrbein/ExMR/blob/main/LICENSE)): for educational, research and |
| non-profit purposes only. The training datasets and the SMPL-X body model are themselves licensed |
| for non-commercial research only. |
|
|
| `rtmw-x_384x288_fp16.onnx` is a redistribution of RTMW-x by |
| [OpenMMLab (MMPose)](https://github.com/open-mmlab/mmpose/tree/main/projects/rtmpose), licensed |
| under Apache-2.0 (see `RTMW_LICENSE.txt` in this repository). It is the FP16 conversion of the ONNX |
| export `RTMW_x_lastest.onnx` from |
| [hunminkim98/Models_Pose2Sim](https://huggingface.co/hunminkim98/Models_Pose2Sim) (commit |
| `53abe39`), made with onnx 1.23.2 and onnxconverter-common 1.16.0: |
|
|
| ```python |
| import onnx |
| from onnxconverter_common import float16 |
| |
| model = onnx.load("RTMW_x_lastest.onnx") |
| onnx.save(float16.convert_float_to_float16(model, keep_io_types=True), "rtmw-x_384x288_fp16.onnx") |
| ``` |
|
|
| The only change is the conversion of the internal computations to FP16 (inputs and outputs stay |
| FP32). We host it so that ExMR's results can be reproduced with exactly this file. |
|
|
| `wheels/nvdiffrast-0.4.0+pt2.8.0cu128-cp312-cp312-linux_x86_64.whl` is a build of |
| [nvdiffrast](https://github.com/NVlabs/nvdiffrast) v0.4.0 by NVIDIA, licensed under the NVIDIA |
| Source Code License for non-commercial use (research or evaluation) only (see |
| `NVDIFFRAST_LICENSE.txt` in this repository; the wheel contains it too). It is built from the |
| unmodified v0.4.0 source (commit `253ac4f`; only the version label `+pt2.8.0cu128` is added) with |
| `wheels/build_nvdiffrast_wheel.sh`: conda-forge nvcc 12.8 and gcc 14 against glibc 2.28, for the |
| GPUs that torch 2.8.0+cu128 supports. We host it so that ExMR installs without compiling. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @Article{wehrbein26exmr, |
| author = {Wehrbein, Tom and Rosenhahn, Bodo}, |
| title = {{ExMR}: Extremity Motion Refinement for Monocular Human Motion Capture}, |
| journal = {arXiv preprint arXiv:2610.26213}, |
| year = {2026}, |
| } |
| ``` |
|
|
| Please also cite [GVHMR](https://github.com/zju3dv/GVHMR), which ExMR builds on, and the |
| third-party models listed in the ExMR repository's |
| [THIRD_PARTY_NOTICES.md](https://github.com/twehrbein/ExMR/blob/main/THIRD_PARTY_NOTICES.md). |
|
|
