|
Download README.md from fastforks/rtmpose: direct link, hf CLI and curl.
- Browser
- Download file 26.9 kB
-
https://huggingface.co/fastforks/rtmpose/resolve/main/README.md
- Command line
-
hf download hf://fastforks/rtmpose/README.md
-
curl -L -o README.md https://huggingface.co/fastforks/rtmpose/resolve/main/README.md
26.9 kB
| library_name: "fastforks" | |
| pipeline_tag: "keypoint-detection" | |
| license: "apache-2.0" | |
| tags: | |
| - "pose" | |
| - "pose-estimation" | |
| - "human-pose-estimation" | |
| - "keypoint-detection" | |
| - "cuda" | |
| - "pytorch" | |
| - "fastforks" | |
| - "fastforks-card" | |
| # RTMPose (fastforks) | |
| RTMPose (Jiang et al. 2023): real-time top-down 2D human pose from OpenMMLab: a CSPNeXt CNN backbone and a SimCC head (a gated attention unit over one token per keypoint, then x and y classified as 1D bins at half-pixel resolution, no heatmaps). The body7 checkpoints (trained on seven public pose datasets) return the 17 COCO keypoints; the DWPose ones (Yang et al. 2023, distilled on COCO-WholeBody + UBody, the pose model of ControlNet's openpose preprocessors) the 133 COCO-WholeBody keypoints (body, feet, face, both hands). The RTMW ones (Jiang et al. 2024: a PAFPN neck and a two-scale head on the same backbone, trained on 14 datasets) return the same 133 keypoints more accurately. More model-zoo checkpoints of the same network: `-halpe26` (body + feet, 26 keypoints), `-hand5` (21 keypoints of a hand box), `-face6` (106 face landmarks of a face box) and `-ap10k` (17 keypoints of an animal box). Takes person boxes (e.g. from `fastforks.detection`) and crops each at 256 x 192 (or 384 x 288; hand, face and animal boxes: 256 x 256); vendored from mmpose (no mmpose / mmcv dependency). The portable `module()` maps crops to SimCC logits. | |
| This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: <https://github.com/kleinicke/fastforks>. Kernels for the `kernels` backend: <https://huggingface.co/fastforks/kernels>. Original code: <https://github.com/open-mmlab/mmpose> at `5408bc76f5b8`. | |
| Task: pose. Tier: stable. Outputs: `keypoints`, `keypoint_scores`, `boxes`, `scores`. Backends: `portable`, `kernels`. | |
| Input: one RGB image and a box per person. Output: keypoints with confidence scores for every person. | |
| ## Variants | |
| Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in [`card.toml`](https://github.com/kleinicke/fastforks/blob/main/src/fastforks/pose/models/rtmpose/card.toml). | |
| | variant | weights (author link) | SHA-256 | weights licence | commercial use | | |
| |---|---|---|---|---| | |
| | `dwpose-l@1` | [dwpose-l-ucoco-256x192.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-l_simcc-ucoco_dw-ucoco_270e-256x192-4d6dfc62_20230728.pth) | `4d6dfc622672` | Apache-2.0 | yes | | |
| | `dwpose-l-384@1` | [dwpose-l-ucoco-384x288.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-l_simcc-ucoco_dw-ucoco_270e-384x288-2438fd99_20230728.pth) | `2438fd992e65` | Apache-2.0 | yes | | |
| | `dwpose-m@1` | [dwpose-m-ucoco-256x192.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-ucoco_dw-ucoco_270e-256x192-c8b76419_20230728.pth) | `c8b7641988ce` | Apache-2.0 | yes | | |
| | `dwpose-s@1` | [dwpose-s-ucoco-256x192.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-s_simcc-ucoco_dw-ucoco_270e-256x192-3fd922c8_20230728.pth) | `3fd922c87ac2` | Apache-2.0 | yes | | |
| | `dwpose-t@1` | [dwpose-t-ucoco-256x192.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-t_simcc-ucoco_dw-ucoco_270e-256x192-dcf277bf_20230728.pth) | `dcf277bf6ba4` | Apache-2.0 | yes | | |
| | `rtmpose-l@1` | [rtmpose-l-body7-256x192.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-l_simcc-body7_pt-body7_420e-256x192-4dba18fc_20230504.pth) | `4dba18fc4632` | Apache-2.0 | yes | | |
| | `rtmpose-l-384@1` | [rtmpose-l-body7-384x288.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-l_simcc-body7_pt-body7_420e-384x288-3f5a1437_20230504.pth) | `3f5a14373b19` | Apache-2.0 | yes | | |
| | `rtmpose-l-halpe26@1` | [rtmpose-l-halpe26-256x192.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-l_simcc-body7_pt-body7-halpe26_700e-256x192-2abb7558_20230605.pth) | `2abb7558f5bf` | Apache-2.0 | yes | | |
| | `rtmpose-l-halpe26-384@1` | [rtmpose-l-halpe26-384x288.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-l_simcc-body7_pt-body7-halpe26_700e-384x288-734182ce_20230605.pth) | `734182ce2409` | Apache-2.0 | yes | | |
| | `rtmpose-m@1` | [rtmpose-m-body7-256x192.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-body7_pt-body7_420e-256x192-e48f03d0_20230504.pth) | `e48f03d0cfe1` | Apache-2.0 | yes | | |
| | `rtmpose-m-384@1` | [rtmpose-m-body7-384x288.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-body7_pt-body7_420e-384x288-65e718c4_20230504.pth) | `65e718c4ccb6` | Apache-2.0 | yes | | |
| | `rtmpose-m-ap10k@1` | [rtmpose-m-ap10k-256x256.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-ap10k_pt-aic-coco_210e-256x256-7a041aa1_20230206.pth) | `896e3665d849` | Apache-2.0 | yes | | |
| | `rtmpose-m-face6@1` | [rtmpose-m-face6-256x256.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-face6_pt-in1k_120e-256x256-72a37400_20230529.pth) | `72a37400a409` | Apache-2.0 | yes | | |
| | `rtmpose-m-halpe26@1` | [rtmpose-m-halpe26-256x192.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-body7_pt-body7-halpe26_700e-256x192-4d3e73dd_20230605.pth) | `4d3e73ddd312` | Apache-2.0 | yes | | |
| | `rtmpose-m-halpe26-384@1` | [rtmpose-m-halpe26-384x288.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-body7_pt-body7-halpe26_700e-384x288-89e6428b_20230605.pth) | `89e6428b3890` | Apache-2.0 | yes | | |
| | `rtmpose-m-hand5@1` | [rtmpose-m-hand5-256x256.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-hand5_pt-aic-coco_210e-256x256-74fb594_20230320.pth) | `b74fb5941684` | Apache-2.0 | yes | | |
| | `rtmpose-s@1` | [rtmpose-s-body7-256x192.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-s_simcc-body7_pt-body7_420e-256x192-acd4a1ef_20230504.pth) | `acd4a1efd8f0` | Apache-2.0 | yes | | |
| | `rtmpose-s-face6@1` | [rtmpose-s-face6-256x256.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-s_simcc-face6_pt-in1k_120e-256x256-d779fdef_20230529.pth) | `d779fdefe9e7` | Apache-2.0 | yes | | |
| | `rtmpose-s-halpe26@1` | [rtmpose-s-halpe26-256x192.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-s_simcc-body7_pt-body7-halpe26_700e-256x192-7f134165_20230605.pth) | `7f1341650e93` | Apache-2.0 | yes | | |
| | `rtmpose-t@1` | [rtmpose-t-body7-256x192.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-t_simcc-body7_pt-body7_420e-256x192-026a1439_20230504.pth) | `026a1439c569` | Apache-2.0 | yes | | |
| | `rtmpose-t-face6@1` | [rtmpose-t-face6-256x256.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-t_simcc-face6_pt-in1k_120e-256x256-df79d9a5_20230529.pth) | `df79d9a59eee` | Apache-2.0 | yes | | |
| | `rtmpose-t-halpe26@1` | [rtmpose-t-halpe26-256x192.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-t_simcc-body7_pt-body7-halpe26_700e-256x192-6020f8a6_20230605.pth) | `6020f8a67466` | Apache-2.0 | yes | | |
| | `rtmpose-x-halpe26-384@1` | [rtmpose-x-halpe26-384x288.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-x_simcc-body7_pt-body7-halpe26_700e-384x288-7fb6e239_20230606.pth) | `7fb6e2396010` | Apache-2.0 | yes | | |
| | `rtmw-l@1` | [rtmw-l-cocktail14-256x192.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmw/rtmw-dw-x-l_simcc-cocktail14_270e-256x192-20231122.pth) | `ae6459e54e52` | Apache-2.0 | yes | | |
| | `rtmw-l-384@1` | [rtmw-l-cocktail14-384x288.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmw/rtmw-dw-x-l_simcc-cocktail14_270e-384x288-20231122.pth) | `afa589fa16bf` | Apache-2.0 | yes | | |
| | `rtmw-m@1` | [rtmw-m-cocktail14-256x192.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmw/rtmw-dw-l-m_simcc-cocktail14_270e-256x192-20231122.pth) | `6c481e4f9d3b` | Apache-2.0 | yes | | |
| | `rtmw-x@1` | [rtmw-x-cocktail14-256x192.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmw/rtmw-x_simcc-cocktail14_pt-ucoco_270e-256x192-13a2546d_20231208.pth) | `13a2546d5978` | Apache-2.0 | yes | | |
| | `rtmw-x-384@1` | [rtmw-x-cocktail14-384x288.pth](https://download.openmmlab.com/mmpose/v1/projects/rtmw/rtmw-x_simcc-cocktail14_pt-ucoco_270e-384x288-f840f204_20231122.pth) | `f840f2044fe4` | Apache-2.0 | yes | | |
| ## Licence | |
| Original-code strings, recorded as they are (research-use package; check them before any other use). Code: Apache-2.0. Licence files: `LICENSE`. | |
| Training data: COCO 2017 (train, instance annotations), AI Challenger human keypoints (2017), CrowdPose, MPII Human Pose, sub-JHMDB (J-HMDB), Halpe-FullBody, PoseTrack 2018, ImageNet-1k (ILSVRC 2012 classification) (body7: COCO, AI Challenger, CrowdPose, MPII, sub-JHMDB, Halpe and PoseTrack18 keypoints; the CSPNeXt backbone was pre-trained on ImageNet-1k (and on the same pose data, `pt-body7`)). | |
| Licence evidence: <https://github.com/open-mmlab/mmpose/blob/5408bc76f5b848cf925a0d1857899011d8c5b497/README.md#license> | |
| ## Input size | |
| any H x W >= 32, padded to a multiple of 32 | |
| Tested up to (portable, fp16, RTX 5060 Ti 16 GB): `dwpose-l` 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); `dwpose-l-384` 3000x4000 (0.6 GiB peak allocated, 1.0 reserved); `dwpose-m` 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); `dwpose-s` 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); `dwpose-t` 3000x4000 (0.4 GiB peak allocated, 0.5 reserved); `rtmpose-l` 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); `rtmpose-l-384` 3000x4000 (0.6 GiB peak allocated, 1.0 reserved); `rtmpose-l-halpe26` 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); `rtmpose-l-halpe26-384` 3000x4000 (0.6 GiB peak allocated, 1.0 reserved); `rtmpose-m` 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); `rtmpose-m-384` 3000x4000 (0.5 GiB peak allocated, 0.6 reserved); `rtmpose-m-ap10k` 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); `rtmpose-m-face6` 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); `rtmpose-m-halpe26` 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); `rtmpose-m-halpe26-384` 3000x4000 (0.5 GiB peak allocated, 0.6 reserved); `rtmpose-m-hand5` 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); `rtmpose-s` 3000x4000 (0.4 GiB peak allocated, 0.5 reserved); `rtmpose-s-face6` 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); `rtmpose-s-halpe26` 3000x4000 (0.4 GiB peak allocated, 0.5 reserved); `rtmpose-t` 3000x4000 (0.4 GiB peak allocated, 0.5 reserved); `rtmpose-t-face6` 3000x4000 (0.4 GiB peak allocated, 0.5 reserved); `rtmpose-t-halpe26` 3000x4000 (0.4 GiB peak allocated, 0.5 reserved); `rtmpose-x-halpe26-384` 3000x4000 (0.8 GiB peak allocated, 0.9 reserved); `rtmw-l` 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); `rtmw-l-384` 3000x4000 (0.7 GiB peak allocated, 1.0 reserved); `rtmw-m` 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); `rtmw-x` 3000x4000 (0.6 GiB peak allocated, 0.7 reserved); `rtmw-x-384` 3000x4000 (0.8 GiB peak allocated, 1.0 reserved) | |
| ## Speed and memory | |
| ### Pose, synthetic, 1080x1920 | |
| | model | time (ms): original fp32 | original | portable | kernels | kernels speed-up: vs original | vs portable | peak memory (MiB, allocated / reserved): original | portable | kernels | | |
| | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | |
| | `dwpose-l` | 178 | 184 | 10.2 | 6.20 | 28.7x | 1.7x | 393 / 428 | 222 / 280 | 186 / 306 | | |
| | `dwpose-l-384` | 194 | 203 | 21.2 | 12.5 | 15.5x | 1.7x | 773 / 816 | 351 / 460 | 240 / 424 | | |
| | `dwpose-m` | 169 | 164 | 7.25 | 4.86 | 33.8x | 1.5x | 756 / 792 | 162 / 196 | 157 / 220 | | |
| | `dwpose-s` | 173 | 180 | 5.03 | 3.30 | 52.3x | 1.5x | 806 / 848 | 129 / 174 | 138 / 198 | | |
| | `dwpose-t` | 160 | 174 | 4.47 | 3.15 | 50.8x | 1.4x | 832 / 858 | 124 / 152 | 133 / 192 | | |
| | `rtmpose-l` | 161 | 176 | 8.94 | 5.63 | 28.6x | 1.6x | 1105 / 1154 | 211 / 236 | 175 / 264 | | |
| | `rtmpose-l-384` | 192 | 171 | 19.3 | 11.2 | 15.3x | 1.7x | 366 / 410 | 340 / 436 | 229 / 406 | | |
| | `rtmpose-l-halpe26` | 180 | 164 | 9.06 | 5.65 | 29.1x | 1.6x | 529 / 572 | 212 / 276 | 176 / 264 | | |
| | `rtmpose-l-halpe26-384` | 206 | 206 | 19.3 | 11.2 | 18.4x | 1.7x | 975 / 1038 | 341 / 436 | 230 / 406 | | |
| | `rtmpose-m` | 179 | 180 | 6.28 | 4.42 | 40.4x | 1.4x | 316 / 364 | 153 / 192 | 148 / 200 | | |
| | `rtmpose-m-384` | 174 | 167 | 13.1 | 8.97 | 18.6x | 1.5x | 505 / 582 | 243 / 352 | 202 / 340 | | |
| | `rtmpose-m-ap10k` | 196 | 186 | 7.98 | 5.41 | 34.5x | 1.5x | 757 / 814 | 177 / 234 | 163 / 220 | | |
| | `rtmpose-m-face6` | 198 | 207 | 8.69 | 5.78 | 34.3x | 1.5x | 652 / 698 | 184 / 256 | 169 / 242 | | |
| | `rtmpose-m-halpe26` | 179 | 181 | 6.35 | 4.46 | 40.2x | 1.4x | 283 / 318 | 154 / 192 | 149 / 220 | | |
| | `rtmpose-m-halpe26-384` | 192 | 193 | 13.2 | 9.01 | 21.3x | 1.5x | 690 / 764 | 244 / 352 | 203 / 340 | | |
| | `rtmpose-m-hand5` | 192 | 195 | 8.04 | 5.44 | 35.3x | 1.5x | 425 / 472 | 178 / 234 | 163 / 240 | | |
| | `rtmpose-s` | 161 | 165 | 4.92 | 3.01 | 53.4x | 1.6x | 449 / 480 | 124 / 172 | 133 / 178 | | |
| | `rtmpose-s-face6` | 258 | 237 | 5.03 | 3.75 | 63.2x | 1.3x | 504 / 540 | 147 / 174 | 151 / 222 | | |
| | `rtmpose-s-halpe26` | 172 | 173 | 4.98 | 3.04 | 56.5x | 1.6x | 162 / 200 | 124 / 172 | 133 / 198 | | |
| | `rtmpose-t` | 165 | 151 | 4.49 | 2.90 | 52.0x | 1.5x | 459 / 490 | 120 / 150 | 129 / 176 | | |
| | `rtmpose-t-face6` | 188 | 204 | 4.58 | 3.63 | 51.8x | 1.3x | 424 / 464 | 134 / 152 | 146 / 220 | | |
| | `rtmpose-t-halpe26` | 181 | 172 | 4.48 | 2.90 | 59.1x | 1.5x | 103 / 144 | 120 / 150 | 129 / 196 | | |
| | `rtmpose-x-halpe26-384` | 247 | 217 | 33.5 | 18.7 | 11.6x | 1.8x | 470 / 556 | 401 / 498 | 280 / 470 | | |
| | `rtmw-l` | 213 | 203 | 15.1 | 10.2 | 19.8x | 1.5x | 865 / 908 | 267 / 308 | 232 / 336 | | |
| | `rtmw-l-384` | 274 | 275 | 31.5 | 20.2 | 13.6x | 1.6x | 1094 / 1144 | 396 / 508 | 286 / 476 | | |
| | `rtmw-m` | 169 | 205 | 11.2 | 7.54 | 22.5x | 1.5x | 379 / 410 | 189 / 224 | 184 / 268 | | |
| | `rtmw-x` | 218 | 207 | 23.9 | 14.3 | 14.5x | 1.7x | 604 / 636 | 342 / 392 | 298 / 394 | | |
| | `rtmw-x-384` | 306 | 291 | 50.6 | 50.8 | 5.7x | 1.0x | 1874 / 1940 | 482 / 592 | 496 / 674 | | |
| ### Startup and first call | |
| | model | load (s): original | portable | kernels | first call (s): original | portable | kernels | | |
| | --- | --- | --- | --- | --- | --- | --- | | |
| | `dwpose-l` | 0.50 | 0.88 | 1.64 | 1.14 | 1.83 | 0.28 | | |
| | `dwpose-l-384` | 0.42 | 0.58 | 1.67 | 0.42 | 1.10 | 0.28 | | |
| | `dwpose-m` | 0.26 | 0.24 | 1.51 | 0.60 | 1.10 | 0.29 | | |
| | `dwpose-s` | 0.19 | 0.12 | 1.48 | 0.47 | 1.02 | 0.29 | | |
| | `dwpose-t` | 0.16 | 0.08 | 1.42 | 0.33 | 0.89 | 0.29 | | |
| | `rtmpose-l` | 0.38 | 0.94 | 1.60 | 0.18 | 0.86 | 0.29 | | |
| | `rtmpose-l-384` | 0.58 | 0.41 | 1.56 | 0.17 | 1.00 | 0.29 | | |
| | `rtmpose-l-halpe26` | 0.37 | 0.25 | 1.55 | 0.45 | 1.13 | 0.29 | | |
| | `rtmpose-l-halpe26-384` | 0.41 | 0.26 | 1.57 | 0.38 | 1.24 | 0.29 | | |
| | `rtmpose-m` | 0.23 | 0.22 | 1.50 | 0.16 | 1.10 | 0.29 | | |
| | `rtmpose-m-384` | 0.23 | 0.20 | 1.48 | 0.35 | 1.25 | 0.29 | | |
| | `rtmpose-m-ap10k` | 0.23 | 0.13 | 1.48 | 0.19 | 1.14 | 0.29 | | |
| | `rtmpose-m-face6` | 0.29 | 0.15 | 1.49 | 0.20 | 1.15 | 0.29 | | |
| | `rtmpose-m-halpe26` | 0.25 | 0.14 | 1.51 | 0.45 | 1.04 | 0.30 | | |
| | `rtmpose-m-halpe26-384` | 0.26 | 0.13 | 1.48 | 0.34 | 1.15 | 0.29 | | |
| | `rtmpose-m-hand5` | 0.23 | 0.13 | 1.49 | 0.22 | 1.13 | 0.29 | | |
| | `rtmpose-s` | 0.16 | 0.10 | 1.40 | 0.16 | 1.03 | 0.29 | | |
| | `rtmpose-s-face6` | 0.31 | 0.08 | 1.43 | 0.35 | 1.18 | 0.29 | | |
| | `rtmpose-s-halpe26` | 0.19 | 0.07 | 1.46 | 0.61 | 0.94 | 0.30 | | |
| | `rtmpose-t` | 0.13 | 0.10 | 1.42 | 0.15 | 0.88 | 0.29 | | |
| | `rtmpose-t-face6` | 0.18 | 0.06 | 1.45 | 0.34 | 1.29 | 0.29 | | |
| | `rtmpose-t-halpe26` | 0.18 | 0.05 | 1.48 | 0.71 | 0.81 | 0.29 | | |
| | `rtmpose-x-halpe26-384` | 0.89 | 0.41 | 1.76 | 0.94 | 1.17 | 0.28 | | |
| | `rtmw-l` | 0.76 | 0.39 | 1.72 | 0.63 | 1.03 | 0.31 | | |
| | `rtmw-l-384` | 0.75 | 0.34 | 1.69 | 0.56 | 1.20 | 0.28 | | |
| | `rtmw-m` | 0.54 | 0.35 | 1.64 | 1.21 | 1.20 | 0.29 | | |
| | `rtmw-x` | 1.23 | 1.35 | 2.10 | 0.64 | 0.83 | 0.28 | | |
| | `rtmw-x-384` | 1.05 | 0.81 | 2.20 | 0.42 | 0.96 | 0.34 | | |
| ### torch.compile and TensorRT | |
| | model | input | time: portable | compile | TensorRT fp16 | kernels | preparation: compile (cold first call) | ONNX + engine build | peak MiB (alloc / reserved): portable | compile | kernels | TensorRT (engine + torch) | | |
| | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | |
| | `dwpose-l` | 1080x1920 | 10.2 ms | 5.75 ms | 3.33 ms (network on person crops only) | 6.20 ms | 14.8 s | 20.0 s | 222 / 280 | 197 / 216 | 186 / 306 | 277 / 294 | | |
| | `dwpose-l-384` | 1080x1920 | 21.2 ms | 11.9 ms | 7.46 ms (network on person crops only) | 12.5 ms | 14.9 s | 20.6 s | 351 / 460 | 253 / 296 | 240 / 424 | 347 / 370 | | |
| | `dwpose-m` | 1080x1920 | 7.25 ms | 3.80 ms | 2.13 ms (network on person crops only) | 4.86 ms | 12.9 s | 18.9 s | 162 / 196 | 156 / 174 | 157 / 220 | 203 / 228 | | |
| | `dwpose-s` | 1080x1920 | 5.03 ms | 2.27 ms | 1.17 ms (network on person crops only) | 3.30 ms | 11.8 s | 16.7 s | 129 / 174 | 135 / 150 | 138 / 198 | 149 / 170 | | |
| | `dwpose-t` | 1080x1920 | 4.47 ms | 1.96 ms | 0.940 ms (network on person crops only) | 3.15 ms | 11.2 s | 16.1 s | 124 / 152 | 129 / 148 | 133 / 192 | 136 / 160 | | |
| | `rtmpose-l` | 1080x1920 | 8.94 ms | 5.32 ms | 3.20 ms (network on person crops only) | 5.63 ms | 14.2 s | 19.4 s | 211 / 236 | 185 / 194 | 175 / 264 | 230 / 266 | | |
| | `rtmpose-l-384` | 1080x1920 | 19.3 ms | 10.7 ms | 7.08 ms (network on person crops only) | 11.2 ms | 15.0 s | 20.3 s | 340 / 436 | 242 / 274 | 229 / 406 | 297 / 322 | | |
| | `rtmpose-l-halpe26` | 1080x1920 | 9.06 ms | 5.34 ms | 3.20 ms (network on person crops only) | 5.65 ms | 14.3 s | 19.6 s | 212 / 276 | 186 / 194 | 176 / 264 | 231 / 266 | | |
| | `rtmpose-l-halpe26-384` | 1080x1920 | 19.3 ms | 10.6 ms | 7.08 ms (network on person crops only) | 11.2 ms | 15.1 s | 20.6 s | 341 / 436 | 243 / 276 | 230 / 406 | 299 / 322 | | |
| | `rtmpose-m` | 1080x1920 | 6.28 ms | 3.42 ms | 2.02 ms (network on person crops only) | 4.42 ms | 12.7 s | 17.8 s | 153 / 192 | 147 / 172 | 148 / 200 | 157 / 176 | | |
| | `rtmpose-m-384` | 1080x1920 | 13.1 ms | 7.09 ms | 4.33 ms (network on person crops only) | 8.97 ms | 13.6 s | 18.6 s | 243 / 352 | 190 / 212 | 202 / 340 | 212 / 242 | | |
| | `rtmpose-m-ap10k` | 1080x1920 | 7.98 ms | 4.54 ms | 2.57 ms (network on person crops only) | 5.41 ms | 13.0 s | 18.2 s | 177 / 234 | 176 / 192 | 163 / 220 | 172 / 188 | | |
| | `rtmpose-m-face6` | 1080x1920 | 8.69 ms | 4.79 ms | 2.67 ms (network on person crops only) | 5.78 ms | 12.8 s | 18.4 s | 184 / 256 | 182 / 192 | 169 / 242 | 216 / 240 | | |
| | `rtmpose-m-halpe26` | 1080x1920 | 6.35 ms | 3.44 ms | 2.02 ms (network on person crops only) | 4.46 ms | 12.9 s | 18.0 s | 154 / 192 | 148 / 172 | 149 / 220 | 160 / 180 | | |
| | `rtmpose-m-halpe26-384` | 1080x1920 | 13.2 ms | 7.12 ms | 4.34 ms (network on person crops only) | 9.01 ms | 13.7 s | 18.9 s | 244 / 352 | 190 / 212 | 203 / 340 | 212 / 238 | | |
| | `rtmpose-m-hand5` | 1080x1920 | 8.04 ms | 4.51 ms | 2.58 ms (network on person crops only) | 5.44 ms | 12.9 s | 18.3 s | 178 / 234 | 176 / 192 | 163 / 240 | 174 / 192 | | |
| | `rtmpose-s` | 1080x1920 | 4.92 ms | 2.07 ms | 1.06 ms (network on person crops only) | 3.01 ms | 11.5 s | 16.5 s | 124 / 172 | 129 / 150 | 133 / 178 | 137 / 170 | | |
| | `rtmpose-s-face6` | 1080x1920 | 5.03 ms | 2.69 ms | 1.28 ms (network on person crops only) | 3.75 ms | 11.9 s | 16.9 s | 147 / 174 | 164 / 190 | 151 / 222 | 158 / 178 | | |
| | `rtmpose-s-halpe26` | 1080x1920 | 4.98 ms | 2.11 ms | 1.07 ms (network on person crops only) | 3.04 ms | 11.7 s | 16.6 s | 124 / 172 | 130 / 150 | 133 / 198 | 138 / 170 | | |
| | `rtmpose-t` | 1080x1920 | 4.49 ms | 1.91 ms | 0.866 ms (network on person crops only) | 2.90 ms | 10.8 s | 15.9 s | 120 / 150 | 125 / 128 | 129 / 176 | 127 / 164 | | |
| | `rtmpose-t-face6` | 1080x1920 | 4.58 ms | 2.23 ms | 1.00 ms (network on person crops only) | 3.63 ms | 11.5 s | 16.2 s | 134 / 152 | 159 / 168 | 146 / 220 | 145 / 168 | | |
| | `rtmpose-t-halpe26` | 1080x1920 | 4.48 ms | 1.94 ms | 0.871 ms (network on person crops only) | 2.90 ms | 13.5 s | 17.2 s | 120 / 150 | 125 / 128 | 129 / 196 | 128 / 164 | | |
| | `rtmpose-x-halpe26-384` | 1080x1920 | 33.5 ms | 21.1 ms | 13.3 ms (network on person crops only) | 18.7 ms | 17.4 s | 23.0 s | 401 / 498 | 262 / 298 | 280 / 470 | 430 / 456 | | |
| | `rtmw-l` | 1080x1920 | 15.1 ms | 9.34 ms | 5.17 ms (network on person crops only) | 10.2 ms | 18.8 s | 24.9 s | 267 / 308 | 242 / 264 | 232 / 336 | 354 / 396 | | |
| | `rtmw-l-384` | 1080x1920 | 31.5 ms | 19.5 ms | 11.2 ms (network on person crops only) | 20.2 ms | 19.8 s | 26.8 s | 396 / 508 | 298 / 344 | 286 / 476 | 424 / 452 | | |
| | `rtmw-m` | 1080x1920 | 11.2 ms | 6.23 ms | 3.34 ms (network on person crops only) | 7.54 ms | 18.7 s | 23.1 s | 189 / 224 | 183 / 202 | 184 / 268 | 231 / 256 | | |
| | `rtmw-x` | 1080x1920 | 23.9 ms | 15.3 ms | 8.47 ms (network on person crops only) | 14.3 ms | 20.4 s | 27.0 s | 342 / 392 | 320 / 328 | 298 / 394 | 497 / 514 | | |
| | `rtmw-x-384` | 1080x1920 | 50.6 ms | 34.0 ms | 19.3 ms (network on person crops only) | 50.8 ms | 21.2 s | 29.8 s | 482 / 592 | 361 / 388 | 496 / 674 | 581 / 624 | | |
| Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB). | |
| Hardware, protocol and caveats: [benchmarks](https://github.com/kleinicke/fastforks/blob/main/docs/benchmarks.md) (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or `(fp32)` where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's. | |
| ## Accuracy | |
| ### On public datasets | |
| Top-down on the same every-tenth-image subset for all modes, ground-truth person boxes, no flip test; people ranked by the mean of their first 17 keypoint scores; pycocotools OKS AP on COCO body joints. Top-down on the ground-truth person boxes (non-crowd, at least one labelled keypoint), no flip test; people ranked by the mean of their 17 keypoint scores (not mmpose's rescoring and OKS-NMS), so not the papers' AP; pycocotools OKS AP; whole-body models scored on their 17 body keypoints. Original = the original code named under Details, same checkpoint and protocol. | |
| | variant | dataset | original fp32 | original fp16 | portable | kernels | note | | |
| |---|---|---|---|---|---|---| | |
| | `rtmpose-t` | COCO val2017, ground-truth person boxes, keypoint AP | 65.2 | 65.1 | 65.2 | 65.2 | | | |
| | `rtmpose-s` | COCO val2017, ground-truth person boxes, keypoint AP | 69.8 | 69.8 | 69.8 | 69.8 | | | |
| | `rtmpose-m` | COCO val2017, ground-truth person boxes, keypoint AP | 75.6 | 75.6 | 75.6 | 75.6 | | | |
| | `rtmpose-l` | COCO val2017, ground-truth person boxes, keypoint AP | 77.9 | 77.9 | 77.9 | 78.0 | | | |
| | `rtmpose-m-384` | COCO val2017, ground-truth person boxes, keypoint AP | 78.0 | 77.9 | 77.9 | 78.0 | | | |
| | `rtmpose-l-384` | COCO val2017, ground-truth person boxes, keypoint AP | 80.0 | 80.0 | 80.0 | 79.9 | | | |
| | `dwpose-t` | COCO val2017, ground-truth person boxes, keypoint AP | 60.2 | 60.2 | 60.1 | 60.1 | | | |
| | `dwpose-s` | COCO val2017, ground-truth person boxes, keypoint AP | 65.6 | 65.6 | 65.6 | 65.5 | | | |
| | `dwpose-m` | COCO val2017, ground-truth person boxes, keypoint AP | 71.7 | 71.8 | 71.8 | 71.8 | | | |
| | `dwpose-l` | COCO val2017, ground-truth person boxes, keypoint AP | 73.9 | 73.9 | 74.0 | 74.0 | | | |
| | `dwpose-l-384` | COCO val2017, ground-truth person boxes, keypoint AP | 75.6 | 75.6 | 75.7 | 75.6 | | | |
| | `rtmw-m` | COCO val2017, ground-truth person boxes, keypoint AP | 74.5 | 74.5 | 74.5 | 74.6 | | | |
| | `rtmw-l` | COCO val2017, ground-truth person boxes, keypoint AP | 76.9 | 76.9 | 77.0 | 76.9 | | | |
| | `rtmw-x` | COCO val2017, ground-truth person boxes, keypoint AP | 77.3 | 77.4 | 77.3 | 77.4 | | | |
| | `rtmw-l-384` | COCO val2017, every 10th image, GT person boxes, keypoint AP | 80.2 | 80.3 | 80.2 | 80.1 | | | |
| | `rtmw-l-384` | COCO val2017, ground-truth person boxes, keypoint AP | 79.4 | 79.4 | 79.5 | 79.5 | | | |
| | `rtmw-x-384` | COCO val2017, every 10th image, GT person boxes, keypoint AP | 80.3 | 80.2 | 80.1 | 80.4 | | | |
| | `rtmw-x-384` | COCO val2017, ground-truth person boxes, keypoint AP | 79.6 | 79.6 | 79.6 | 79.6 | | | |
| | `rtmpose-t-halpe26` | COCO val2017, ground-truth person boxes, keypoint AP | 65.3 | 65.5 | 65.3 | 65.3 | | | |
| | `rtmpose-s-halpe26` | COCO val2017, every 10th image, GT person boxes, keypoint AP | 71.4 | 71.3 | 71.4 | 71.2 | | | |
| | `rtmpose-s-halpe26` | COCO val2017, ground-truth person boxes, keypoint AP | 70.0 | 70.0 | 69.9 | 70.0 | | | |
| | `rtmpose-m-halpe26` | COCO val2017, ground-truth person boxes, keypoint AP | 76.3 | 76.2 | 76.2 | 76.2 | | | |
| | `rtmpose-l-halpe26` | COCO val2017, ground-truth person boxes, keypoint AP | 78.5 | 78.5 | 78.5 | 78.5 | | | |
| | `rtmpose-m-halpe26-384` | COCO val2017, ground-truth person boxes, keypoint AP | 78.3 | 78.4 | 78.3 | 78.3 | | | |
| | `rtmpose-l-halpe26-384` | COCO val2017, ground-truth person boxes, keypoint AP | 79.8 | 79.8 | 79.8 | 80.0 | | | |
| | `rtmpose-x-halpe26-384` | COCO val2017, ground-truth person boxes, keypoint AP | 80.5 | 80.6 | 80.5 | 80.4 | | | |
| Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 2241b095+uncommitted / be54b8b0, 2026-11-09 to 2026-11-11. | |
| ### Parity with the original code | |
| Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.66 and p99 <= 3.6 (absolute error of the output `keypoints`). | |
| ## Usage | |
| ```python | |
| from fastforks import pose | |
| model = pose.load("dwpose-l") # mode="auto": the preferred backend from recorded measurements | |
| result = model("photo.jpg", boxes=[[x0, y0, x1, y1]]) # one box per person, or a fastforks.detection result | |
| result.keypoints, result.keypoint_scores # [P, K, 2] input pixels, [P, K] | |
| result.backend # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16') | |
| ``` | |
| Install with `pip install fastforks` (see <https://github.com/kleinicke/fastforks>). Other variants: `dwpose-l`, `dwpose-l-384`, `dwpose-m`, `dwpose-s`, `dwpose-t`, `rtmpose-l`, `rtmpose-l-384`, `rtmpose-l-halpe26`, `rtmpose-l-halpe26-384`, `rtmpose-m`, `rtmpose-m-384`, `rtmpose-m-ap10k`, `rtmpose-m-face6`, `rtmpose-m-halpe26`, `rtmpose-m-halpe26-384`, `rtmpose-m-hand5`, `rtmpose-s`, `rtmpose-s-face6`, `rtmpose-s-halpe26`, `rtmpose-t`, `rtmpose-t-face6`, `rtmpose-t-halpe26`, `rtmpose-x-halpe26-384`, `rtmw-l`, `rtmw-l-384`, `rtmw-m`, `rtmw-x`, `rtmw-x-384`. | |
| ## Kernels backend | |
| `pose.load("dwpose-l", mode="kernels")` runs our CUDA kernels (`cudnn_hacc`, `fvm_conv`, `fvm_extra`, `fvm_mb`, `fvm_r3`). It needs a prebuilt kernel for your GPU architecture (downloaded from <https://huggingface.co/fastforks/kernels> on first use) and a recent NVIDIA driver; if there is none it raises `BackendUnavailableError` with the reason and never falls back silently. `mode="auto"` picks `kernels` when it can run and logs the choice. | |
| <!-- staged from fastforks 1cbf1671 (2026-11-15) --> | |