rtmpose / README.md
Flor's picture
card: update from fastforks 1cbf1671
1b9e815 verified
|
Raw History Blame Contribute Delete
26.9 kB
metadata
library_name: fastforks
pipeline_tag: keypoint-detection
license: apache-2.0
tags:
  - pose
  - pose-estimation
  - human-pose-estimation
  - keypoint-detection
  - cuda
  - pytorch
  - fastforks
  - fastforks-card

RTMPose (fastforks)

RTMPose (Jiang et al. 2023): real-time top-down 2D human pose from OpenMMLab: a CSPNeXt CNN backbone and a SimCC head (a gated attention unit over one token per keypoint, then x and y classified as 1D bins at half-pixel resolution, no heatmaps). The body7 checkpoints (trained on seven public pose datasets) return the 17 COCO keypoints; the DWPose ones (Yang et al. 2023, distilled on COCO-WholeBody + UBody, the pose model of ControlNet's openpose preprocessors) the 133 COCO-WholeBody keypoints (body, feet, face, both hands). The RTMW ones (Jiang et al. 2024: a PAFPN neck and a two-scale head on the same backbone, trained on 14 datasets) return the same 133 keypoints more accurately. More model-zoo checkpoints of the same network: -halpe26 (body + feet, 26 keypoints), -hand5 (21 keypoints of a hand box), -face6 (106 face landmarks of a face box) and -ap10k (17 keypoints of an animal box). Takes person boxes (e.g. from fastforks.detection) and crops each at 256 x 192 (or 384 x 288; hand, face and animal boxes: 256 x 256); vendored from mmpose (no mmpose / mmcv dependency). The portable module() maps crops to SimCC logits.

This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: https://github.com/kleinicke/fastforks. Kernels for the kernels backend: https://huggingface.co/fastforks/kernels. Original code: https://github.com/open-mmlab/mmpose at 5408bc76f5b8.

Task: pose. Tier: stable. Outputs: keypoints, keypoint_scores, boxes, scores. Backends: portable, kernels.

Input: one RGB image and a box per person. Output: keypoints with confidence scores for every person.

Variants

Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in card.toml.

variant weights (author link) SHA-256 weights licence commercial use
dwpose-l@1 dwpose-l-ucoco-256x192.pth 4d6dfc622672 Apache-2.0 yes
dwpose-l-384@1 dwpose-l-ucoco-384x288.pth 2438fd992e65 Apache-2.0 yes
dwpose-m@1 dwpose-m-ucoco-256x192.pth c8b7641988ce Apache-2.0 yes
dwpose-s@1 dwpose-s-ucoco-256x192.pth 3fd922c87ac2 Apache-2.0 yes
dwpose-t@1 dwpose-t-ucoco-256x192.pth dcf277bf6ba4 Apache-2.0 yes
rtmpose-l@1 rtmpose-l-body7-256x192.pth 4dba18fc4632 Apache-2.0 yes
rtmpose-l-384@1 rtmpose-l-body7-384x288.pth 3f5a14373b19 Apache-2.0 yes
rtmpose-l-halpe26@1 rtmpose-l-halpe26-256x192.pth 2abb7558f5bf Apache-2.0 yes
rtmpose-l-halpe26-384@1 rtmpose-l-halpe26-384x288.pth 734182ce2409 Apache-2.0 yes
rtmpose-m@1 rtmpose-m-body7-256x192.pth e48f03d0cfe1 Apache-2.0 yes
rtmpose-m-384@1 rtmpose-m-body7-384x288.pth 65e718c4ccb6 Apache-2.0 yes
rtmpose-m-ap10k@1 rtmpose-m-ap10k-256x256.pth 896e3665d849 Apache-2.0 yes
rtmpose-m-face6@1 rtmpose-m-face6-256x256.pth 72a37400a409 Apache-2.0 yes
rtmpose-m-halpe26@1 rtmpose-m-halpe26-256x192.pth 4d3e73ddd312 Apache-2.0 yes
rtmpose-m-halpe26-384@1 rtmpose-m-halpe26-384x288.pth 89e6428b3890 Apache-2.0 yes
rtmpose-m-hand5@1 rtmpose-m-hand5-256x256.pth b74fb5941684 Apache-2.0 yes
rtmpose-s@1 rtmpose-s-body7-256x192.pth acd4a1efd8f0 Apache-2.0 yes
rtmpose-s-face6@1 rtmpose-s-face6-256x256.pth d779fdefe9e7 Apache-2.0 yes
rtmpose-s-halpe26@1 rtmpose-s-halpe26-256x192.pth 7f1341650e93 Apache-2.0 yes
rtmpose-t@1 rtmpose-t-body7-256x192.pth 026a1439c569 Apache-2.0 yes
rtmpose-t-face6@1 rtmpose-t-face6-256x256.pth df79d9a59eee Apache-2.0 yes
rtmpose-t-halpe26@1 rtmpose-t-halpe26-256x192.pth 6020f8a67466 Apache-2.0 yes
rtmpose-x-halpe26-384@1 rtmpose-x-halpe26-384x288.pth 7fb6e2396010 Apache-2.0 yes
rtmw-l@1 rtmw-l-cocktail14-256x192.pth ae6459e54e52 Apache-2.0 yes
rtmw-l-384@1 rtmw-l-cocktail14-384x288.pth afa589fa16bf Apache-2.0 yes
rtmw-m@1 rtmw-m-cocktail14-256x192.pth 6c481e4f9d3b Apache-2.0 yes
rtmw-x@1 rtmw-x-cocktail14-256x192.pth 13a2546d5978 Apache-2.0 yes
rtmw-x-384@1 rtmw-x-cocktail14-384x288.pth f840f2044fe4 Apache-2.0 yes

Licence

Original-code strings, recorded as they are (research-use package; check them before any other use). Code: Apache-2.0. Licence files: LICENSE.

Training data: COCO 2017 (train, instance annotations), AI Challenger human keypoints (2017), CrowdPose, MPII Human Pose, sub-JHMDB (J-HMDB), Halpe-FullBody, PoseTrack 2018, ImageNet-1k (ILSVRC 2012 classification) (body7: COCO, AI Challenger, CrowdPose, MPII, sub-JHMDB, Halpe and PoseTrack18 keypoints; the CSPNeXt backbone was pre-trained on ImageNet-1k (and on the same pose data, pt-body7)).

Licence evidence: https://github.com/open-mmlab/mmpose/blob/5408bc76f5b848cf925a0d1857899011d8c5b497/README.md#license

Input size

any H x W >= 32, padded to a multiple of 32 Tested up to (portable, fp16, RTX 5060 Ti 16 GB): dwpose-l 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); dwpose-l-384 3000x4000 (0.6 GiB peak allocated, 1.0 reserved); dwpose-m 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); dwpose-s 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); dwpose-t 3000x4000 (0.4 GiB peak allocated, 0.5 reserved); rtmpose-l 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); rtmpose-l-384 3000x4000 (0.6 GiB peak allocated, 1.0 reserved); rtmpose-l-halpe26 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); rtmpose-l-halpe26-384 3000x4000 (0.6 GiB peak allocated, 1.0 reserved); rtmpose-m 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); rtmpose-m-384 3000x4000 (0.5 GiB peak allocated, 0.6 reserved); rtmpose-m-ap10k 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); rtmpose-m-face6 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); rtmpose-m-halpe26 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); rtmpose-m-halpe26-384 3000x4000 (0.5 GiB peak allocated, 0.6 reserved); rtmpose-m-hand5 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); rtmpose-s 3000x4000 (0.4 GiB peak allocated, 0.5 reserved); rtmpose-s-face6 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); rtmpose-s-halpe26 3000x4000 (0.4 GiB peak allocated, 0.5 reserved); rtmpose-t 3000x4000 (0.4 GiB peak allocated, 0.5 reserved); rtmpose-t-face6 3000x4000 (0.4 GiB peak allocated, 0.5 reserved); rtmpose-t-halpe26 3000x4000 (0.4 GiB peak allocated, 0.5 reserved); rtmpose-x-halpe26-384 3000x4000 (0.8 GiB peak allocated, 0.9 reserved); rtmw-l 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); rtmw-l-384 3000x4000 (0.7 GiB peak allocated, 1.0 reserved); rtmw-m 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); rtmw-x 3000x4000 (0.6 GiB peak allocated, 0.7 reserved); rtmw-x-384 3000x4000 (0.8 GiB peak allocated, 1.0 reserved)

Speed and memory

Pose, synthetic, 1080x1920

model time (ms): original fp32 original portable kernels kernels speed-up: vs original vs portable peak memory (MiB, allocated / reserved): original portable kernels
dwpose-l 178 184 10.2 6.20 28.7x 1.7x 393 / 428 222 / 280 186 / 306
dwpose-l-384 194 203 21.2 12.5 15.5x 1.7x 773 / 816 351 / 460 240 / 424
dwpose-m 169 164 7.25 4.86 33.8x 1.5x 756 / 792 162 / 196 157 / 220
dwpose-s 173 180 5.03 3.30 52.3x 1.5x 806 / 848 129 / 174 138 / 198
dwpose-t 160 174 4.47 3.15 50.8x 1.4x 832 / 858 124 / 152 133 / 192
rtmpose-l 161 176 8.94 5.63 28.6x 1.6x 1105 / 1154 211 / 236 175 / 264
rtmpose-l-384 192 171 19.3 11.2 15.3x 1.7x 366 / 410 340 / 436 229 / 406
rtmpose-l-halpe26 180 164 9.06 5.65 29.1x 1.6x 529 / 572 212 / 276 176 / 264
rtmpose-l-halpe26-384 206 206 19.3 11.2 18.4x 1.7x 975 / 1038 341 / 436 230 / 406
rtmpose-m 179 180 6.28 4.42 40.4x 1.4x 316 / 364 153 / 192 148 / 200
rtmpose-m-384 174 167 13.1 8.97 18.6x 1.5x 505 / 582 243 / 352 202 / 340
rtmpose-m-ap10k 196 186 7.98 5.41 34.5x 1.5x 757 / 814 177 / 234 163 / 220
rtmpose-m-face6 198 207 8.69 5.78 34.3x 1.5x 652 / 698 184 / 256 169 / 242
rtmpose-m-halpe26 179 181 6.35 4.46 40.2x 1.4x 283 / 318 154 / 192 149 / 220
rtmpose-m-halpe26-384 192 193 13.2 9.01 21.3x 1.5x 690 / 764 244 / 352 203 / 340
rtmpose-m-hand5 192 195 8.04 5.44 35.3x 1.5x 425 / 472 178 / 234 163 / 240
rtmpose-s 161 165 4.92 3.01 53.4x 1.6x 449 / 480 124 / 172 133 / 178
rtmpose-s-face6 258 237 5.03 3.75 63.2x 1.3x 504 / 540 147 / 174 151 / 222
rtmpose-s-halpe26 172 173 4.98 3.04 56.5x 1.6x 162 / 200 124 / 172 133 / 198
rtmpose-t 165 151 4.49 2.90 52.0x 1.5x 459 / 490 120 / 150 129 / 176
rtmpose-t-face6 188 204 4.58 3.63 51.8x 1.3x 424 / 464 134 / 152 146 / 220
rtmpose-t-halpe26 181 172 4.48 2.90 59.1x 1.5x 103 / 144 120 / 150 129 / 196
rtmpose-x-halpe26-384 247 217 33.5 18.7 11.6x 1.8x 470 / 556 401 / 498 280 / 470
rtmw-l 213 203 15.1 10.2 19.8x 1.5x 865 / 908 267 / 308 232 / 336
rtmw-l-384 274 275 31.5 20.2 13.6x 1.6x 1094 / 1144 396 / 508 286 / 476
rtmw-m 169 205 11.2 7.54 22.5x 1.5x 379 / 410 189 / 224 184 / 268
rtmw-x 218 207 23.9 14.3 14.5x 1.7x 604 / 636 342 / 392 298 / 394
rtmw-x-384 306 291 50.6 50.8 5.7x 1.0x 1874 / 1940 482 / 592 496 / 674

Startup and first call

model load (s): original portable kernels first call (s): original portable kernels
dwpose-l 0.50 0.88 1.64 1.14 1.83 0.28
dwpose-l-384 0.42 0.58 1.67 0.42 1.10 0.28
dwpose-m 0.26 0.24 1.51 0.60 1.10 0.29
dwpose-s 0.19 0.12 1.48 0.47 1.02 0.29
dwpose-t 0.16 0.08 1.42 0.33 0.89 0.29
rtmpose-l 0.38 0.94 1.60 0.18 0.86 0.29
rtmpose-l-384 0.58 0.41 1.56 0.17 1.00 0.29
rtmpose-l-halpe26 0.37 0.25 1.55 0.45 1.13 0.29
rtmpose-l-halpe26-384 0.41 0.26 1.57 0.38 1.24 0.29
rtmpose-m 0.23 0.22 1.50 0.16 1.10 0.29
rtmpose-m-384 0.23 0.20 1.48 0.35 1.25 0.29
rtmpose-m-ap10k 0.23 0.13 1.48 0.19 1.14 0.29
rtmpose-m-face6 0.29 0.15 1.49 0.20 1.15 0.29
rtmpose-m-halpe26 0.25 0.14 1.51 0.45 1.04 0.30
rtmpose-m-halpe26-384 0.26 0.13 1.48 0.34 1.15 0.29
rtmpose-m-hand5 0.23 0.13 1.49 0.22 1.13 0.29
rtmpose-s 0.16 0.10 1.40 0.16 1.03 0.29
rtmpose-s-face6 0.31 0.08 1.43 0.35 1.18 0.29
rtmpose-s-halpe26 0.19 0.07 1.46 0.61 0.94 0.30
rtmpose-t 0.13 0.10 1.42 0.15 0.88 0.29
rtmpose-t-face6 0.18 0.06 1.45 0.34 1.29 0.29
rtmpose-t-halpe26 0.18 0.05 1.48 0.71 0.81 0.29
rtmpose-x-halpe26-384 0.89 0.41 1.76 0.94 1.17 0.28
rtmw-l 0.76 0.39 1.72 0.63 1.03 0.31
rtmw-l-384 0.75 0.34 1.69 0.56 1.20 0.28
rtmw-m 0.54 0.35 1.64 1.21 1.20 0.29
rtmw-x 1.23 1.35 2.10 0.64 0.83 0.28
rtmw-x-384 1.05 0.81 2.20 0.42 0.96 0.34

torch.compile and TensorRT

model input time: portable compile TensorRT fp16 kernels preparation: compile (cold first call) ONNX + engine build peak MiB (alloc / reserved): portable compile kernels TensorRT (engine + torch)
dwpose-l 1080x1920 10.2 ms 5.75 ms 3.33 ms (network on person crops only) 6.20 ms 14.8 s 20.0 s 222 / 280 197 / 216 186 / 306 277 / 294
dwpose-l-384 1080x1920 21.2 ms 11.9 ms 7.46 ms (network on person crops only) 12.5 ms 14.9 s 20.6 s 351 / 460 253 / 296 240 / 424 347 / 370
dwpose-m 1080x1920 7.25 ms 3.80 ms 2.13 ms (network on person crops only) 4.86 ms 12.9 s 18.9 s 162 / 196 156 / 174 157 / 220 203 / 228
dwpose-s 1080x1920 5.03 ms 2.27 ms 1.17 ms (network on person crops only) 3.30 ms 11.8 s 16.7 s 129 / 174 135 / 150 138 / 198 149 / 170
dwpose-t 1080x1920 4.47 ms 1.96 ms 0.940 ms (network on person crops only) 3.15 ms 11.2 s 16.1 s 124 / 152 129 / 148 133 / 192 136 / 160
rtmpose-l 1080x1920 8.94 ms 5.32 ms 3.20 ms (network on person crops only) 5.63 ms 14.2 s 19.4 s 211 / 236 185 / 194 175 / 264 230 / 266
rtmpose-l-384 1080x1920 19.3 ms 10.7 ms 7.08 ms (network on person crops only) 11.2 ms 15.0 s 20.3 s 340 / 436 242 / 274 229 / 406 297 / 322
rtmpose-l-halpe26 1080x1920 9.06 ms 5.34 ms 3.20 ms (network on person crops only) 5.65 ms 14.3 s 19.6 s 212 / 276 186 / 194 176 / 264 231 / 266
rtmpose-l-halpe26-384 1080x1920 19.3 ms 10.6 ms 7.08 ms (network on person crops only) 11.2 ms 15.1 s 20.6 s 341 / 436 243 / 276 230 / 406 299 / 322
rtmpose-m 1080x1920 6.28 ms 3.42 ms 2.02 ms (network on person crops only) 4.42 ms 12.7 s 17.8 s 153 / 192 147 / 172 148 / 200 157 / 176
rtmpose-m-384 1080x1920 13.1 ms 7.09 ms 4.33 ms (network on person crops only) 8.97 ms 13.6 s 18.6 s 243 / 352 190 / 212 202 / 340 212 / 242
rtmpose-m-ap10k 1080x1920 7.98 ms 4.54 ms 2.57 ms (network on person crops only) 5.41 ms 13.0 s 18.2 s 177 / 234 176 / 192 163 / 220 172 / 188
rtmpose-m-face6 1080x1920 8.69 ms 4.79 ms 2.67 ms (network on person crops only) 5.78 ms 12.8 s 18.4 s 184 / 256 182 / 192 169 / 242 216 / 240
rtmpose-m-halpe26 1080x1920 6.35 ms 3.44 ms 2.02 ms (network on person crops only) 4.46 ms 12.9 s 18.0 s 154 / 192 148 / 172 149 / 220 160 / 180
rtmpose-m-halpe26-384 1080x1920 13.2 ms 7.12 ms 4.34 ms (network on person crops only) 9.01 ms 13.7 s 18.9 s 244 / 352 190 / 212 203 / 340 212 / 238
rtmpose-m-hand5 1080x1920 8.04 ms 4.51 ms 2.58 ms (network on person crops only) 5.44 ms 12.9 s 18.3 s 178 / 234 176 / 192 163 / 240 174 / 192
rtmpose-s 1080x1920 4.92 ms 2.07 ms 1.06 ms (network on person crops only) 3.01 ms 11.5 s 16.5 s 124 / 172 129 / 150 133 / 178 137 / 170
rtmpose-s-face6 1080x1920 5.03 ms 2.69 ms 1.28 ms (network on person crops only) 3.75 ms 11.9 s 16.9 s 147 / 174 164 / 190 151 / 222 158 / 178
rtmpose-s-halpe26 1080x1920 4.98 ms 2.11 ms 1.07 ms (network on person crops only) 3.04 ms 11.7 s 16.6 s 124 / 172 130 / 150 133 / 198 138 / 170
rtmpose-t 1080x1920 4.49 ms 1.91 ms 0.866 ms (network on person crops only) 2.90 ms 10.8 s 15.9 s 120 / 150 125 / 128 129 / 176 127 / 164
rtmpose-t-face6 1080x1920 4.58 ms 2.23 ms 1.00 ms (network on person crops only) 3.63 ms 11.5 s 16.2 s 134 / 152 159 / 168 146 / 220 145 / 168
rtmpose-t-halpe26 1080x1920 4.48 ms 1.94 ms 0.871 ms (network on person crops only) 2.90 ms 13.5 s 17.2 s 120 / 150 125 / 128 129 / 196 128 / 164
rtmpose-x-halpe26-384 1080x1920 33.5 ms 21.1 ms 13.3 ms (network on person crops only) 18.7 ms 17.4 s 23.0 s 401 / 498 262 / 298 280 / 470 430 / 456
rtmw-l 1080x1920 15.1 ms 9.34 ms 5.17 ms (network on person crops only) 10.2 ms 18.8 s 24.9 s 267 / 308 242 / 264 232 / 336 354 / 396
rtmw-l-384 1080x1920 31.5 ms 19.5 ms 11.2 ms (network on person crops only) 20.2 ms 19.8 s 26.8 s 396 / 508 298 / 344 286 / 476 424 / 452
rtmw-m 1080x1920 11.2 ms 6.23 ms 3.34 ms (network on person crops only) 7.54 ms 18.7 s 23.1 s 189 / 224 183 / 202 184 / 268 231 / 256
rtmw-x 1080x1920 23.9 ms 15.3 ms 8.47 ms (network on person crops only) 14.3 ms 20.4 s 27.0 s 342 / 392 320 / 328 298 / 394 497 / 514
rtmw-x-384 1080x1920 50.6 ms 34.0 ms 19.3 ms (network on person crops only) 50.8 ms 21.2 s 29.8 s 482 / 592 361 / 388 496 / 674 581 / 624

Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).

Hardware, protocol and caveats: benchmarks (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or (fp32) where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.

Accuracy

On public datasets

Top-down on the same every-tenth-image subset for all modes, ground-truth person boxes, no flip test; people ranked by the mean of their first 17 keypoint scores; pycocotools OKS AP on COCO body joints. Top-down on the ground-truth person boxes (non-crowd, at least one labelled keypoint), no flip test; people ranked by the mean of their 17 keypoint scores (not mmpose's rescoring and OKS-NMS), so not the papers' AP; pycocotools OKS AP; whole-body models scored on their 17 body keypoints. Original = the original code named under Details, same checkpoint and protocol.

variant dataset original fp32 original fp16 portable kernels note
rtmpose-t COCO val2017, ground-truth person boxes, keypoint AP 65.2 65.1 65.2 65.2
rtmpose-s COCO val2017, ground-truth person boxes, keypoint AP 69.8 69.8 69.8 69.8
rtmpose-m COCO val2017, ground-truth person boxes, keypoint AP 75.6 75.6 75.6 75.6
rtmpose-l COCO val2017, ground-truth person boxes, keypoint AP 77.9 77.9 77.9 78.0
rtmpose-m-384 COCO val2017, ground-truth person boxes, keypoint AP 78.0 77.9 77.9 78.0
rtmpose-l-384 COCO val2017, ground-truth person boxes, keypoint AP 80.0 80.0 80.0 79.9
dwpose-t COCO val2017, ground-truth person boxes, keypoint AP 60.2 60.2 60.1 60.1
dwpose-s COCO val2017, ground-truth person boxes, keypoint AP 65.6 65.6 65.6 65.5
dwpose-m COCO val2017, ground-truth person boxes, keypoint AP 71.7 71.8 71.8 71.8
dwpose-l COCO val2017, ground-truth person boxes, keypoint AP 73.9 73.9 74.0 74.0
dwpose-l-384 COCO val2017, ground-truth person boxes, keypoint AP 75.6 75.6 75.7 75.6
rtmw-m COCO val2017, ground-truth person boxes, keypoint AP 74.5 74.5 74.5 74.6
rtmw-l COCO val2017, ground-truth person boxes, keypoint AP 76.9 76.9 77.0 76.9
rtmw-x COCO val2017, ground-truth person boxes, keypoint AP 77.3 77.4 77.3 77.4
rtmw-l-384 COCO val2017, every 10th image, GT person boxes, keypoint AP 80.2 80.3 80.2 80.1
rtmw-l-384 COCO val2017, ground-truth person boxes, keypoint AP 79.4 79.4 79.5 79.5
rtmw-x-384 COCO val2017, every 10th image, GT person boxes, keypoint AP 80.3 80.2 80.1 80.4
rtmw-x-384 COCO val2017, ground-truth person boxes, keypoint AP 79.6 79.6 79.6 79.6
rtmpose-t-halpe26 COCO val2017, ground-truth person boxes, keypoint AP 65.3 65.5 65.3 65.3
rtmpose-s-halpe26 COCO val2017, every 10th image, GT person boxes, keypoint AP 71.4 71.3 71.4 71.2
rtmpose-s-halpe26 COCO val2017, ground-truth person boxes, keypoint AP 70.0 70.0 69.9 70.0
rtmpose-m-halpe26 COCO val2017, ground-truth person boxes, keypoint AP 76.3 76.2 76.2 76.2
rtmpose-l-halpe26 COCO val2017, ground-truth person boxes, keypoint AP 78.5 78.5 78.5 78.5
rtmpose-m-halpe26-384 COCO val2017, ground-truth person boxes, keypoint AP 78.3 78.4 78.3 78.3
rtmpose-l-halpe26-384 COCO val2017, ground-truth person boxes, keypoint AP 79.8 79.8 79.8 80.0
rtmpose-x-halpe26-384 COCO val2017, ground-truth person boxes, keypoint AP 80.5 80.6 80.5 80.4

Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 2241b095+uncommitted / be54b8b0, 2026-11-09 to 2026-11-11.

Parity with the original code

Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.66 and p99 <= 3.6 (absolute error of the output keypoints).

Usage

from fastforks import pose

model = pose.load("dwpose-l")  # mode="auto": the preferred backend from recorded measurements
result = model("photo.jpg", boxes=[[x0, y0, x1, y1]])  # one box per person, or a fastforks.detection result
result.keypoints, result.keypoint_scores  # [P, K, 2] input pixels, [P, K]
result.backend  # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')

Install with pip install fastforks (see https://github.com/kleinicke/fastforks). Other variants: dwpose-l, dwpose-l-384, dwpose-m, dwpose-s, dwpose-t, rtmpose-l, rtmpose-l-384, rtmpose-l-halpe26, rtmpose-l-halpe26-384, rtmpose-m, rtmpose-m-384, rtmpose-m-ap10k, rtmpose-m-face6, rtmpose-m-halpe26, rtmpose-m-halpe26-384, rtmpose-m-hand5, rtmpose-s, rtmpose-s-face6, rtmpose-s-halpe26, rtmpose-t, rtmpose-t-face6, rtmpose-t-halpe26, rtmpose-x-halpe26-384, rtmw-l, rtmw-l-384, rtmw-m, rtmw-x, rtmw-x-384.

Kernels backend

pose.load("dwpose-l", mode="kernels") runs our CUDA kernels (cudnn_hacc, fvm_conv, fvm_extra, fvm_mb, fvm_r3). It needs a prebuilt kernel for your GPU architecture (downloaded from https://huggingface.co/fastforks/kernels on first use) and a recent NVIDIA driver; if there is none it raises BackendUnavailableError with the reason and never falls back silently. mode="auto" picks kernels when it can run and logs the choice.