Download README.md from fastforks/vitpose: direct link, hf CLI and curl.
- Browser
- Download file 10.3 kB
-
https://huggingface.co/fastforks/vitpose/resolve/main/README.md
- Command line
-
hf download hf://fastforks/vitpose/README.md
-
curl -L -o README.md https://huggingface.co/fastforks/vitpose/resolve/main/README.md
library_name: fastforks
pipeline_tag: keypoint-detection
license: apache-2.0
tags:
- pose
- pose-estimation
- human-pose-estimation
- keypoint-detection
- cuda
- pytorch
- fastforks
- fastforks-card
ViTPose / ViTPose++ (fastforks)
ViTPose (Xu et al. 2022): top-down 2D human pose with a plain ViT encoder and a light deconvolution (or bilinear) decoder; ViTPose++ trains one backbone on six pose datasets with per-dataset MLP experts. Takes person boxes (e.g. from fastforks.detection), crops each at 256 x 192 and returns the 17 COCO keypoints with DARK sub-pixel decoding. The portable module() maps crops to heatmaps.
This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: https://github.com/kleinicke/fastforks. Kernels for the kernels backend: https://huggingface.co/fastforks/kernels. Original code: https://github.com/huggingface/transformers at v4.57.1.
Author weight repos: https://huggingface.co/usyd-community/vitpose-base, https://huggingface.co/usyd-community/vitpose-base-simple, https://huggingface.co/usyd-community/vitpose-plus-base, https://huggingface.co/usyd-community/vitpose-plus-huge, https://huggingface.co/usyd-community/vitpose-plus-large, https://huggingface.co/usyd-community/vitpose-plus-small.
Task: pose. Tier: stable. Outputs: keypoints, keypoint_scores, boxes, scores. Backends: portable, kernels.
Input: one RGB image and a box per person. Output: keypoints with confidence scores for every person.
Variants
Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in card.toml.
| variant | weights (author link) | SHA-256 | weights licence | commercial use |
|---|---|---|---|---|
vitpose-base@1 |
vitpose-base-cd9a4e6c.safetensors | cd9a4e6cefc3 |
Apache-2.0 | yes |
vitpose-base-simple@1 |
vitpose-base-simple-85375373.safetensors | 85375373893d |
Apache-2.0 | yes |
vitpose-plus-base@1 |
vitpose-plus-base-640225e4.safetensors | 640225e4a9dd |
Apache-2.0 | yes |
vitpose-plus-huge@1 |
vitpose-plus-huge-0ecb49f1.safetensors | 0ecb49f1ab0b |
Apache-2.0 | yes |
vitpose-plus-large@1 |
vitpose-plus-large-8b249f74.safetensors | 8b249f74fca7 |
Apache-2.0 | yes |
vitpose-plus-small@1 |
vitpose-plus-small-f7bad8ed.safetensors | f7bad8ed09ee |
Apache-2.0 | yes |
Licence
Original-code strings, recorded as they are (research-use package; check them before any other use). Code: Apache-2.0. Licence files: LICENSE.
Training data: COCO 2017 (train, instance annotations), ImageNet-1k (ILSVRC 2012 classification) (COCO 2017 keypoints; the ViT was initialised from MAE ImageNet-1k pre-training).
Licence evidence: https://huggingface.co/usyd-community/vitpose-base-simple (model card licence)
Input size
any H x W >= 32, padded to a multiple of 32
Tested up to (portable, fp16, RTX 5060 Ti 16 GB): vitpose-base 3000x4000 (4.5 GiB peak allocated, 4.8 reserved); vitpose-base-simple 3000x4000 (4.5 GiB peak allocated, 4.8 reserved); vitpose-plus-base 3000x4000 (4.5 GiB peak allocated, 4.8 reserved); vitpose-plus-huge 3000x4000 (5.5 GiB peak allocated, 5.7 reserved); vitpose-plus-large 3000x4000 (4.9 GiB peak allocated, 5.2 reserved); vitpose-plus-small 3000x4000 (4.4 GiB peak allocated, 4.6 reserved)
Speed and memory
Pose, synthetic, 1080x1920
| model | time (ms): original fp32 | original | portable | kernels | kernels speed-up: vs original | vs portable | peak memory (MiB, allocated / reserved): original | portable | kernels |
|---|---|---|---|---|---|---|---|---|---|
vitpose-base |
139 | 105 | 17.8 | 12.0 | 8.8x | 1.5x | 713 / 762 | 299 / 322 | 274 / 364 |
vitpose-base-simple |
145 | 109 | 20.1 | 12.1 | 9.0x | 1.7x | 828 / 962 | 590 / 642 | 295 / 344 |
vitpose-plus-base |
162 | 113 | 19.9 | 11.9 | 9.5x | 1.7x | 918 / 1002 | 597 / 642 | 274 / 344 |
vitpose-plus-huge |
610 | 242 | 104 | 62.5 | 3.9x | 1.7x | 5732 / 5846 | 1641 / 1704 | 1351 / 1406 |
vitpose-plus-large |
331 | 165 | 53.7 | 33.2 | 5.0x | 1.6x | 2873 / 3004 | 1014 / 1062 | 706 / 764 |
vitpose-plus-small |
120 | 104 | 7.83 | 6.05 | 17.2x | 1.3x | 312 / 376 | 164 / 196 | 145 / 198 |
Startup and first call
| model | load (s): original | portable | kernels | first call (s): original | portable | kernels |
|---|---|---|---|---|---|---|
vitpose-base |
0.16 | 1.16 | 0.34 | 0.33 | 0.35 | 0.39 |
vitpose-base-simple |
0.15 | 0.40 | 1.04 | 0.17 | 0.41 | 0.41 |
vitpose-plus-base |
0.19 | 0.65 | 1.10 | 0.12 | 0.34 | 0.34 |
vitpose-plus-huge |
0.65 | 2.69 | 2.26 | 0.25 | 0.44 | 0.40 |
vitpose-plus-large |
0.41 | 1.50 | 1.63 | 0.17 | 0.41 | 0.41 |
vitpose-plus-small |
0.15 | 0.81 | 1.09 | 0.12 | 0.46 | 0.50 |
torch.compile and TensorRT
| model | input | time: portable | compile | TensorRT fp16 | kernels | preparation: compile (cold first call) | ONNX + engine build | peak MiB (alloc / reserved): portable | compile | kernels | TensorRT (engine + torch) |
|---|---|---|---|---|---|---|---|---|---|---|---|
vitpose-base |
1080x1920 | 17.8 ms | 16.4 ms | 8.88 ms (network on person crops only) | 12.0 ms | 10.5 s | 35.8 s | 299 / 322 | 597 / 622 | 274 / 364 | 473 / 516 |
vitpose-base-simple |
1080x1920 | 20.1 ms | 16.0 ms | 9.36 ms (network on person crops only) | 12.1 ms | 11.8 s | 35.0 s | 590 / 642 | 590 / 622 | 295 / 344 | 499 / 550 |
vitpose-plus-base |
1080x1920 | 19.9 ms | 16.4 ms | 8.87 ms (network on person crops only) | 11.9 ms | 10.6 s | 35.0 s | 597 / 642 | 597 / 622 | 274 / 344 | 473 / 516 |
vitpose-plus-huge |
1080x1920 | 104 ms | 92.9 ms | 58.7 ms (network on person crops only) | 62.5 ms | 21.2 s | 47.3 s | 1641 / 1704 | 1641 / 1682 | 1351 / 1406 | 2595 / 2654 |
vitpose-plus-large |
1080x1920 | 53.7 ms | 46.9 ms | 29.3 ms (network on person crops only) | 33.2 ms | 17.9 s | 39.0 s | 1014 / 1062 | 1014 / 1062 | 706 / 764 | 1312 / 1338 |
vitpose-plus-small |
1080x1920 | 7.83 ms | 7.54 ms | 3.39 ms (network on person crops only) | 6.05 ms | 10.6 s | 37.5 s | 164 / 196 | 472 / 496 | 145 / 198 | 214 / 256 |
Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).
Hardware, protocol and caveats: benchmarks (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or (fp32) where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.
Accuracy
On public datasets
Top-down on the ground-truth person boxes (non-crowd, at least one labelled keypoint), no flip test; people ranked by the mean of their 17 keypoint scores (not mmpose's rescoring and OKS-NMS), so not the papers' AP; pycocotools OKS AP; whole-body models scored on their 17 body keypoints. Original = the original code named under Details, same checkpoint and protocol.
| variant | dataset | original fp32 | original fp16 | portable | kernels | note |
|---|---|---|---|---|---|---|
vitpose-base-simple |
COCO val2017, ground-truth person boxes, keypoint AP | 76.3 | 76.2 | 76.3 | 76.3 | |
vitpose-base |
COCO val2017, ground-truth person boxes, keypoint AP | 76.4 | 76.4 | 76.4 | 76.4 | |
vitpose-plus-small |
COCO val2017, ground-truth person boxes, keypoint AP | 76.8 | 76.8 | 76.8 | 76.8 | |
vitpose-plus-base |
COCO val2017, ground-truth person boxes, keypoint AP | 78.6 | 78.6 | 78.6 | 78.6 | |
vitpose-plus-large |
COCO val2017, ground-truth person boxes, keypoint AP | 80.6 | 80.6 | 80.5 | 80.5 | |
vitpose-plus-huge |
COCO val2017, ground-truth person boxes, keypoint AP | 81.0 | 81.0 | 80.9 | 80.9 |
Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 2241b095+uncommitted, 2026-11-09.
Parity with the original code
Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.005 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.13 and p99 <= 0.75 (absolute error of the output keypoints).
Usage
from fastforks import pose
model = pose.load("vitpose-base") # mode="auto": the preferred backend from recorded measurements
result = model("photo.jpg", boxes=[[x0, y0, x1, y1]]) # one box per person, or a fastforks.detection result
result.keypoints, result.keypoint_scores # [P, K, 2] input pixels, [P, K]
result.backend # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')
Install with pip install fastforks (see https://github.com/kleinicke/fastforks). Other variants: vitpose-base, vitpose-base-simple, vitpose-plus-base, vitpose-plus-huge, vitpose-plus-large, vitpose-plus-small.
Kernels backend
pose.load("vitpose-base", mode="kernels") runs our CUDA kernels (fvm_conv). It needs a prebuilt kernel for your GPU architecture (downloaded from https://huggingface.co/fastforks/kernels on first use) and a recent NVIDIA driver; if there is none it raises BackendUnavailableError with the reason and never falls back silently. mode="auto" picks kernels when it can run and logs the choice.