RF-DETR Keypoint (fastforks)

Roboflow RF-DETR Keypoint Preview: a DINOv2-small backbone with windowed attention and a deformable transformer decoder that predicts people and 17 COCO joints directly from the whole image. The 40.7M-parameter Apache-2.0 preview uses 576 x 576 square inputs and conditional keypoint queries. Person confidence includes the upstream localization-uncertainty fusion; joint confidence is findability. No person detector, boxes or NMS are required.

This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: https://github.com/kleinicke/fastforks. Kernels for the kernels backend: https://huggingface.co/fastforks/kernels. Original code: https://github.com/roboflow/rf-detr at 5b22dfc09406.

Task: pose. Tier: stable. Outputs: keypoints, keypoint_scores, boxes, scores. Backends: portable, kernels.

Input: one RGB image and a box per person. Output: keypoints with confidence scores for every person.

Variants

Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in card.toml.

variant weights (author link) SHA-256 weights licence commercial use
rf-detr-keypoint-preview@1 rf-detr-keypoint-preview-xlarge.pth 2b70ddf51d2b Apache-2.0 yes

Licence

Original-code strings, recorded as they are (research-use package; check them before any other use). Code: Apache-2.0. Licence files: LICENSE.

Training data: COCO 2017 (train, instance annotations), LVD-142M (DINOv2's curated web-image set, not released) (Author README: pretrained on COCO person keypoints; DINOv2-small backbone originates from LVD-142M pretraining. Full intermediate training provenance is not stated.).

Licence evidence: https://github.com/roboflow/rf-detr/blob/5b22dfc0940697ce1abdfa681a5f7392b3eddbfb/README.md#keypoints

Input size

any H x W >= 32, padded to a multiple of 32 Tested up to (portable, fp16, RTX 5060 Ti 16 GB): rf-detr-keypoint-preview 3000x4000 (0.5 GiB peak allocated, 0.6 reserved)

Speed and memory

Pose, synthetic, 1080x1920

model time (ms): original fp32 original portable kernels kernels speed-up: vs original vs portable peak memory (MiB, allocated / reserved): original portable kernels
rf-detr-keypoint-preview 29.4 19.3 18.3 9.48 2.0x 1.9x 186 / 244 179 / 204 176 / 224

Startup and first call

model load (s): original portable kernels first call (s): original portable kernels
rf-detr-keypoint-preview 1.49 2.66 2.70 0.08 0.39 0.38

torch.compile and TensorRT

model input time: portable compile TensorRT fp16 kernels preparation: compile (cold first call) ONNX + engine build peak MiB (alloc / reserved): portable compile kernels TensorRT (engine + torch)
rf-detr-keypoint-preview 1080x1920 18.3 ms 8.00 ms 4.28 ms (network on person crops only) 9.48 ms 47.9 s 67.2 s 179 / 204 191 / 220 176 / 224 264 / 294

Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).

Hardware, protocol and caveats: benchmarks (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or (fp32) where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.

Accuracy

On public datasets

Bottom-up on all 5,000 validation images, no ground-truth boxes; people ranked by their predicted person score, after model-specific decoding; pycocotools OKS AP. Bottom-up on the same every-tenth-image subset for all modes, ranked by predicted person score after model-specific decoding; pycocotools OKS AP. Original = the original code named under Details, same checkpoint and protocol.

variant dataset original fp32 original fp16 portable kernels note
rf-detr-keypoint-preview COCO val2017, predicted people, keypoint AP 71.8 71.8 71.8 71.8
rf-detr-keypoint-preview COCO val2017, every 10th image, predicted people, keypoint AP 72.7 72.9 72.9 72.8

Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 0050000f, 2026-11-11.

Parity with the original code

Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.0001 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.12 and p99 <= 0.32 (absolute error of the output keypoints).

Limitations

  • Requires the optional transformers and torchvision libraries.
  • This is the Apache-2.0 Keypoint Preview checkpoint; restricted RF-DETR Plus detection variants are not included.
  • module() exports raw boxes, class logits and keypoint predictions; confidence fusion and thresholding remain outside the exported graph.
  • Float16 and float32 may retain different people near the score threshold; raw query ordering can differ across export engines.
  • torch.compile uses upstream positional-interpolation graph breaks; fullgraph=True is unsupported. Fixed-shape torch.export and ONNX remain supported.

Usage

from fastforks import pose

model = pose.load("rf-detr-keypoint-preview")  # mode="auto": the preferred backend from recorded measurements
result = model("photo.jpg", boxes=[[x0, y0, x1, y1]])  # one box per person, or a fastforks.detection result
result.keypoints, result.keypoint_scores  # [P, K, 2] input pixels, [P, K]
result.backend  # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')

Install with pip install fastforks (see https://github.com/kleinicke/fastforks). Other variants: rf-detr-keypoint-preview.

Kernels backend

pose.load("rf-detr-keypoint-preview", mode="kernels") runs our CUDA kernels (fastbb). It needs a prebuilt kernel for your GPU architecture (downloaded from https://huggingface.co/fastforks/kernels on first use) and a recent NVIDIA driver; if there is none it raises BackendUnavailableError with the reason and never falls back silently. mode="auto" picks kernels when it can run and logs the choice.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Collection including fastforks/rfdetr-keypoint