File size: 3,278 Bytes
1dc3703 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 | ---
license: other
library_name: safetensors
tags:
- comfyui
- pose-estimation
- object-detection
- wholebody
pipeline_tag: keypoint-detection
---
# BCVideoNodes models
The detection and pose models used by
[ComfyUI-BCVideoNodes](https://github.com/0xBeycan/ComfyUI-BCVideoNodes).
The nodes download whatever they need from here on first use, so nothing in this repository
has to be fetched by hand. It exists so the weights have a stable home, and so anyone can
see exactly which files a node is pulling.
## What is here
| file | model | source | licence |
|---|---|---|---|
| `vitpose_h_wholebody_fp16.safetensors` | ViTPose-H, 133 COCO-WholeBody keypoints | [Kijai/vitpose_comfy](https://huggingface.co/Kijai/vitpose_comfy) | Apache-2.0 |
| `rtmw_l_wholebody_384x288_fp32.safetensors` | RTMW-l, 133 COCO-WholeBody keypoints | OpenMMLab's own export, via [bukuroo/RTMW-ONNX](https://huggingface.co/bukuroo/RTMW-ONNX) | Apache-2.0 |
| `yolov10x_fp32.safetensors` | YOLOv10-X person detection | [onnx-community/yolov10x](https://huggingface.co/onnx-community/yolov10x) | AGPL-3.0 |
`vitpose_h_wholebody` is fp16; the other two are fp32. See below.
## Why these are not the original files
Upstream ships these as ONNX. The nodes run them as plain torch modules instead — the
architecture is written out by hand and the weights are loaded into it, which removes the
`onnx` dependency, removes a protobuf parse and a graph walk from every model load, and
replaces ViTPose's two-file external-data export (a 420 KB graph plus a 2.5 GB `.bin`) with
one file.
Each file carries its architecture in the safetensors metadata, so the loader needs nothing
but the file.
## Precision
ViTPose-H is fp16, the other two are fp32, because fp16 is not free and only one of them is
large enough for it to pay:
| model | fp32 | fp16 | saved |
|---|---|---|---|
| ViTPose-H | 2.43 GB | 1.22 GB | 1.2 GB |
| RTMW-l | 219 MB | 110 MB | 110 MB |
| YOLOv10x | 113 MB | 57 MB | 56 MB |
What it costs, measured on 60 real frames over every keypoint the model draws, against the
same model in fp32 on identical crops: ViTPose-H, 5230 keypoints compared, 20 moved, largest
move **1 heatmap cell**. RTMW-l, 5012 compared, 401 moved, largest move **1.5 px**. Worth
paying for 1.2 GB, not worth paying for 110 MB.
Keypoints the model does not draw are excluded from those figures, and that matters: on
those the score distribution is flat, so the argmax is noise in either precision, and fp16
appears to move them by up to 215 px. That number is what a flat distribution does, not what
fp16 does.
## Conversion
The converter lives in the node repository,
[`scripts/convert_models.py`](https://github.com/0xBeycan/ComfyUI-BCVideoNodes/blob/main/scripts/convert_models.py). It reads the upstream ONNX file, builds the
native module from it, and writes the module's own state dict plus its architecture. Rerun
it to update a model; do not hand-edit the files here.
## Credit
The pose models are OpenMMLab's and the ViTPose authors' work; YOLOv10 is
[THU-MIG](https://github.com/THU-MIG/yolov10). This repository only re-packages them. The
wholebody pose utilities in the node repository are vendored from the Alibaba Wan team's
WanAnimate preprocess and keep their copyright header.
|