Infant posture-based action recognition: weights
Weights for the 2D infant_action_pipeline (video -> infant action), a self-contained rework of
ostadabbas/Infant-Posture-based-Action-Recognition
(Huang et al., Posture-based Infant Action Recognition in the Wild with Very Limited Data, CVPRW 2023).
Folders follow the pipeline stages, in order:
video frame
-> 1. detection/ infant bounding box
-> 2. pose_2d/ 17 COCO keypoints in the box
-> 3. posture_2d/ 5-class posture per frame: supine / prone / sitting / standing / all-fours
-> 4. transition_segmentor/ onset/offset of the posture transition over the clip
-> action = majority posture before onset -> majority posture after offset
| Stage | File | Model | Original source | md5 |
|---|---|---|---|---|
| 1 | detection/yolo_infant.pt |
YOLOv8 (ultralytics) infant detector | weights/best.pt of the Augmented Cognition Lab FiDIP demo app |
e09fecd55011d46a82a6fc5165bf0c33 |
| 2 | pose_2d/hrnet_fidip.pth |
FiDIP, HRNet-W48 384x288 | FiDIP release, Drive id 19tBMoVS8wTza7VqVfPc6KBQ_POPAEIqb |
c1b4994799f08e6cbc44942f44472790 |
| 3 | posture_2d/posture_2d_5class.pth |
keypoint MLP, 24-d input (12 body joints) -> 5 classes | Drive folder 1X_d_Rle9aDeyCNECaH8Wu1N2IHdF9gAJ (kpts_ckpt.pth) |
e60082f855d06da69333175d9933dc60 |
| 4 | transition_segmentor/prob_2d_best.pth |
BiRNN on the 5 posture scores | authors' action-recognition snapshot, checkpoints/ |
61d288bec55edfb97d782048ce488a60 |
| 4 | transition_segmentor/features_2d_best.pth |
BiRNN on the posture MLP's 16-d hidden features (PCA to 10) | same | 4dd39738689f64da4477ae079dd42adb |
| 4 | transition_segmentor/joints_2d_best.pth |
BiRNN on the 34 keypoint coordinates (PCA to 10) | same | 31a02212b2056c11de91fbcca4122d37 |
Segmentor checkpoints are dicts with state_dict, input_dim, hidden_size, num_layers, pca_mean, pca_axes
and val_mae (22-28 frames on the authors' validation split). The pipeline uses features by default.
Download
python download_weights.py # from the infant_action_pipeline repo; fills weights/ and md5-checks it
or huggingface_hub.snapshot_download('omrastogi/infact_action_weights', local_dir='weights').
License
The upstream code and models are released for non-commercial use only (Augmented Cognition Lab, Northeastern University). FiDIP / HRNet code parts are MIT (Microsoft).