Infant posture-based action recognition: weights
Weights for the 2D infant_action_pipeline (video -> infant action), a self-contained rework of
ostadabbas/Infant-Posture-based-Action-Recognition
(Huang et al., Posture-based Infant Action Recognition in the Wild with Very Limited Data, CVPRW 2023).
Folders follow the pipeline stages, in order:
video frame
-> 1. detection/ infant bounding box
-> 2. pose_2d/ 17 COCO keypoints in the box
-> 3. posture_2d/ 4-class posture per frame (DINOv2+FiDIP): supine / prone / sitting / standing
(+ the old 5-class keypoint MLP, whose scores/features feed stage 4)
-> 4. transition_segmentor/ onset/offset of the posture transition over the clip
-> action = majority posture before onset -> majority posture after offset
| Stage | File | Model | Original source | md5 |
|---|---|---|---|---|
| 1 | detection/yolo_al03_hn.pt |
YOLOv8n (ultralytics 8.2.75) 1-class infant detector | twinsee active-learning round al03 + hard negatives, fine-tuned from yolo_al01.pt (lineage: FiDIP demo app best.pt) |
d88cf270305b2180ed8db877ebeee2b1 |
| 2 | pose_2d/hrnet_fidip.pth |
FiDIP, HRNet-W48 384x288 | FiDIP release, Drive id 19tBMoVS8wTza7VqVfPc6KBQ_POPAEIqb |
c1b4994799f08e6cbc44942f44472790 |
| 3 | posture_2d/posture_2d_5class.pth |
keypoint MLP, 24-d input (12 body joints) -> 5 classes | Drive folder 1X_d_Rle9aDeyCNECaH8Wu1N2IHdF9gAJ (kpts_ckpt.pth) |
e60082f855d06da69333175d9933dc60 |
| 3 | posture_2d/posture_dino_fidip_4class.pt |
DINOv2 ViT-B/14-reg (fully fine-tuned) on the 224 px infant crop + FiDIP keypoints -> 4 classes; 90.7% acc / 89.9 macro-F1 on the 878-frame portal posture bench | twinsee posture ablation, full finetune, seed 0 | c0342658c39e40f9789182f7a5897186 |
| 4 | transition_segmentor/prob_2d_best.pth |
BiRNN on the 5 posture scores | authors' action-recognition snapshot, checkpoints/ |
61d288bec55edfb97d782048ce488a60 |
| 4 | transition_segmentor/features_2d_best.pth |
BiRNN on the posture MLP's 16-d hidden features (PCA to 10) | same | 4dd39738689f64da4477ae079dd42adb |
| 4 | transition_segmentor/joints_2d_best.pth |
BiRNN on the 34 keypoint coordinates (PCA to 10) | same | 31a02212b2056c11de91fbcca4122d37 |
Segmentor checkpoints are dicts with state_dict, input_dim, hidden_size, num_layers, pca_mean, pca_axes
and val_mae (22-28 frames on the authors' validation split). The pipeline uses features by default.
Download
python download_weights.py # from the infant_action_pipeline repo; fills weights/ and md5-checks it
or huggingface_hub.snapshot_download('omrastogi/infact_action_weights', local_dir='weights').
License
The upstream code and models are released for non-commercial use only (Augmented Cognition Lab, Northeastern University). FiDIP / HRNet code parts are MIT (Microsoft).