Upload reports/RESULTS_FINAL.md
Browse files- reports/RESULTS_FINAL.md +255 -0
reports/RESULTS_FINAL.md
ADDED
|
@@ -0,0 +1,255 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# SkelGym: Resistance Exercise Recognition — Master Experiment Results & Benchmark Template
|
| 2 |
+
|
| 3 |
+
> **Document Status:** Master Results Execution Template (Clean Slate — Fresh Benchmark Run)
|
| 4 |
+
> **Authoritative Baseline Split:** Video-Level Partition 6:2:2 (580 Train / 208 Validation / 236 Test; 233 Valid Test Videos $\ge 32$ Frames)
|
| 5 |
+
> **Repository:** `Cuong2004/gym-exercise-classification` (Model Hub) | `Cuong2004/gym-exercise-landmarks` (Dataset Hub)
|
| 6 |
+
> **Backbone Budget:** Calibrated compact parameter budget ($\approx 300\text{K} - 350\text{K}$ parameters across all backbones)
|
| 7 |
+
|
| 8 |
+
---
|
| 9 |
+
|
| 10 |
+
## 1. Experimental Protocol & Specifications
|
| 11 |
+
|
| 12 |
+
- **Dataset Scale:** 1,024 unique video recordings ($\approx 10.2$ GB) spanning 22 fine-grained resistance exercises, trimmed into 1,108 clean action segments.
|
| 13 |
+
- **Strict Video-Level Partitioning:** Zero source-video overlap across Train ($N=580$), Validation ($N=208$), and Test ($N=236$) sets. Sliding windows ($T=32$ frames) are extracted strictly within individual trimmed action boundaries ($S=16$ train: 13,136 windows; $S=32$ val: 2,075 windows; $S=32$ test: 2,743 windows across 233 valid videos).
|
| 14 |
+
- **Physical Kinematics:** 13 anatomically calibrated keypoints from MediaPipe Pose Heavy.
|
| 15 |
+
- **Scale-Normalized Relative Coordinates (`rel_3d_norm`, 39-d):** Origin centered at pelvic mid-hip midpoint ($p_{\text{hip\_mid}}$), lateral $X$-axis normalized by pelvic hip width $L_{\text{hip}}$, and longitudinal $Y, Z$-axes normalized by torso length $L_{\text{torso}}$.
|
| 16 |
+
- **Kinematic Angles (`angle_kinematic_24`, 24-d):** 10 3D joint articulation flexion/extension triplets (knees, elbows, hips, shoulders) and 14 spatial limb segment orientation angles.
|
| 17 |
+
- **Biomechanical Mix v2 (`mix_v2`, 63-d):** Dual-branch concatenation of 39-d scale-norm coordinates and 24-d kinematic angles.
|
| 18 |
+
- **Model Capacity:** All sequence backbones standardized to $\approx 300\text{K}$ parameters (Transformer: $d_{\text{model}}=112, d_{\text{ff}}=168, \text{nhead}=4, \text{layers}=3$; LSTM: $h=160, \text{layers}=2$; BiLSTM: $h=96, \text{layers}=2$).
|
| 19 |
+
- **Augmentation Pipeline (SkelGym-Aug):** Dynamic on-the-fly transformations (Bilateral Sagittal Reflection + Gravitational 3D Yaw $\pm 15^\circ$ + Proportional Scaling $\pm 10\%$ + Gaussian Sensor Jitter $\sigma=0.008$).
|
| 20 |
+
- **Multi-Stream Ensemble:** 5-stream cross-paradigm late fusion (Transformer on 63-d mix_v2 + 4 spatial-temporal AAGCN streams: Bone 3D, Relative 3D, Joint-Motion 3D, Bone-Motion 3D) calibrated via SLSQP on the validation partition.
|
| 21 |
+
|
| 22 |
+
---
|
| 23 |
+
|
| 24 |
+
## Table 1: Dataset Partition & Provenance Breakdown
|
| 25 |
+
|
| 26 |
+
| Split Name | Source Videos | Ratio (%) | Action Segments | Valid Videos ($\ge 32$ frames) | Extracted Windows ($T=32$) | Stride ($S$) | Status |
|
| 27 |
+
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 28 |
+
| **Train Set** | 580 | 56.6% | 639 | 580 | 13,136 | 16 (50% overlap) | Verified |
|
| 29 |
+
| **Validation Set** | 208 | 20.3% | 210 | 208 | 2,075 | 32 (non-overlapping) | Verified |
|
| 30 |
+
| **Held-Out Test Set** | 236 | 23.1% | 259 | 233 | 2,743 | 32 (non-overlapping) | Verified |
|
| 31 |
+
| **Total Corpus** | **1,024** | **100.0%** | **1,108** | **1,021** | **17,954** | — | **Verified** |
|
| 32 |
+
|
| 33 |
+
---
|
| 34 |
+
|
| 35 |
+
## Table 2: Feature Representation Benchmark across Sequence Architectures (Paper Table 1)
|
| 36 |
+
|
| 37 |
+
*Objective:* Evaluate 3 sequence architectures (LSTM, BiLSTM, Transformer) across spatial coordinate, angular, and biomechanical representations under unaugmented baseline protocol.
|
| 38 |
+
*Execution Command:* `python run.py train --model <MODEL> --feature <FEATURE> --exp_id <ID> --device auto`
|
| 39 |
+
|
| 40 |
+
| Exp ID | Model Architecture | Feature Representation | Dimension | Train Loss | Train Acc (%) | Val Loss | Val Acc (%) | Val Macro F1 | Test Win Acc (%) | Test Macro F1 | Status |
|
| 41 |
+
| :---: | :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 42 |
+
| **T1.1** | LSTM | Raw 2D Coordinates | 26 | 0.1183 | — | 2.0490 | 69.25% ± 0.98% | — | 54.26% ± 0.27% | 0.5352 ± 0.0067 | Verified |
|
| 43 |
+
| **T1.2** | LSTM | Root-Relative 2D Coordinates | 26 | 0.0805 | — | 1.8685 | 75.29% ± 1.30% | — | 58.54% ± 1.05% | 0.5708 ± 0.0130 | Verified |
|
| 44 |
+
| **T1.3** | LSTM | Pairwise Joint Angles 2D | 286 | 0.1343 | — | 2.5289 | 66.12% ± 1.06% | — | 51.53% ± 1.57% | 0.4993 ± 0.0119 | Verified |
|
| 45 |
+
| **T1.4** | LSTM | Adjacent Joint Angles 2D | 78 | 0.1327 | — | 2.3319 | 69.98% ± 0.79% | — | 55.73% ± 1.29% | 0.5412 ± 0.0147 | Verified |
|
| 46 |
+
| **T1.5** | LSTM | Raw 3D Coordinates | 39 | 0.1183 | — | 2.0571 | 68.47% ± 1.67% | — | 53.53% ± 1.18% | 0.5117 ± 0.0133 | Verified |
|
| 47 |
+
| **T1.6** | LSTM | Root-Relative 3D Coordinates | 39 | 0.1548 | — | 1.6984 | 73.09% ± 1.02% | — | 58.77% ± 1.38% | 0.5738 ± 0.0127 | Verified |
|
| 48 |
+
| **T1.7** | LSTM | Pairwise Joint Angles 3D | 286 | 0.1426 | — | 2.3883 | 65.16% ± 0.97% | — | 50.68% ± 0.51% | 0.4923 ± 0.0040 | Verified |
|
| 49 |
+
| **T1.8** | LSTM | Adjacent Joint Angles 3D | 78 | 0.0943 | — | 1.9178 | 72.84% ± 1.06% | — | 57.42% ± 0.62% | 0.5680 ± 0.0097 | Verified |
|
| 50 |
+
| **T1.9** | LSTM | Hybrid Geometric Multi-Feature (mix_v2) | 63 | 0.0417 | — | 1.9940 | 77.78% ± 1.15% | — | 51.17% ± 13.08% | 0.5014 ± 0.1232 | Verified |
|
| 51 |
+
| **T1.10** | BiLSTM | Raw 2D Coordinates | 26 | 0.0747 | — | 2.2106 | 70.81% ± 0.73% | — | 54.28% ± 0.08% | 0.5338 ± 0.0033 | Verified |
|
| 52 |
+
| **T1.11** | BiLSTM | Root-Relative 2D Coordinates | 26 | 0.0790 | — | 1.8599 | 75.21% ± 0.91% | — | 58.95% ± 0.84% | 0.5700 ± 0.0101 | Verified |
|
| 53 |
+
| **T1.12** | BiLSTM | Pairwise Joint Angles 2D | 286 | 0.1284 | — | 2.5607 | 65.96% ± 0.47% | — | 52.37% ± 1.98% | 0.5043 ± 0.0206 | Verified |
|
| 54 |
+
| **T1.13** | BiLSTM | Adjacent Joint Angles 2D | 78 | 0.0623 | — | 2.5136 | 71.08% ± 0.17% | — | 56.53% ± 0.29% | 0.5508 ± 0.0072 | Verified |
|
| 55 |
+
| **T1.14** | BiLSTM | Raw 3D Coordinates | 39 | 0.1339 | — | 2.1974 | 68.19% ± 0.86% | — | 53.22% ± 0.90% | 0.5045 ± 0.0074 | Verified |
|
| 56 |
+
| **T1.15** | BiLSTM | Root-Relative 3D Coordinates | 39 | 0.1059 | — | 1.9566 | 73.24% ± 0.83% | — | 58.43% ± 0.99% | 0.5634 ± 0.0034 | Verified |
|
| 57 |
+
| **T1.16** | BiLSTM | Pairwise Joint Angles 3D | 286 | 0.1690 | — | 2.0893 | 64.47% ± 0.69% | — | 51.25% ± 1.82% | 0.4922 ± 0.0194 | Verified |
|
| 58 |
+
| **T1.17** | BiLSTM | Adjacent Joint Angles 3D | 78 | 0.1432 | — | 1.6524 | 73.19% ± 0.67% | — | 57.90% ± 1.11% | 0.5719 ± 0.0098 | Verified |
|
| 59 |
+
| **T1.18** | BiLSTM | Hybrid Geometric Multi-Feature (mix_v2) | 63 | 0.0828 | — | 1.6630 | 76.16% ± 2.19% | — | 49.75% ± 13.47% | 0.4921 ± 0.1297 | Verified |
|
| 60 |
+
| **T1.19** | Transformer | Raw 2D Coordinates | 26 | 0.3905 | — | 1.1568 | 77.75% ± 0.53% | — | 63.62% ± 1.06% | 0.6258 ± 0.0118 | Verified |
|
| 61 |
+
| **T1.20** | Transformer | Root-Relative 2D Coordinates | 26 | 0.5088 | — | 1.2146 | 75.44% ± 0.79% | — | 61.22% ± 1.61% | 0.5981 ± 0.0172 | Verified |
|
| 62 |
+
| **T1.21** | Transformer | Pairwise Joint Angles 2D | 286 | 0.4268 | — | 1.4610 | 69.85% ± 0.47% | — | 55.95% ± 0.74% | 0.5310 ± 0.0115 | Verified |
|
| 63 |
+
| **T1.22** | Transformer | Adjacent Joint Angles 2D | 78 | 0.4115 | — | 1.4490 | 70.15% ± 1.19% | — | 55.12% ± 1.68% | 0.5407 ± 0.0205 | Verified |
|
| 64 |
+
| **T1.23** | Transformer | Raw 3D Coordinates | 39 | 0.4239 | — | 1.1319 | 77.24% ± 0.59% | — | 62.87% ± 0.69% | 0.6202 ± 0.0097 | Verified |
|
| 65 |
+
| **T1.24** | Transformer | Root-Relative 3D Coordinates | 39 | 0.4344 | — | 1.1740 | 77.88% ± 0.89% | — | 63.87% ± 0.73% | 0.6179 ± 0.0080 | Verified |
|
| 66 |
+
| **T1.25** | Transformer | Pairwise Joint Angles 3D | 286 | 0.4029 | — | 1.5516 | 69.51% ± 1.03% | — | 57.41% ± 0.54% | 0.5570 ± 0.0062 | Verified |
|
| 67 |
+
| **T1.26** | Transformer | Adjacent Joint Angles 3D | 78 | 0.3812 | — | 1.4079 | 73.51% ± 1.10% | — | 59.56% ± 0.66% | 0.5922 ± 0.0054 | Verified |
|
| 68 |
+
| **T1.27** | Transformer | Hybrid Geometric Multi-Feature (mix_v2) | 63 | 0.3777 | — | 1.0535 | 80.98% ± 0.80% | — | 58.57% ± 12.11% | 0.5830 ± 0.1145 | Verified |
|
| 69 |
+
|
| 70 |
+
---
|
| 71 |
+
|
| 72 |
+
## Table 2b: Multi-Seed Controlled-Capacity Transformer Feature Benchmark (300K Budget, Seeds 42, 123, 3407)
|
| 73 |
+
|
| 74 |
+
*Objective:* Multi-seed validation of Raw 3D, Scale-Norm Relative 3D, and Biomechanical Mix v2 on Transformer (~301K params).
|
| 75 |
+
*Execution Command:* `python scripts/evaluate_upgrade_mix_all.py`
|
| 76 |
+
|
| 77 |
+
| Feature Paradigm | Input Dim | Params | Train Loss | Val Loss | Val Win Acc (%) | Val Win Macro F1 | Val Vid Acc (%) | Val Vid Macro F1 | Test Win Acc (%) | Test Win Macro F1 | Test Vid Acc (%) | Test Vid Macro F1 | Status |
|
| 78 |
+
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 79 |
+
| **Raw 3D Coordinates** | 39 | 301K | 0.3895 | 1.1361 | 77.04% ± 0.38% | 0.7662 ± 0.0063 | 78.90% ± 1.39% | 0.7972 ± 0.0076 | 63.36% ± 1.48% | 0.6228 ± 0.0176 | 72.39% ± 0.88% | 0.7045 ± 0.0130 | Verified |
|
| 80 |
+
| **Scale-Norm Rel 3D (`rel_3d_norm`)** | 39 | 301K | 0.4241 | 1.1391 | 77.98% ± 0.86% | 0.7691 ± 0.0080 | 81.32% ± 0.46% | 0.8148 ± 0.0014 | 63.46% ± 0.40% | 0.6221 ± 0.0064 | 73.53% ± 1.07% | 0.7067 ± 0.0071 | Verified |
|
| 81 |
+
| **Biomechanical Mix v2 (`mix_v2`)** | 63 | 300K | 0.4239 | 1.1005 | 77.70% ± 1.05% | 0.7724 ± 0.0118 | 81.00% ± 1.49% | 0.8075 ± 0.0236 | 63.90% ± 1.01% | 0.6318 ± 0.0086 | 74.82% ± 1.01% | 0.7265 ± 0.0108 | Verified |
|
| 82 |
+
|
| 83 |
+
---
|
| 84 |
+
|
| 85 |
+
## Table 3: Spatial-Temporal Graph Kinematic Streams (Paper Table 4)
|
| 86 |
+
|
| 87 |
+
*Objective:* Evaluate static physical adjacency ($A_{\text{phys}}$) versus learnable adaptive topology ($B_k + C_k$) across 4 kinematic modalities.
|
| 88 |
+
*Execution Command:* `python run.py train --model AAGCN --feature <STREAM> --augment skel_gym_aug --device auto`
|
| 89 |
+
|
| 90 |
+
| Exp ID | Model Architecture | Kinematic Stream | Augmentation Protocol | Train Loss | Train Acc (%) | Val Loss | Val Acc (%) | Val Macro F1 | Test Win Acc (%) | Test Vid Acc (%) | Status |
|
| 91 |
+
| :---: | :--- | :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 92 |
+
| **T3.1** | ST-GCN Baseline | Raw 3D Joint | None (Clean) | - | - | - | - | - | - | - | Pending |
|
| 93 |
+
| **T3.2** | ST-GCN Baseline | Relative 3D Joint | None (Clean) | - | - | - | - | - | - | - | Pending |
|
| 94 |
+
| **T3.3** | AAGCN Baseline | Bone 3D Stream | None (Clean) | - | - | - | - | - | - | - | Pending |
|
| 95 |
+
| **T3.4** | AAGCN | Bone 3D Stream | SkelGym-Aug (Proposed) | - | - | - | - | - | - | - | Pending |
|
| 96 |
+
| **T3.5** | AAGCN | Joint Stream (Rel 3D) | SkelGym-Aug (Proposed) | - | - | - | - | - | - | - | Pending |
|
| 97 |
+
| **T3.6** | AAGCN | Joint Motion 3D ($\Delta X$) | SkelGym-Aug (Proposed) | - | - | - | - | - | - | - | Pending |
|
| 98 |
+
| **T3.7** | AAGCN | Bone Motion 3D ($\Delta B$) | SkelGym-Aug (Proposed) | - | - | - | - | - | - | - | Pending |
|
| 99 |
+
| **T3.8** | Two-Stream AAGCN | Joint + Bone | Late Fusion (Equal Weights) | - | - | - | - | - | - | - | Pending |
|
| 100 |
+
| **T3.9** | Four-Stream AAGCN | 4 Streams Unified | Late Fusion (SLSQP Calibrated) | - | - | - | - | - | - | - | Pending |
|
| 101 |
+
|
| 102 |
+
---
|
| 103 |
+
|
| 104 |
+
## Table 4: Systematic Leave-One-Out (LOO) Augmentation Ablation on Transformer Mix (Paper Table 2)
|
| 105 |
+
|
| 106 |
+
*Objective:* Evaluate necessity of individual operators by excluding one at a time from Candidate Full (5-op) across 3 seeds ($42, 123, 3407$).
|
| 107 |
+
*Execution Command:* `python scripts/run_augmentation_experiments.py --mode loo --seeds 42 123 3407 --force_retrain`
|
| 108 |
+
|
| 109 |
+
| Augmentation Configuration | Excluded Operator / Domain | Val Loss | Val Window Acc (%) | Val Macro F1 | Test Window Acc (%) | Test Macro F1 | $\Delta$ vs Full (Test Win) | Status |
|
| 110 |
+
| :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 111 |
+
| **Candidate Full (All 5 Ops)** | None (Reference Suite) | - | - | - | - | - | 0.00% (Ref) | Pending |
|
| 112 |
+
| **Minus Noise / Jitter** | Gaussian Coordinate Jitter ($\sigma=0.008$) | - | - | - | - | - | - | Pending |
|
| 113 |
+
| **Minus Mirroring** | Sagittal Horizontal Flip ($p=0.5$) | - | - | - | - | - | - | Pending |
|
| 114 |
+
| **Minus Rotation / Yaw** | Gravitational Yaw Rotation ($\pm 15^\circ$) | - | - | - | - | - | - | Pending |
|
| 115 |
+
| **Minus Scaling** | Proportional Scale Variation ($\pm 10\%$) | - | - | - | - | - | - | Pending |
|
| 116 |
+
| **Minus Time Interpolation** | Temporal Resampling ($0.8\times - 1.2\times$) | - | - | - | - | - | - | Pending |
|
| 117 |
+
|
| 118 |
+
---
|
| 119 |
+
|
| 120 |
+
## Table 5: Single-Component Isolated Augmentation Ablation on Transformer Mix (Paper Table 3)
|
| 121 |
+
|
| 122 |
+
*Objective:* Evaluate standalone individual gain for each augmentation operator relative to unaugmented baseline.
|
| 123 |
+
*Execution Command:* `python scripts/run_augmentation_experiments.py --mode single --seeds 42 123 3407 --force_retrain`
|
| 124 |
+
|
| 125 |
+
| Augmentation Strategy | Isolated Operator Description | Val Loss | Val Window Acc (%) | Val Macro F1 | Test Window Acc (%) | Test Macro F1 | Gain (+$\Delta$%) | Status |
|
| 126 |
+
| :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 127 |
+
| **None (Clean Baseline)** | Unaugmented Native Window Sequences | - | - | - | - | - | 0.00% (Ref) | Pending |
|
| 128 |
+
| **Only Jitter** | Gaussian Noise ($\sigma=0.008$) | - | - | - | - | - | - | Pending |
|
| 129 |
+
| **Only Mirroring** | Sagittal Bilateral Reflection | - | - | - | - | - | - | Pending |
|
| 130 |
+
| **Only Rotation** | 3D Yaw Perturbation ($\pm 15^\circ$) | - | - | - | - | - | - | Pending |
|
| 131 |
+
| **Only Scaling** | Proportional Scale Jitter ($\pm 10\%$) | - | - | - | - | - | - | Pending |
|
| 132 |
+
| **Only Time Interpolation** | Linear Sequence Resampling | - | - | - | - | - | - | Pending |
|
| 133 |
+
|
| 134 |
+
---
|
| 135 |
+
|
| 136 |
+
## Table 6: Multi-Stream Cross-Paradigm Ensemble Comparison (Paper Table 6)
|
| 137 |
+
|
| 138 |
+
*Objective:* Benchmark 5 systematic fusion methods and multi-seed downstream consistency across seeds $42, 123, 3407$.
|
| 139 |
+
*Execution Command:* `python scripts/run_multi_seed_experiments.py --seeds 42 123 3407`
|
| 140 |
+
|
| 141 |
+
| Architecture / Configuration | Fusion Protocol & Weighting | Val Win Acc (%) | Val Win F1 | Val Vid Acc (%) | Val Vid F1 | Test Win Acc (%) | Test Win F1 | Test Vid Acc (%) | Test Vid F1 | Status |
|
| 142 |
+
| :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 143 |
+
| **Transformer Mix (63-d)** | Single Sequence Backbone | - | - | - | - | - | - | - | - | Pending |
|
| 144 |
+
| **AAGCN Bone Stream** | Single Graph Backbone | - | - | - | - | - | - | - | - | Pending |
|
| 145 |
+
| **Four-Stream AAGCN** | 4 Streams Unified Graph | - | - | - | - | - | - | - | - | Pending |
|
| 146 |
+
| **Hard Majority Voting** | Discrete mode over class predictions | - | - | - | - | - | - | - | - | Pending |
|
| 147 |
+
| **Uniform Average Soft Voting** | Equal weights ($w_i = 1/5 = 0.20$) | - | - | - | - | - | - | - | - | Pending |
|
| 148 |
+
| **Accuracy-Weighted Soft Voting** | Validation accuracy weights | - | - | - | - | - | - | - | - | Pending |
|
| 149 |
+
| **SLSQP Soft Voting** | SLSQP Constrained Calibration ($\sum w_i = 1$) | - | - | - | - | - | - | - | - | Pending |
|
| 150 |
+
| **Stacking Meta-Classifier** | Ridge Classifier on Val Probs | - | - | - | - | - | - | - | - | Pending |
|
| 151 |
+
| **SkelGym-Lite (2 Models)** | Trans + Bone AAGCN (SLSQP Calibrated) | - | - | - | - | - | - | - | - | Pending |
|
| 152 |
+
| **SkelGym-Full (5 Streams)** | Trans + 4 AAGCN (SLSQP Calibrated) | - | - | - | - | - | - | - | - | Pending |
|
| 153 |
+
|
| 154 |
+
---
|
| 155 |
+
|
| 156 |
+
## Table 7: Window-Level vs Video Consensus Predictions & Parameter Footprints (Paper Table 7)
|
| 157 |
+
|
| 158 |
+
*Objective:* Quantify consensus pooling accuracy gains and compare total trainable parameters across all 11 architectures.
|
| 159 |
+
*Execution Command:* `python run.py evaluate --checkpoint <CKPT> --video_level --device auto`
|
| 160 |
+
|
| 161 |
+
| Model Architecture | Input Modality / Paradigm | Trainable Params | Test Win Acc (%) | Test Win Macro F1 | Test Vid Acc (%) | Test Vid Macro F1 | Video Gain (+$\Delta$%) | Status |
|
| 162 |
+
| :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 163 |
+
| **Baseline LSTM (Mix 63-d)** | Sequential Recurrent Model | ~378K | - | - | - | - | - | Pending |
|
| 164 |
+
| **Baseline BiLSTM (Mix 63-d)** | Bidirectional Recurrent Model | ~367K | - | - | - | - | - | Pending |
|
| 165 |
+
| **Transformer (Mix 63-d, Clean)** | Self-Attention Baseline | ~300K | - | - | - | - | - | Pending |
|
| 166 |
+
| **Baseline ST-GCN (Rel 3D)** | Rigid Static Graph ($A_{\text{phys}}$) | ~350K | - | - | - | - | - | Pending |
|
| 167 |
+
| **Clean Baseline AAGCN (Bone 3D)** | Adaptive Skeletal Graph (Unaugmented) | ~378K | - | - | - | - | - | Pending |
|
| 168 |
+
| **SkelGym-Aug AAGCN (Bone 3D)** | Adaptive Skeletal Graph + Augmentation | ~378K | - | - | - | - | - | Pending |
|
| 169 |
+
| **SkelGym-Aug Transformer (Mix)** | Self-Attention + Augmentation | ~300K | - | - | - | - | - | Pending |
|
| 170 |
+
| **Two-Stream AAGCN (Aug)** | Joint + Bone Stream Fusion | ~756K | - | - | - | - | - | Pending |
|
| 171 |
+
| **Four-Stream AAGCN (Aug)** | 4-Stream Graph Late Fusion | ~1.51M | - | - | - | - | - | Pending |
|
| 172 |
+
| **SkelGym-Lite (2 Models)** | Transformer + Bone AAGCN | ~678K | - | - | - | - | - | Pending |
|
| 173 |
+
| **SkelGym-Full (5 Streams)** | Cross-Paradigm SLSQP Ensemble | ~1.81M | - | - | - | - | - | Pending |
|
| 174 |
+
|
| 175 |
+
---
|
| 176 |
+
|
| 177 |
+
## Table 8: Paired Statistical Hypothesis Testing (Paper Table 8)
|
| 178 |
+
|
| 179 |
+
*Objective:* Verify pairwise model superiority with McNemar test on test windows ($N=2,743$) and Wilcoxon signed-rank + paired $t$-test on video clusters ($N=233$).
|
| 180 |
+
*Execution Command:* `python scripts/compute_statistical_tests.py`
|
| 181 |
+
|
| 182 |
+
| Pairwise Comparison ($M_A$ vs. $M_B$) | Window McNemar $\chi^2$ | Window $p$-value | Window Odds Ratio | Video Wilcoxon $W$ | Video $p$-value | Video Paired $t$ | Video Cohen's $d$ | Status |
|
| 183 |
+
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 184 |
+
| **Unaugmented Trans vs SkelGym-Aug Trans** | - | - | - | - | - | - | - | Pending |
|
| 185 |
+
| **Fixed ST-GCN vs Adaptive Four-Stream AAGCN** | - | - | - | - | - | - | - | Pending |
|
| 186 |
+
| **Single Sequence (Trans) vs SkelGym-Full** | - | - | - | - | - | - | - | Pending |
|
| 187 |
+
| **Single Graph (AAGCN Bone) vs SkelGym-Full** | - | - | - | - | - | - | - | Pending |
|
| 188 |
+
| **Four-Stream Graph AAGCN vs SkelGym-Full** | - | - | - | - | - | - | - | Pending |
|
| 189 |
+
|
| 190 |
+
---
|
| 191 |
+
|
| 192 |
+
## Table 9: Non-Parametric Video-Level Cluster Bootstrap (B=1,000 Resamples) (Paper Table 9)
|
| 193 |
+
|
| 194 |
+
*Objective:* Quantify sampling stability and compute unbiased 95% Confidence Intervals via video-cluster resampling.
|
| 195 |
+
*Execution Command:* `python scripts/compute_statistical_tests.py`
|
| 196 |
+
|
| 197 |
+
| Model Architecture | Window Test Acc [95% CI] | Window Macro F1 [95% CI] | Video Consensus Acc [95% CI] | Video Macro F1 [95% CI] | Status |
|
| 198 |
+
| :--- | :---: | :---: | :---: | :---: | :---: |
|
| 199 |
+
| **LSTM (Mix 63-d)** | - | - | - | - | Pending |
|
| 200 |
+
| **BiLSTM (Mix 63-d)** | - | - | - | - | Pending |
|
| 201 |
+
| **ST-GCN (Rel 3D)** | - | - | - | - | Pending |
|
| 202 |
+
| **Transformer (Mix 63-d)** | - | - | - | - | Pending |
|
| 203 |
+
| **AAGCN (Bone 3D)** | - | - | - | - | Pending |
|
| 204 |
+
| **SkelGym-Lite (2 Models)** | - | - | - | - | Pending |
|
| 205 |
+
| **SkelGym-Full (5 Streams)** | - | - | - | - | Pending |
|
| 206 |
+
|
| 207 |
+
---
|
| 208 |
+
|
| 209 |
+
## Table 10: Per-Class Performance Breakdown (Paper Table 10 & 13)
|
| 210 |
+
|
| 211 |
+
*Objective:* Detailed per-class precision, recall, and F1 metrics for SkelGym-Full on held-out test windows ($N=2,743$) and test videos ($N=233$).
|
| 212 |
+
|
| 213 |
+
| Exercise Class | Window Precision | Window Recall | Window F1 | Window Support | Video Precision | Video Recall | Video F1 | Video Support | Status |
|
| 214 |
+
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 215 |
+
| **barbell biceps curl** | - | - | - | 69 | - | - | - | 14 | Pending |
|
| 216 |
+
| **bench press** | - | - | - | 96 | - | - | - | 14 | Pending |
|
| 217 |
+
| **chest fly machine** | - | - | - | 82 | - | - | - | 8 | Pending |
|
| 218 |
+
| **deadlift** | - | - | - | 67 | - | - | - | 10 | Pending |
|
| 219 |
+
| **decline bench press** | - | - | - | 134 | - | - | - | 8 | Pending |
|
| 220 |
+
| **hammer curl** | - | - | - | 161 | - | - | - | 14 | Pending |
|
| 221 |
+
| **hip thrust** | - | - | - | 236 | - | - | - | 9 | Pending |
|
| 222 |
+
| **incline bench press** | - | - | - | 76 | - | - | - | 9 | Pending |
|
| 223 |
+
| **lat pulldown** | - | - | - | 100 | - | - | - | 13 | Pending |
|
| 224 |
+
| **lateral raise** | - | - | - | 155 | - | - | - | 15 | Pending |
|
| 225 |
+
| **leg extension** | - | - | - | 125 | - | - | - | 13 | Pending |
|
| 226 |
+
| **leg raises** | - | - | - | 114 | - | - | - | 11 | Pending |
|
| 227 |
+
| **plank** | - | - | - | 56 | - | - | - | 2 | Pending |
|
| 228 |
+
| **pull Up** | - | - | - | 82 | - | - | - | 10 | Pending |
|
| 229 |
+
| **push-up** | - | - | - | 87 | - | - | - | 12 | Pending |
|
| 230 |
+
| **romanian deadlift** | - | - | - | 146 | - | - | - | 6 | Pending |
|
| 231 |
+
| **russian twist** | - | - | - | 137 | - | - | - | 6 | Pending |
|
| 232 |
+
| **shoulder press** | - | - | - | 152 | - | - | - | 13 | Pending |
|
| 233 |
+
| **squat** | - | - | - | 238 | - | - | - | 15 | Pending |
|
| 234 |
+
| **t bar row** | - | - | - | 117 | - | - | - | 10 | Pending |
|
| 235 |
+
| **tricep Pushdown** | - | - | - | 93 | - | - | - | 12 | Pending |
|
| 236 |
+
| **tricep dips** | - | - | - | 220 | - | - | - | 9 | Pending |
|
| 237 |
+
| **Overall Accuracy** | - | - | - | **2743** | - | - | - | **233** | **Pending** |
|
| 238 |
+
| **Macro Average** | - | - | - | **2743** | - | - | - | **233** | **Pending** |
|
| 239 |
+
|
| 240 |
+
---
|
| 241 |
+
|
| 242 |
+
## Table 11: Computational Complexity & Inference Latency (Paper Table 11)
|
| 243 |
+
|
| 244 |
+
*Objective:* Measure parameters, window FLOPs, and latency across server and edge devices.
|
| 245 |
+
*Execution Command:* `python scripts/benchmark_hardware_latency.py`
|
| 246 |
+
|
| 247 |
+
| Model Architecture | Parameters | FLOPs per Window | RTX PRO 6000 (CUDA) | Apple M4 (MPS) | Apple M4 (CPU) | Status |
|
| 248 |
+
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 249 |
+
| **Transformer Mix v2 (63-d)** | ~300K | - | - | - | - | Pending |
|
| 250 |
+
| **AAGCN Joint Stream** | ~378K | - | - | - | - | Pending |
|
| 251 |
+
| **AAGCN Bone Stream** | ~378K | - | - | - | - | Pending |
|
| 252 |
+
| **AAGCN Joint-Motion Stream** | ~378K | - | - | - | - | Pending |
|
| 253 |
+
| **AAGCN Bone-Motion Stream** | ~378K | - | - | - | - | Pending |
|
| 254 |
+
| **SkelGym-Lite (Transformer + Bone)** | ~678K | - | - | - | - | Pending |
|
| 255 |
+
| **SkelGym-Full (Transformer + 4 AAGCN)** | ~1.81M | - | - | - | - | Pending |
|