Buckets:
| # AIDE head-to-head protocol (locked 2026-09-10, Gate 0 passed) | |
| Everything measured on the AIDE track is defined by `aide/data/protocol.py`; this note records | |
| what was locked, the evidence that the locked choice reproduces the published Driver-WM | |
| (arXiv 2605.05092v2) zero-velocity numbers, and the assumptions we have had to make where the | |
| paper does not specify a detail. Numbers below are copied from script outputs | |
| (`aide/results/cache/verify_protocol.json`, `aide/results/tables/analytic_baselines.md`, | |
| `aide/results/tables/results.csv`); nothing is hand-computed. | |
| ## 1. Data and splits | |
| | item | value | source | | |
| |---|---|---| | |
| | clips | 2,898 local AIDE clips, 4 views, `annotation/<id>.json` HALPE-136 `pose_list` (incar view) | `aide/results/cache/cache_summary.json` | | |
| | official split | train 1,884 / val 405 / test 609, `aide/data/splits_{training,validation,testing}.csv` from github.com/ydk122024/AIDE; local ∩ official = 100 % | `cache_summary.json` (`missing: []` for all three) | | |
| | clip length | 45 frames @ 15 fps for 2,885 clips; 13 clips are longer (60–345 frames) and are sampled with the same `linspace(0, n−1, 10)` rule | `cache_summary.json: long_clip_lengths` | | |
| | keypoint cache | `aide/results/cache/aide_kp.npz` — `S [N,45,136,3]` (all frames, training windows), `S10 [N,10,136,3]` (protocol frames), `S10_alt` (sensitivity variant), labels, bboxes; sha256 digest `fb49ec33…4cea998f9` (recorded in every run json) | `run_build_cache_latest.json` | | |
| ## 2. Locked protocol constants | |
| | constant | value | why | | |
| |---|---|---| | |
| | `T_OBS / T_PRED` | 5 / 5 of 10 uniformly sampled frames | paper Sec. 4 | | |
| | `SAMPLE_GRID` | `floor` = `np.linspace(0, n−1, 10).astype(int)` → 0,4,9,14,19,24,29,34,39,44 for n = 45 | only grid reproducing the paper's ZV All / HM (the `round` grid gives 53.17 / 136.17) | | |
| | `DRIVER_RULE` | `first` — the driver is `result[0]` of each frame; frames without detections are forward-/back-filled | reproduces all five published ZV numbers; equals the largest box in 98.8 % of frames | | |
| | MPJPE | mean over clips × horizons × **all 136 joints** of the Euclidean pixel error, no confidence mask | paper protocol; a conf ≥ 0.1 variant is reported as a secondary metric (`mpjpe_conf0.1`) | | |
| | d-nMPJPE | 100 · MPJPE / diag(1920, 1080) = MPJPE / 22.029 | paper's 2.40 for ZV 52.89 | | |
| | PCK@t | fraction of (clip, horizon, joint) with error ≤ t · **max(W, H)** = 96 px @0.05 | reproduces 85.95; t · diag gives 87.67 (kept as `pck05_diag`) | | |
| | High-Motion (HM) subset | top ⌊0.1 N⌋ clips (60 on test, 40 on val) by mean GT displacement over the **4 future transitions** (frames 5→6…8→9) and all joints; threshold on test 72.05 px | reproduces the paper's HM 139.19 / threshold 72.05 | | |
| | HM h=5 | per-horizon MPJPE on the HM subset at the last horizon | paper quotes ZV 178.62 / Driver-WM 155.82; ours ZV 178.67 | | |
| | CIs | 10,000 clip-level bootstrap resamples; paired per-clip differences vs zero-velocity and named reference runs | `aide/eval/metrics.py`, `harness.py` | | |
| ## 3. Gate 0 — reproduction evidence (official test split, zero-velocity) | |
| | metric | paper | ours (locked protocol) | tolerance | pass | | |
| |---|---|---|---|---| | |
| | All-MPJPE | 52.89 | 52.929 | 0.5 | yes | | |
| | d-nMPJPE | 2.40 | 2.403 | 0.03 | yes | | |
| | HM-MPJPE | 139.19 | 139.228 | 2.0 | yes | | |
| | HM threshold (px) | 72.05 | 72.052 | 2.0 | yes | | |
| | PCK@0.05 | 85.95 | 85.948 | 1.0 | yes | | |
| Sensitivity (48 rows in `verify_protocol.json`, 4 driver rules × 3 grids × 2 HM definitions × 2 PCK | |
| normalisers): the brief's original "largest-bbox + round" rule gives 48.64 / 107.93 (not the paper); | |
| `max_score` and `iou_track` do not reproduce either. The `largest_bbox` + `round` combination is kept | |
| as `--variant alt` (`S10_alt`) for sensitivity rows only. | |
| ## 4. Evaluation contract | |
| * A run is a directory `aide/results/runs/<name>/` with `pred_test.npz` (`pred [N,5,136,2]` px, | |
| `clip_id [N]`; optional `logits_<task>`), optionally `pred_val.npz`. The harness aligns rows by | |
| `clip_id`, requires the exact split clip set, writes `metrics_<split>.json` (append-only: a second | |
| evaluation gets a timestamped name) and appends rows to `results/tables/results.csv` under a file lock. | |
| * Ground truth = frames 5..9 of `S10`; the model side only ever receives frames 0..4 | |
| (`train_kp.split_obs_fut` is the single cut point; `aide/tests/test_no_future_leak.py` and | |
| `test_cwm3d_zero_init.py` NaN-poison the future block and require bit-identical predictions and | |
| rejection of 10-frame inputs). | |
| * Model selection: val All-MPJPE of the EMA weights; test is evaluated once per run. | |
| * Training windows come from the 45-frame `S` array (sub-window augmentation, gaps ∈ {4,5}); val and | |
| test use the protocol frames only. | |
| * Every run writes `run_<name>_<utc>.json` (`common/runmeta.write_run_json`: command, code sha256, | |
| input digest, seed, config, wall/GPU minutes, val/test headline numbers). | |
| ## 5. Analytic bars (test, locked protocol) — from `analytic_baselines.md` | |
| | predictor | All | HM | note | | |
| |---|---|---|---| | |
| | zero-velocity | 52.93 [48.88, 57.25] | 139.23 [116.93, 167.68] | paper 52.89 / 139.19 | | |
| | anti-velocity 0.1 | 52.53 | 138.55 | Δ vs ZV −0.40 [−0.82, +0.13] n.s. | | |
| | ZV-shrink α = 0.44 (α fitted on **val**) | 51.54 [47.46, 55.97] | 134.36 [111.80, 162.87] | the honest parameter-free bar; Δ vs ZV All −1.39 [−2.66, +0.26], HM −4.86 [−7.67, −1.95] | | |
| | paper Driver-WM | 71.47 | 138.03 | PCK@0.05 71.66, HM h=5 155.82 | | |
| | paper static pooling | – | 134.56 | best published HM | | |
| Gate A (plan §4): CWM3-D keypoint-only must reach test All < 52.89 and HM < 134.56 **and** beat the | |
| val-fitted ZV-shrink by ≥ 1 px All and ≥ 3 px HM (3 seeds). The driver `aide/train/run_stage_a.py` | |
| applies the same rule on the **val** split (bootstrap-free) before spending GPU on seeds 123/456. | |
| **Stage A outcome (2026-09-10, `aide/results/tables/stage_a.md`)**: CWM3-D-kp test All 48.63 ± 0.15 / | |
| HM 131.64 ± 0.76 / PCK@0.05 87.04 / HM h=5 163.37 over seeds 42/123/456; paired Δ vs ZV-shrink | |
| −2.91 [−4.51, −1.72] All and −2.72 [−5.31, −0.04] HM; val margin ≥ 1 / ≥ 3 px on every seed → **Gate A | |
| PASS**, with the caveat that the test-split HM margin is 2.72 px and the HM gain over SiMLPe-kp (same | |
| recipe, 50.26 / 131.22) is nil (+0.41 [−1.48, +2.26]). Every learned keypoint-only model is far below | |
| Driver-WM's published 71.47 / 138.03. Details and ablations: `docs/decisions.md` rows of 2026-09-10. | |
| ## 6. Driver-WM reimplementation assumptions (running list) | |
| The paper releases no code. Wherever we had to choose, the choice is listed here and will be | |
| carried into `driverwm_repro.py` and the tables ("paper number" and "our reproduction" are always | |
| shown side by side; SOTA claims are made against the paper numbers). | |
| Protocol / data | |
| 1. Driver identity in multi-person frames = first detection of `pose_list[t].result` (Sec. 2). The | |
| paper's Table 1 is consistent only with this rule; the semantically better largest-box rule is a | |
| sensitivity variant. | |
| 2. Frames without a detection (1 frame in the whole set) are carried over from the previous frame. | |
| 3. Uniform sampling of 10 frames uses the floor grid (Sec. 2); the 13 long clips are sampled over | |
| their full length. | |
| 4. HM uses the 4 future transitions only (the 5-transition variant incl. frame 4→5 is a sensitivity row). | |
| 5. PCK normaliser = max(W, H); d-n normaliser = image diagonal. | |
| 6. Label strings are normalised (`happiness`→`Happiness`, `weariness`→`Weariness`, `Forward`→`Forward Moving`); | |
| the seven-class DBR order and the other class orders follow the paper's tables. | |
| Visual features (Driver-WM App. B.2) | |
| 7. Backbone `Qwen3-VL-2B-Instruct` (local weights), bf16, sdpa attention, frozen. | |
| 8. Prompt = the paper's App. B.2 text verbatim (`protocol.QWEN_PROMPT`). | |
| 9. Feature = mean over the **full** last-layer hidden sequence (`model.model(...).last_hidden_state`, | |
| post-norm), 2048-d; a "visual tokens only" mean is stored as a variant. The paper does not say | |
| which tokens are pooled. | |
| 10. Images at native 1920×1080 (≈ 2,110 visual tokens/frame); one frame per (clip, sampled index, view), | |
| i.e. 10 × 4 frames per clip, stored fp16. | |
| Model (to be used in `driverwm_repro.py`, not yet run) | |
| 11. Identity latent interface on the 2048-d features + view embedding; 1 causal temporal | |
| self-attention layer with 4 heads; Perceiver-style context module with 4 latent queries; | |
| gate = σ(MLP). | |
| 12. Deterministic 5-step mean rollout (closed loop) for the future latents. | |
| 13. ST-GCN skeleton decoder with 2 layers and sigmoid output scaled to (W, H); keypoints with | |
| conf < 0.1 masked in the training loss (evaluation always uses all joints). | |
| 14. Losses L_lat (velocity) + L_skel + L_aux + L_phys with λ_lat = λ_phys = 0.1; AdamW 5e-5, bs 32, | |
| 100 epochs, seeds 42/123/456. | |
| 15. The in-cabin view is the seat ROI crop only if the paper's Fig./B.2 is read that way; we start with | |
| the full frame and record the alternative. | |
| Model / training, continued (decided for `aide/models/driverwm_repro.py`, Stage C) | |
| C10. Skeleton decoder outputs ABSOLUTE normalised coordinates through a sigmoid (Eq. 8), no residual | |
| anchor (dm_bench's "residual convention" is not used): the reproduction therefore does not start at | |
| zero-velocity, unlike our models. | |
| C11. ST-GCN = Linear(2048 → 136 × 32) followed by 2 blocks of {spatial graph conv over the normalised | |
| HALPE-136 adjacency of `protocol.EDGES_ALL` (+ self loops), temporal conv k = 3 over the 5 rolled-out | |
| steps, residual, LayerNorm} and Linear(32 → 2). | |
| C12. Perceiver context (App. B.6): mean-pooled internal history → MLP → 4 latent queries in a 256-d | |
| bottleneck; keys/values = 256-d projections of the external prefix; output → Linear → 2048 added to the | |
| last internal state (= m_t). Gate input = the current (rolled-out) pooled external latent ê_t (Eq. 18). | |
| f_int / f_ext = residual MLPs 2048 → 4096 → 2048; internal and external pre-encoders = one pre-norm | |
| TransformerEncoderLayer(2048, 4 heads, d_ff 4096) each with a causal mask and learned step embeddings. | |
| C13. Losses exactly as written: L_skel Eq. 23 (per-frame conf ≥ 0.1 mask, squared error in normalised | |
| coordinates, summed over the 5 steps), L_lat^vel Eq. 14 (squared norm over the 2048 dims, summed over | |
| the 5 velocities from the last observed latent; targets = the same pre-encoder run causally over the | |
| 10-frame in-cabin feature sequence, detached), L_bone Eq. 25 (body + hand edges), L_smooth Eq. 26 over | |
| [GT last observed; predicted 1..5]; total = 0.1 L_lat + L_skel + L_aux + 0.1 (L_bone + L_smooth). The | |
| literal sums make the latent term dominate the transition core; `--repro-lat-norm mean` (÷ 2048) is run | |
| as a declared port-sensitivity row, not as the paper. | |
| C14. Training: AdamW 5e-5, weight decay 1e-5, cosine to 0 without warm-up, batch 32, 100 epochs (paper: 80), | |
| no early stopping, no augmentation, no EMA; protocol frames only; selection on val All-MPJPE (ours). | |
| C15. Interventions (Table 3): do(Ext = ∅) zeroes the exterior features of the 5 observed frames at the | |
| input; do(Ext = swap_clip) replaces them by another test clip's (fixed derangement, seed 0); | |
| do(λ_CA = 0 / 1) clamps the gate. Deviations are pixel distances between intervened and factual | |
| rollouts (All / HM / h=5 / Head = face + body joints 0–4, 17, 18 / Hands); GT errors also reported. | |
| C16. Parameter count of the reproduction: 123.7 M (dominated by the 2048-d attention/MLP blocks; the paper | |
| gives no count). | |
| CWM3-D-vis / semantic choices (ours, Stage B) | |
| S1. Visual tokens: per observed frame and view, LayerNorm(2048) → Linear(2048 → 256) + view embedding + | |
| step embedding, inserted next to the 4 part tokens (same-frame fully connected, cross-frame causal); | |
| modality dropout 0.3 zeroes the whole visual block of a clip during training. | |
| S2. Training sub-windows use the feature of the NEAREST protocol frame (features exist for the 10 protocol | |
| frames only; offset ≤ 2 frames = 0.13 s); val/test use exact protocol frames. | |
| S3. Semantic heads: DBR / DER from mean(driver part tokens over the observed window) + mean(in-cabin visual | |
| tokens); TCR / VCR from mean(exterior visual tokens); LayerNorm + Linear; CE with label smoothing 0.1 and | |
| inverse-frequency class weights (train split, normalised to mean 1), weight 0.5 per head; observed window | |
| only, labels never enter the geometry path. Without vision (variant ii) all four heads read the driver tokens. | |
| S4. Optional traffic → driver gated cross-attention (variant iii): last-frame driver tokens attend to the | |
| exterior visual tokens of all observed frames; gate = sigmoid(MLP) initialised at 0.018 (near identity). | |
| S5. Selection remains val All-MPJPE; F1 / accuracy are reported at that checkpoint (no semantic selection). | |
| S3b. Variant (v): semantic heads read the frozen 2048-d features directly — DBR/DER = Linear([LN(mean in-cabin | |
| feature over the observed window), mean driver part token]), TCR/VCR = Linear(LN(mean exterior feature)). | |
| S6. Ego-motion token (variant vi): per observed frame, 16 Farneback flow statistics × 3 exterior views (320×180 | |
| grayscale, flow between consecutive protocol frames; step 0 copies step 1), standardised with train-split | |
| mean/std stored as model buffers, Linear(48 → 256) + step embedding as one extra token per frame; the TCR/VCR | |
| direct heads also receive the window-mean of the standardised statistics. Interventions zero / swap it with the | |
| exterior features (`zero_ext_ego`, `swap_ext_ego`). | |
| **Stage B/C outcome (2026-09-10, `aide/results/tables/stage_bc.md`)**: CWM3-D vis+sem test All 48.36 ± 0.17 / | |
| HM 130.28 ± 0.86 / PCK@0.05 87.14 / HM h=5 161.93 ± 0.84 (Δ vs keypoint-only −0.49 / −0.20 / −0.52 px All per seed); | |
| semantic Macro-F1 43.2 / 45.1 / 82.3 / 59.8 → **Gate B FAIL** on all four heads (direct-feature heads 48.9 / 48.2 / | |
| 78.7 / 60.3; + ego-motion token 43.0 / 47.7 / 78.8 / 62.9). **Gate C FAIL vs the paper** (HM h=5 161.9 vs 155.82), | |
| PASS vs our reproduction (194.3). Driver-WM reproduction (C1–C16): All 130.86 ± 3.11 / HM 176.33 / PCK@0.05 54.2 — | |
| geometry not reproduced — but semantics reproduced (F1 71.7 / 68.7 / 89.6 / 68.4; Acc 73.6 / 78.2 / 93.0 / 82.3). | |
| Interventions and per-horizon curves: `stage_bc.md` §3–4; decisions: `docs/decisions.md` rows of 2026-09-10. | |
| S7. Attention-pooling semantic heads (variant vii/viii, `--sem-input attn`): per stream one pre-norm causal | |
| TransformerEncoderLayer(2048, 4 heads, d_ff 4096) over the observed-window features (in-cabin; exterior = | |
| view-embedded mean of the 3 outside views), last state → LayerNorm → Linear; DBR/DER heads also receive the | |
| driver part-token mean; TCR/VCR the window-mean ego statistics when the ego token is on. Plain CE, λ 1, | |
| no class weights, no label smoothing. | |
| S8. Dual selection (`--dual-select`): besides the geometry-selected checkpoint (val All-MPJPE), the checkpoint | |
| with the best val mean macro-F1 is saved and evaluated as a separate run `<name>__f1sel`; its rows are | |
| labelled "DUAL SELECTION" and always carry their own geometry. Selection never touches the test split. | |
| S9. Stage-C sweep flags: `--h-weights 1,1,1,2,3` (horizon weights on the Euclidean + NLL terms, normalised to | |
| mean 1), `--motion-weight 3` (per-clip weight = min(GT future displacement / batch mean, 3), renormalised), | |
| `--d 384 --ffn 1536 --head-hidden 768`, `--layers 6`; `aide/eval/ensemble.py` averages seed predictions. | |
| **Follow-up outcome (2026-09-10, `stage_bc.md` Gate-B blocks and "Stage C late-horizon sweep")**: with the | |
| repro-recipe heads the single geometry-selected checkpoint reaches F1 66.9 / 58.3 / 88.7 / 66.7 (geometry | |
| 48.51 / 130.94, unchanged); the val-F1-selected checkpoint reaches 72.2 / 64.0 / 89.8 / 69.2 at +1.0–1.4 px All; | |
| with the ego-motion token VCR 72.0 (a) / 72.8 (b). **Gate B FAIL under both readings** (DER −8.6 to −14; TCR | |
| −0.3 under (b)). Stage-C sweep: HM h=5 160.7 (motion weights) … 162.1; ensemble 161.5; **none < 155.82 → Stage C | |
| FAIL vs the paper, PASS vs our reproduction (194.3)**. | |
| S10. Per-view attention + learned view gate (variant DER-i, `--sem-view-gate`): all four views pass through the | |
| shared in-cabin causal encoder (+ view embedding); per head a learned softmax over the 4 last states (logits | |
| initialised 0, reported in the run json `model_report`); DBR/DER heads also take the driver part-token mean. | |
| Learned gates ended nearly uniform (≈ 0.23–0.28), i.e. the gain is multi-view pooling, not view selection. | |
| S11. `--sem-frame-avg der` = frame-level DER logits averaged over the 5 OBSERVED frames (never the future); | |
| `--sem-task-weights der=2` = per-head CE multiplier; `sem_only` model = frozen features → S7 heads only, | |
| selected on val mean macro-F1 (its `pred` is the zero-velocity copy so the harness contract holds). | |
| **Closing outcome (2026-09-11, `stage_bc.md` "DER follow-up" + "Final AIDE verdict table")**: semantic-only | |
| reference DER 64.7 ± 0.2; view-gated heads DER (a) 73.1 ± 1.9 / (b) 74.7 ± 0.4 → full Gate B with this head: | |
| (a) single geometry-selected checkpoint FAIL on DBR by 0.5 (67.6 / 73.1 / 91.4 / 69.9), (b) dual selection PASS | |
| on all four seed means (71.1 / 74.7 / 90.3 / 70.3; TCR by 0.14, 1/3 seeds) at +1.4 px All. Gate A PASS, Stage C | |
| FAIL vs 155.82 (HM h=5 162.8 [119.4, 221.7], 60 clips, ±51 px) / PASS vs our reproduction. Track closed. | |
| CWM3-D-specific choices (ours, documented for reproducibility) | |
| 16. Part tokens: 4 per frame (body 0–25, face 26–93, lhand 94–114, rhand 115–135); hands share the | |
| tokenizer + a side embedding; per-joint channels (Δx, Δy)/200 to the last observed pose, x/W, y/H, conf. | |
| 17. Anchor mixture `pred = last + gate ⊙ (Σ_k softmax(logits)_k · a_k − last) + 200·μ` over anchors | |
| {last, mean(obs), last − 0.1·(last − prev)}; head output layer zero-initialised so gate = μ = logits = 0 | |
| and the model is **exactly** zero-velocity at initialisation (`test_cwm3d_zero_init.py`, tolerance 1e-6 px). | |
| The linear gate is what makes a softmax mixture reducible to ZV at zero init (a pure softmax would | |
| start at the anchor mean). | |
| 18. Loss = Euclidean px error + 0.1 isotropic Laplace NLL (b₀ = 30 px) + 0.05 bone length over the | |
| AlphaPose body + hand edges + 0.01 jerk of [last, pred₁..₅]. Recipe: AdamW 3e-4, wd 0.05, | |
| cosine with 10 warm-up epochs, ≤ 200 epochs, bs 32, EMA 0.999, grad-clip 1.0, patience 40. | |
| 19. Augmentation (train only): 10-frame sub-windows with gaps in {4,5} (protocol grid with p = 0.5), | |
| keypoint noise σ = 2·(1 − conf) px, joint dropout 0.1 (xy → last observed, conf → 0), translate | |
| ±20 px and scale ×[0.95, 1.05] about the last-observed centroid (all 10 frames); no flip. | |
| 20. The SiMLPe-kp baseline (`simlpe_kp_*_v2`) is trained with the identical recipe and augmentation | |
| (plain Euclidean loss) so the comparison isolates the architecture. | |
| Hardening (roadmap supplement B §B2, 2026-09-12; `docs/aide_hardening_report.md`, `stage_bc.md` §1b/1c/4b/8) | |
| C17. Shrink-curve analysis and the h=5 trade-off (`aide/eval/baselines_analytic.py`, `tradeoff_all_vs_hm.md`, `pareto_all_vs_hm.png`). | |
| Two parameter-free families are evaluated besides the val-fitted ZV-shrink (α = 0.44): (i) `mean_pose_shrink_a`, | |
| pred_h = (1 − a)·last + a·μ with μ = TRAIN-split mean pose of protocol frame 4 (per joint), a ∈ {0.1, 0.2, 0.3} fixed a | |
| priori; (ii) `zv_shrink_h_val`, α_h = a·h/5 towards the observed mean with a = 0.76 fitted on val All. The curves in the | |
| figure are evaluated on the test split for description only (nothing is selected on them). Result: HM h=5 and All are | |
| anti-correlated along the mean-pose curve on both splits. Driver-WM's All / HM (71.47 / 138.03) sit on the shrinkage | |
| curve, but its HM h=5 does not: on TEST a = 0.3 gives 65.78 / 134.09 / 166.00 and the family bottoms out at HM h=5 | |
| 163.74 (a = 0.50, All 81.53), 8.5 px above 155.82 at the paper's All; the val curve (a = 0.3 → 69.76 / 134.44 / 155.98) | |
| does not transfer to test, so shrinkage does NOT reproduce the paper's 155.82. The 5–7 px h=5 gap between the paper and | |
| our best (160.7) is unexplained by shrinkage and unresolvable at ±51 px (n = 60 HM clips; single paper seed). | |
| Honesty clause: had `zv_shrink_h_val` beaten CWM3-D-kp on test HM, the Gate-A HM caveat would have become a limitation; | |
| it did not (132.34 vs 131.64, 0.7 px) — the caveat stands as a caveat. HM d-nMPJPE = 100·HM/diag is now a reported | |
| column (ZV 6.320 = paper 6.32; `test_metrics.py`). | |
| C18. Full Driver-WM Table-3 mirror (`aide/eval/interventions.py`, `stage_bc.md` §4b). Operators on the OBSERVED window only: | |
| swap (fixed derangement), remove ext (zero), shift large / small (exterior features delayed 2 / 1 protocol steps, | |
| edge-padded — frame t receives frame max(t − k, 0)), drop view (one exterior view zeroed; mean and max over the three | |
| views reported), `ext_scale` λ ∈ {0, 2} (exterior visual TOKENS scaled before the trunk; λ = 1 is bit-identical to the | |
| factual pass, unit-tested), `gate_override` λ_CA ∈ {0, 1, 2} for the traffic-gate variant (iii; seeds 123 / 456 trained | |
| so it is a 3-seed row) and the reproduction's gate. Deviations are reported in px and as Δ/All (÷ the model's factual GT | |
| All-MPJPE) because pixel deviations are not comparable at All 48 vs 71; the GT error of every intervened rollout and its | |
| paired difference to the factual rollout are reported (2k clip bootstrap). CWM3-D has no gate: "disable pathway" and | |
| "over-injection" are mirrored by λ = 0 / 2 on the exterior tokens (analogue), "force" is undefined for it. | |
| C19. Fair keypoint baselines on the shared recipe (hardening D; `aide/models/motionbert_kp.py`, `aide/models/stgcn_kp.py`; 3 seeds each, | |
| Stage-A recipe + augmentation, plain Euclidean loss, val-selected EMA weights). MotionBERT-kp = DSTformer-style alternating | |
| spatial (over the 136 joints of a frame) / temporal (over the 5 observed frames of a joint) pre-norm attention, 3 blocks, d 128, | |
| 4 heads, temporal Linear(5 → 5) read-out, zero-initialised Linear(d → 2) residual over the last observed pose (813,088 params). | |
| ST-GCN-kp = Linear(5 → 128) per joint + 8 `STGCNBlock`s taken unchanged from the Driver-WM reproduction's skeleton decoder | |
| (normalised HALPE-136 adjacency `EDGES_ALL`, temporal conv k = 3, residual, LayerNorm), same read-out (530,080 params). Both are | |
| exactly zero-velocity at initialisation and are pure functions of the observed block (`aide/tests/test_hardening.py`). | |
| Result (test, 3 seeds): MotionBERT-kp 48.38 ± 0.10 / d-n 2.196 / HM 129.91 ± 0.65 / HM d-n 5.897 / PCK@0.05 87.15 / HM h=5 162.93; | |
| ST-GCN-kp 49.22 ± 0.18 / 2.234 / 130.15 ± 0.52 / 5.908 / 86.86 / 162.97; vs CWM3-D-kp 48.63 ± 0.15 / 131.64 ± 0.76 and the | |
| published MotionBERT 73.51 / 141.53, ST-GCN 110.98 / 158.10. Consequence for the paper: the ≈ 25 px gap between the published | |
| motion baselines and our keypoint models is the training recipe (residual-over-last-pose parametrisation, zero-init output, | |
| sub-window / noise / joint-drop augmentation, val-selected EMA), which all our keypoint models share; the CWM3-D architecture | |
| adds ≈ 0 px over MotionBERT-kp on All (+0.25) and is 1.7 px worse on HM. Geometry SOTA claims are therefore made for the | |
| recipe-plus-model family, not for the part-token architecture alone. | |
| C20. Hardening variants and their pre-registered kill rules (roadmap B §B2; all decided on the VALIDATION split; test in | |
| `stage_bc.md` §1 / §8; 3 seeds each; recipe and augmentation of the parent row). | |
| A1 `--patience-f1 40` (view-gate heads, dual selection): early stop only when val All AND val mean macro-F1 have both stalled | |
| for 40 epochs. Rule: kill if no seed's best val F1 exceeds its old run's maximum by > 0.3. Result: +0.00 / +0.44 / +0.00 | |
| (epochs 110 / 149 / 135 vs 89 / 102 / 95; the shared epochs are bit-identical) → survives on the rule, but the extra | |
| epochs change no test number: (a) is the same geometry checkpoint as DER (i) (a); (b) 70.9 / 74.5 / 90.4 / 70.1 vs 71.1 / | |
| 74.7 / 90.3 / 70.3 at All 49.83 vs 49.71. Reported as "no effect". | |
| A2 `--sem-view-gate --egomotion`: view-gate heads + the ego-motion token (S6). Rule: kill if val mean-F1 (b) < 78.2 on 2/3 | |
| seeds or val All (b) > 57 on 2/3. Result: val F1 (b) 76.57 / 77.97 / 76.62 (3/3 below) → KILLED (test (b) 70.9 / 73.5 / | |
| 90.9 / 71.9 at All 50.88, for the record). | |
| A3 logit ensemble of the three DER (i) (b) seeds (`aide/eval/ensemble.py`, mean of logits and coordinates): a separate | |
| "ensemble, not a single model" row (test F1 73.4 / 77.3 / 91.2 / 72.3; All 48.82 / HM 131.23). | |
| B1 (i) `--motion-weight 5 --h-weights 1,1,1,2,3`; (ii) `--motion-oversample 2` (TRAIN clips drawn with replacement per | |
| epoch, p_i ∝ 1 + 2·s_i/mean(s), s_i = GT future displacement on the clip's protocol frames). Rule: kill if val HM h=5 ≥ | |
| 144.5 on 2/3 seeds or val All > 56. Result: (i) 143.5 / 145.2 / 144.9, (ii) 145.8 / 144.9 / 146.3 → both KILLED; their | |
| test HM h=5 (160.71 ± 0.38 and 161.04 ± 0.47) are the lowest single-model values on record but are not selected on. | |
| B2 `--dct` (μ parametrised by DCT-II coefficients over the 5 horizons, heads.py). Rule: kill if val All worse than the | |
| same-seed B2 main run by > 0.3 on 2/3 seeds. Result: +0.23 / +0.29 / +0.14 → survives (test 48.49 / 130.73 / 162.62 vs | |
| 48.36 / 130.28 / 161.93: no gain). | |
| B3 `--refine-late`: zero-initialised MLP on [last-frame part tokens, detached (h ≤ 3 prediction − last)/200] adding Δ to | |
| h = 4, 5 only (identity at init, bit-identical to the parent; `test_hardening.py`). Same rule as B2. Result: +0.30 / | |
| −0.03 / −0.29 → survives (test 48.34 ± 0.16 / 130.26 ± 0.08 / HM h=5 161.37 ± 0.55 vs 161.93 ± 0.84: within noise). | |
| B4 ensembles: 3 motion-weighted seeds (test 48.54 / 129.73 / HM h=5 160.26) and 9 runs mw + hw + dct (48.23 / 129.83 / | |
| 161.08); separate rows, never single models. | |
| Hybrid `cwm3d_kp_spatial` (coordinator request): CWM3-D-kp with two DSTformer-style spatial attention blocks (d_s 32, | |
| 4 heads) over the 136 joints of each observed frame in front of the part tokenizer (attended features concatenated to the | |
| raw 5 channels; anchor mixture kept; 6.74 M params; `aide/models/cwm3d_spatial.py`). Adoption rule: paired Δ HM vs the | |
| SAME-seed CWM3-D-kp with CI excluding 0 on ≥ 2/3 seeds and All not worse. Result: Δ HM −2.79 [−5.09, −0.62] / −1.45 | |
| [−3.21, +0.16] / −1.49 [−3.09, +0.08] (1/3), Δ All −0.07 / −0.01 / +0.03 (never worse) → NOT adopted; reported as a row | |
| (test 48.61 ± 0.11 / 129.73 ± 0.14 / HM h=5 161.12 ± 0.70). | |
| Killed on paper, not run: temperature scaling (argmax-invariant), flip TTA (item 19; asymmetric exterior views, unflipped | |
| features, left/right classes), trunk-state semantic heads (falsified), constant-velocity anchor (worse at every gain), | |
| wider / deeper / EMA (already run), train + val (breaks val-only selection and the val-fitted analytic bars). | |
| C21. BATON → AIDE in-cabin pose pretraining (reverse transfer; GPU-baselines agent, 2026-09-12; table `aide/results/tables/baton_pretrain.md`). | |
| Corpus (`aide/features/build_baton_pose_corpus.py` → `aide/results/cache/baton_pose_corpus.npz`, an index over the route npz): the 440 BATON TRAIN-split | |
| routes of the 96 train-only devices (the s12 splits are device-disjoint — 96 train vs 66 val/test devices, 0 overlap, asserted; no val / test route or device | |
| enters), HALPE-136 RTMW keypoints at 10 Hz on the causal driver box (`results/pose_causal_routes_v2` for the 11 re-extracted train routes, v1 for the rest); | |
| 3,410,103 frames, 68.7 % valid (driver detected, finite nose / neck / box, ≥ 80 % of body AND of face joints with score ≥ 0.3, both shoulders ≥ 0.3); | |
| 10-frame windows at 0.3 s steps (3 × 10 Hz; AIDE grid 0.267 / 0.333 s, span 2.7 vs 2.93 s — the closest realisable stride), anchors every 0.3 s, contiguous | |
| time required → 642,179 windows (< the 1 M cap), 22 held-out routes for the pretraining early stop. Coordinates: per-window isotropic similarity into the | |
| AIDE pixel frame (window-median shoulder width → AIDE-train median 309 px, shoulder centre → AIDE-train median (373, 762) px; a first box-height scaling left | |
| the shoulders at 201 px and was replaced before any GPU time); HALPE 17 (head-top, absent from RTMW) = nose + 0.65 (nose − neck); RTMW scores quantile-mapped | |
| per part to the AIDE-TRAIN score distribution. Residual domain gap: BATON future displacement median 13.3 px vs AIDE 29.1 px (real driving vs | |
| behaviour-selected clips). AIDE side: TRAIN-split statistics only; AIDE test never touched except by one evaluation per finished run. | |
| Pretraining (`aide/train/pretrain_cwm3d_baton.py`, run `cwm3d_kp_batonpre_pretrain_s42`): CWM3-D-kp (5.77 M, same init as cwm3d_kp_s42_v2), AIDE task and | |
| loss (Euclidean + 0.1 NLL + 0.05 bone + 0.01 jerk), train_kp augmentation, AdamW 3e-4 / wd 0.05, bs 256, 6 epochs, EMA; selection on the held-out BATON | |
| routes (best epoch 4: All 27.19 px); 16 ms/step, 4.0 GPU-min. The AIDE-val zero-shot curve is diagnostic only (58.6 → 60.2 px over the epochs; ZV 61.16). | |
| Zero-shot run `cwm3d_kp_batonpre_zeroshot_s42_v2` (no AIDE training): test All 52.75 / HM 136.75 / PCK 85.90 / HM h=5 170.96 (ZV 52.93 / 139.23). | |
| Fine-tuning: `aide/train/train_kp.py --init-ckpt <ckpt_best.pt> --init-parts all` (opt-in flag; default off = unchanged trainer) with the Stage-A / B2 recipes | |
| verbatim; runs `cwm3d_kp_batonpre_s{42,123,456}_v2`, `cwm3d_vis_sem_batonpre_s{42,123,456}_v2`; paired per-clip bootstrap vs `cwm3d_kp_s<k>_v2` / | |
| `cwm3d_vis_sem_s<k>_v2` (n = 609, 10k). **Pre-registered rule (adopt only if Δ All CI excl. 0 on ≥ 2/3 seeds, improving): NOT met.** kp: Δ All −0.31 ± 0.02 px | |
| (CIs cover 0 on 3/3 seeds), Δ HM −2.38 ± 0.40 (1/3 CI excl. 0), Δ HM h=5 −1.21 ± 0.16 (INCONCLUSIVE), PCK +0.11 (NOT_WORSE), left hand −1.3 px, right hand | |
| +0.4 px; vis+sem: Δ All −0.10 ± 0.25, HM −1.14 ± 1.33, HM h=5 −0.16 ± 0.77 (all INCONCLUSIVE). Reading: the pretrained init converges ~2× faster (best epoch | |
| 22–29 vs 44–58) and moves every keypoint-only cell in the improving direction, but never beyond seed / clip noise → reported as "no reliable transfer", not | |
| adopted; the B2 main model is unchanged. A trunk-only-init secondary arm (`cwm3d_kp_batonpre_trunk_s<k>_v2`) is appended to the same table when finished. | |
| C21 addendum (12:55 EDT): a POST HOC trunk-only-init arm (`--init-parts trunk`, head at zero init; `cwm3d_kp_batonpre_trunk_s<k>_v2`) meets the numeric | |
| criterion (Δ All −0.49 ± 0.14 px, 2/3 CIs excl. 0; Δ HM −2.61 ± 0.79, 3/3) but was added after the pre-registered arms' result, so it is exploratory. | |
| Pre-registered before running: an independent pretraining seed (123) with the same trunk-only fine-tunes (`cwm3d_kp_batonpre_trunkp123_s<k>_v2`); | |
| the trunk-only pretrained init is adopted iff its Δ All is BEATS again. Every arm keeps val-only selection and one test evaluation per run. | |
| C21 outcome (13:00 EDT): the pre-registered replication REPLICATED. Independent pretraining `cwm3d_kp_batonpre_pretrain_s123` (best epoch 4, held-out All | |
| 27.20 px, 1.8 GPU-min; zero-shot test All 53.56) → trunk-only fine-tunes `cwm3d_kp_batonpre_trunkp123_s{42,123,456}_v2` vs `cwm3d_kp_s<k>_v2`: Δ All | |
| −0.79 [−1.29,−0.30] / −0.62 [−1.17,−0.11] / −0.45 [−0.89,−0.02] = **−0.62 ± 0.17 px, 3/3 CIs excl. 0 (BEATS)**; Δ HM −2.95 ± 1.07 (2/3, BEATS); Δ HM h=5 | |
| −3.88 ± 0.85 (1/3 CI excl. 0, INCONCLUSIVE); body −0.54 ± 0.26 BEATS; hands −1.0 to −1.2 px and every horizon negative (INCONCLUSIVE). Test All 47.98 / | |
| 48.04 / 48.01 (mean 48.01 vs 48.63 seed init; HM 128.7 vs 131.6; HM h=5 159.5 vs 163.4). Rule → the trunk-only BATON-pretrained init of CWM3-D-kp is | |
| adopted as a reported model row ("CWM3-D-kp + BATON in-cabin pretraining (trunk)"), NOT as a replacement of the B2 main model (the vis+sem arm was | |
| full-init and INCONCLUSIVE; a trunk-only vis+sem arm was not run). Full-init (trunk + horizon head) does not transfer (Δ All −0.31, all CIs cover 0): the | |
| pretrained head encodes BATON's 2–3× smaller motion magnitudes; keeping the head at its zero init (exact zero-velocity start) is what makes the trunk | |
| features usable. Every arm: AIDE val-only selection, one test evaluation per run; no AIDE test clip and no BATON val / test device in any training. | |
| (C21 timestamps: replication registered/launched 12:35 EDT, outcome 12:41 EDT; the vis+sem and view-gate trunk-only arms registered 12:43 EDT and launched | |
| 12:44 EDT — the "12:55 / 13:00 / 13:1x" wall-clock stamps written above were mis-read by the agent; run json `t_start_utc` fields are authoritative.) | |
| C21 outcome 2 (13:08 EDT; the two arms registered 12:43 EDT): (i) vis+sem B2 main recipe with the trunk-only BATON init | |
| (`cwm3d_vis_sem_batonpre_trunkp123_s<k>_v2` vs `cwm3d_vis_sem_s<k>_v2`): Δ All −0.29 [−0.75,+0.16] / −0.53 [−0.99,−0.07] / −0.23 [−0.69,+0.22] = | |
| −0.35 ± 0.16 px (1/3 CI excl. 0) → **INCONCLUSIVE, not adopted**; Δ HM −1.76 ± 0.88, HM h=5 −2.32 ± 1.74 (INCONCLUSIVE); semantic F1 of the geometry | |
| checkpoint lower (DBR 36.7–41.3 vs 39.7–45.8). (ii) view-gate attention-head recipe, geometry-selected checkpoint (a) | |
| (`cwm3d_vis_sema_vg_batonpre_trunkp123_s<k>_v2` vs `cwm3d_vis_sema_vg_s<k>_v2`): Δ All −0.49 [−0.94,−0.05] / −0.57 [−1.04,−0.09] / −0.70 [−1.13,−0.26] = | |
| **−0.59 ± 0.10 px, 3/3 CIs excl. 0 → BEATS, adopted as a reported row**; Δ HM −2.58 ± 0.24 (2/3, BEATS); left hand −1.50 ± 0.90 (BEATS); HM h=5 −2.68 ± 0.65 | |
| (INCONCLUSIVE); test All 48.15 / 48.06 / 47.95 vs 48.64 / 48.63 / 48.65. Caveat to print next to that row: the geometry-selected checkpoint comes ~20 | |
| epochs earlier with the pretrained trunk (best epoch 33–35 vs 48–61), so its semantic macro-F1 is markedly lower (DBR 55–59 vs 67–68, DER 61–63 vs 72–75, | |
| TCR 87–91 vs 91–92, VCR 64–65 vs 70; descriptive, geometry checkpoint (a)); the `__f1sel` dual-selection companions were written as usual and are the | |
| (b) rows for the AIDE agent's tables — the geometry gain does not come for free on the semantic side at the (a) checkpoint. Reading across the four | |
| trunk-only arms: the BATON trunk transfers to the geometry of every keypoint path it was tested on (kp ×2 pretraining seeds, view-gate (a)) and is | |
| directionally positive but inconclusive on the B2 vis+sem recipe (class-balanced label-smoothed CE heads); the transfer is a trunk (feature) effect and is | |
| lost when the horizon head is also copied. | |
| C22. 5-seed extension of the two locked main models — POST HOC, pre-registered reporting rule (2026-09-13, decisions row of the same date written | |
| before any run). Seeds 789 and 1011 are added to CWM3-D-kp (`cwm3d_kp_s<k>_v2`) and CWM3-D vis+sem (`cwm3d_vis_sem_s<k>_v2`) with the recipes of | |
| the locked runs frozen verbatim (Stage-A recipe; vis+sem adds --visual --modality-dropout 0.3 --semantic --lambda-sem 0.5 --label-smoothing 0.1 | |
| --class-balanced 1), val-only selection per seed, one test evaluation per run. Reporting rule: the locked 3-seed rows (42 / 123 / 456) remain the | |
| headline numbers of stage_a.md / stage_bc.md; the 5-seed per-seed values, mean ± sd and the margin over the best published value per geometry column | |
| (recomputed with the 5-seed sd) live in `aide/results/tables/stage_a_5seed.md` and `stage_bc_5seed.md` (+ a section appended to stage_bc.md); a worse | |
| seed is reported, never dropped. Nothing else in the protocol changes. | |
| C22 outcome (02:39 EDT): kp seeds 789 / 1011 test All 48.54 / 48.57 (5-seed 48.60 ± 0.12, HM 131.57 ± 0.57, PCK 87.02 ± 0.06); vis+sem seeds 789 / 1011 | |
| All 48.44 / 48.55 — both above the locked 3-seed mean 48.36 (reported, not dropped) — 5-seed 48.42 ± 0.15, HM 130.07 ± 0.71, PCK 87.11 ± 0.08. Every one of | |
| the 10 runs beats the best published value in all 5 geometry columns; 5-seed margins kp All −4.29 / HM −2.99 / PCK +1.07, vis+sem −4.47 / −4.49 / +1.16. | |
| Tables `aide/results/tables/stage_a_5seed.md`, `stage_bc_5seed.md`, stage_bc.md §9. | |
| C23. K-v2 programme (CockpitWM-K improvement, `aide/models/cwm3d_v2.py`, `--model cwm3d_v2`) — PRE-REGISTERED 2026-09-14, S0 UTC 2026-09-14T22:51:07Z | |
| (decisions row of the same date "K-v2 programme — S0 pre-registration", written before any run; `aide/results/kv2/S0_UTC.txt`). Honesty clause: | |
| the design was motivated by TEST diagnostics (`arch_attribution.md` §3.2), so every K-v2 configuration is fixed on VAL only and MotionBERT-kp gets | |
| the same val budget (10 val points each). Nothing in §4 changes: val-only selection on All-MPJPE of the EMA weights, one test evaluation per run | |
| by the harness, no flip TTA, ensembles only as labelled rows, Stage-A recipe verbatim, paired per-clip clip-bootstrap 10k seed 0, verdict rule of | |
| `arch_attribution.md` (BEATS / WORSE = CI excl. 0 on ≥ ceil(2n/3) seeds, same sign on all; NOT_WORSE = all CIs cover 0 and |Δ| < 0.25; else | |
| INCONCLUSIVE). Decisions are computed by `aide/eval/kv2_gate.py` from pred_val.npz (UTC-stamped json in `aide/results/kv2/`), never by hand. | |
| S1 (seed 42, n = 405): 9 configs `cwm3d_v2[_{nospatial,khead,nohaz,hazaux,plain,q060,q090,lam03}]_s42_v2` vs `cwm3d_kp_s42_v2`; statistic = paired | |
| per-clip Δ val All; SURVIVE iff mean Δ ≤ −0.30 px; ≤ 2 survivors (ties within 0.10 px → fewer components: full/q060/q090/lam03 = 5, | |
| nospatial/khead/hazaux/plain = 4, nohaz = 3); HM guard KILL iff Δ val HM (40 clips) mean > +3.0 AND CI excl. 0 — the 40-clip val HM subset is never | |
| a ranking statistic (level CI width ≈ 37 px vs 8.3 for val All; seed sd 0.93 vs 0.21; val threshold 83.3 ≠ test 72.05). Test numbers recorded only. | |
| S1b: survivors at seed 123; continue to 456 / 789 / 1011 iff Δ val(123) < 0 and the 2-seed mean ≤ −0.25; else "val-killed at seed 123". | |
| S2: survivors × 5 seeds without / with BATON trunk-only init (`cwm3d_v2_batonpre_pretrain_s123`, corpus sha256 465dac1cf0a8265a, bs 256, 6 epochs, | |
| held-out-route selection, `--zeroshot 0`, unless `kv2_gate.py pretrain-check` shows K's checkpoint loads every trunk tensor with zero shape skips); | |
| K's +pre arm completed with `cwm3d_kp_batonpre_trunkp123_s{789,1011}_v2`. | |
| S3 (MB fairness): (i) `motionbert_kp_{d192,L4,lr1e-4,lr1e-3,wd0.01,wd0.1,do0.1,d192_L4,d192_L4_do0.1}_s42_v2` + default = 10 val points; MB-best = | |
| argmin val All (ties 0.10 → fewer params); MB-best at 5 seeds if ≠ default; (ii) MB-best + trunk-only init at 5 seeds | |
| (`motionbert_kp_batonpre_trunkp123_s{789,1011}_v2` unconditionally; new pretraining only if MB-best changes d / L). | |
| K-v2(best) = the S2 survivor with the lowest mean VAL All over its finished seeds (ties 0.10 → fewer components). | |
| PRIMARY (test All, 5 seeds, both sides same pretraining state): P1 = K-v2(best) − MB-best; P2 = K-v2(best)+pre − MB-best+pre. ADOPTION: A1 = | |
| K-v2(best) − K BEATS on All over 5 seeds → main-table row (K's row unchanged), else ablation row; A2 = K-v2+pre − K+pre BEATS → "+ BATON trunk" row. | |
| SECONDARY (descriptive, CIs, never an adoption basis): d-n, HM, PCK@0.05, HM h=5, per-part, per-horizon, haz_P AUC vs HM membership. | |
| Wording consequences: P1 ∧ P2 → architecture advantage may be written for K-v2 (K's attribution sentence unchanged); P1 ∧ ¬P2 → "advantage | |
| without pretraining that the BATON trunk equalises"; ¬P1 ∧ P2 → "advantage conditional on the pretrained trunk"; neither → wording unchanged, | |
| K-v2 = ablation family + NEGATIVES entry; any WORSE reported as is. | |
| Multiplicity: ≤ ~49 new single test evaluations, none used for selection; confirmatory claims exactly P1, P2, A1, A2. Published numbers untouched; | |
| results only in `aide/results/tables/kv2_program.md` (+ .json, `aide/eval/report_kv2_program.py`, append-only, machine-readable | |
| `**ADOPTION A1: ADOPTED|NOT ADOPTED**`). Launcher `aide/train/run_kv2_program.sh` (project GPU gate, finished runs never re-run, | |
| pids `aide/results/kv2/pids.txt`); every programme run json must have t_start_utc later than the S0 UTC above. | |
| C23 outcome (2026-09-15; report = last block of `aide/results/tables/kv2_program.md`, generated 01:31:47Z from run outputs only; every | |
| decision by `aide/eval/kv2_gate.py` from pred_val.npz with UTC-stamped json in `aide/results/kv2/`; 42 new runs, 139.4 GPU-min; nothing | |
| in §2 / §4 changed). S1 (23:45:48Z, n = 405): survivors nospatial (Δ val All −0.579 [−1.275,+0.058], 4 components, 2,666,906 params) and | |
| lam03 (−0.348 [−1.018,+0.310]); the other seven stopped (full −0.197, khead +0.115, nohaz −0.042, hazaux −0.227, plain −0.149, q060 −0.047 | |
| and HM guard +3.26 [+0.21,+7.50], q090 −0.239). S3 (23:49:07Z): MB-best = do0.1 (val All 55.215 vs default 55.550, Δ −0.335 [−0.888,+0.129]; | |
| no d / L change → existing `motionbert_kp_batonpre_pretrain_s123` reused). S1b (00:00:38Z / 00:00:39Z): nospatial CONTINUE (Δ val s123 | |
| −0.566 [−1.121,−0.038], 2-seed mean −0.573); lam03 val-killed at seed 123 (−0.070 [−0.678,+0.534], mean −0.209) → no S2 arm, no full-trunk | |
| pretraining. pretrain-check nospatial vs K's checkpoint: NEED_V2_PRETRAIN (47 / 54 trunk tensors, 7 missing) → own trunk pretraining | |
| `cwm3d_v2_nospatial_batonpre_pretrain_s123` (same corpus / bs 256 / 6 epochs / held-out-route selection, best epoch 5, 10.72 GPU-min; 54 / 54 | |
| trunk tensors loaded, 0 shape-skipped, 0 missing before any +pre fine-tune). K-v2(best) = nospatial (val All 54.906 over 5 seeds). | |
| Test All-MPJPE, 5 seeds, one evaluation per run: K-v2 48.27 / 48.53 / 48.53 / 48.44 / 48.30 (+pre 47.98 / 47.95 / 48.04 / 47.81 / 48.10); | |
| K 48.77 / 48.66 / 48.46 / 48.54 / 48.57 (+pre 47.98 / 48.04 / 48.01 / 48.12 / 48.11); MB-best 48.39 / 48.41 / 48.21 / 48.37 / 48.92 | |
| (+pre 47.78 / 47.44 / 47.74 / 47.74 / 47.77). Paired Δ (BEATS rule 4/5): P1 K-v2 − MB-best −0.05 ± 0.36 (1/5 CI excl. 0, mixed sign) | |
| INCONCLUSIVE; P2 (+pre both) +0.28 ± 0.16 (0/5) INCONCLUSIVE; A1 K-v2 − K −0.19 ± 0.21 (1/5) INCONCLUSIVE; A2 (+pre both) −0.08 ± 0.14 (0/5) | |
| INCONCLUSIVE → **ADOPTION A1: NOT ADOPTED**, **ADOPTION A2: NOT ADOPTED**; consequence row "neither": paper wording unchanged verbatim, | |
| K-v2 = ablation family + NEGATIVES entry; no WORSE on any primary cell. Secondary: HM −1.62 ± 1.36 and HM h=5 −1.84 ± 1.75 vs K | |
| INCONCLUSIVE; PCK NOT_WORSE. Descriptive: K+pre − K −0.55 ± 0.15 BEATS; MB-best+pre − MB-best −0.77 ± 0.28 BEATS; K − MB-best +0.14 ± 0.28 | |
| and K+pre − MB-best+pre +0.36 ± 0.15 INCONCLUSIVE. Deviations (protocol-neutral; `aide/results/kv2/NOTES.md`): (a) `train_kp.py:630` | |
| tensor-output guard fixed 23:11:53Z after all 11 MotionBERT-kp launches died before writing any output (relaunched 23:12:51Z; four | |
| dict-output runs ran under the pre-fix file, behaviour identical, code_sha256 per run in the report §6); (b) per-architecture pretraining | |
| name with the survivor's own flags instead of the single pre-registered name, whose full-trunk pretraining was never needed; (c) S1b | |
| before any pretraining (order of GPU work); (d) lane-B bash aborted after its last run (stale byte offset after a live launcher edit; | |
| bookkeeping only, launcher now execs a per-launch snapshot). Decisions row "K-v2 outcome" 2026-09-15 (also records the derived five-seed | |
| means ± sd, which the report does not print). | |
Xet Storage Details
- Size:
- 44.4 kB
- Xet hash:
- 9a1fb1b93b7e4066c67fcffb3a00ae8959e17922ddefcb654f7c54bf64856ee1
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.