code/aide/docs/protocol.md

AIDE head-to-head protocol (locked 2026-09-10, Gate 0 passed)

Everything measured on the AIDE track is defined by aide/data/protocol.py; this note records what was locked, the evidence that the locked choice reproduces the published Driver-WM (arXiv 2605.05092v2) zero-velocity numbers, and the assumptions we have had to make where the paper does not specify a detail. Numbers below are copied from script outputs (aide/results/cache/verify_protocol.json, aide/results/tables/analytic_baselines.md, aide/results/tables/results.csv); nothing is hand-computed.

1. Data and splits

item	value	source
clips	2,898 local AIDE clips, 4 views, annotation/<id>.json HALPE-136 pose_list (incar view)	aide/results/cache/cache_summary.json
official split	train 1,884 / val 405 / test 609, aide/data/splits_{training,validation,testing}.csv from github.com/ydk122024/AIDE; local ∩ official = 100 %	cache_summary.json (missing: [] for all three)
clip length	45 frames @ 15 fps for 2,885 clips; 13 clips are longer (60–345 frames) and are sampled with the same linspace(0, n−1, 10) rule	cache_summary.json: long_clip_lengths
keypoint cache	aide/results/cache/aide_kp.npz — S [N,45,136,3] (all frames, training windows), S10 [N,10,136,3] (protocol frames), S10_alt (sensitivity variant), labels, bboxes; sha256 digest fb49ec33…4cea998f9 (recorded in every run json)	run_build_cache_latest.json

2. Locked protocol constants

constant	value	why
T_OBS / T_PRED	5 / 5 of 10 uniformly sampled frames	paper Sec. 4
SAMPLE_GRID	floor = np.linspace(0, n−1, 10).astype(int) → 0,4,9,14,19,24,29,34,39,44 for n = 45	only grid reproducing the paper's ZV All / HM (the round grid gives 53.17 / 136.17)
DRIVER_RULE	first — the driver is result[0] of each frame; frames without detections are forward-/back-filled	reproduces all five published ZV numbers; equals the largest box in 98.8 % of frames
MPJPE	mean over clips × horizons × all 136 joints of the Euclidean pixel error, no confidence mask	paper protocol; a conf ≥ 0.1 variant is reported as a secondary metric (mpjpe_conf0.1)
d-nMPJPE	100 · MPJPE / diag(1920, 1080) = MPJPE / 22.029	paper's 2.40 for ZV 52.89
PCK@t	fraction of (clip, horizon, joint) with error ≤ t · max(W, H) = 96 px @0.05	reproduces 85.95; t · diag gives 87.67 (kept as pck05_diag)
High-Motion (HM) subset	top ⌊0.1 N⌋ clips (60 on test, 40 on val) by mean GT displacement over the 4 future transitions (frames 5→6…8→9) and all joints; threshold on test 72.05 px	reproduces the paper's HM 139.19 / threshold 72.05
HM h=5	per-horizon MPJPE on the HM subset at the last horizon	paper quotes ZV 178.62 / Driver-WM 155.82; ours ZV 178.67
CIs	10,000 clip-level bootstrap resamples; paired per-clip differences vs zero-velocity and named reference runs	aide/eval/metrics.py, harness.py

3. Gate 0 — reproduction evidence (official test split, zero-velocity)

metric	paper	ours (locked protocol)	tolerance	pass
All-MPJPE	52.89	52.929	0.5	yes
d-nMPJPE	2.40	2.403	0.03	yes
HM-MPJPE	139.19	139.228	2.0	yes
HM threshold (px)	72.05	72.052	2.0	yes
PCK@0.05	85.95	85.948	1.0	yes

Sensitivity (48 rows in verify_protocol.json, 4 driver rules × 3 grids × 2 HM definitions × 2 PCK normalisers): the brief's original "largest-bbox + round" rule gives 48.64 / 107.93 (not the paper); max_score and iou_track do not reproduce either. The largest_bbox + round combination is kept as --variant alt (S10_alt) for sensitivity rows only.

4. Evaluation contract

• A run is a directory aide/results/runs/<name>/ with pred_test.npz (pred [N,5,136,2] px, clip_id [N]; optional logits_<task>), optionally pred_val.npz. The harness aligns rows by clip_id, requires the exact split clip set, writes metrics_<split>.json (append-only: a second evaluation gets a timestamped name) and appends rows to results/tables/results.csv under a file lock.
• Ground truth = frames 5..9 of S10; the model side only ever receives frames 0..4 (train_kp.split_obs_fut is the single cut point; aide/tests/test_no_future_leak.py and test_cwm3d_zero_init.py NaN-poison the future block and require bit-identical predictions and rejection of 10-frame inputs).
• Model selection: val All-MPJPE of the EMA weights; test is evaluated once per run.
• Training windows come from the 45-frame S array (sub-window augmentation, gaps ∈ {4,5}); val and test use the protocol frames only.
• Every run writes run_<name>_<utc>.json (common/runmeta.write_run_json: command, code sha256, input digest, seed, config, wall/GPU minutes, val/test headline numbers).

5. Analytic bars (test, locked protocol) — from analytic_baselines.md

predictor	All	HM	note
zero-velocity	52.93 [48.88, 57.25]	139.23 [116.93, 167.68]	paper 52.89 / 139.19
anti-velocity 0.1	52.53	138.55	Δ vs ZV −0.40 [−0.82, +0.13] n.s.
ZV-shrink α = 0.44 (α fitted on val)	51.54 [47.46, 55.97]	134.36 [111.80, 162.87]	the honest parameter-free bar; Δ vs ZV All −1.39 [−2.66, +0.26], HM −4.86 [−7.67, −1.95]
paper Driver-WM	71.47	138.03	PCK@0.05 71.66, HM h=5 155.82
paper static pooling	–	134.56	best published HM

Gate A (plan §4): CWM3-D keypoint-only must reach test All < 52.89 and HM < 134.56 and beat the val-fitted ZV-shrink by ≥ 1 px All and ≥ 3 px HM (3 seeds). The driver aide/train/run_stage_a.py applies the same rule on the val split (bootstrap-free) before spending GPU on seeds 123/456.

Stage A outcome (2026-09-10, aide/results/tables/stage_a.md): CWM3-D-kp test All 48.63 ± 0.15 / HM 131.64 ± 0.76 / PCK@0.05 87.04 / HM h=5 163.37 over seeds 42/123/456; paired Δ vs ZV-shrink −2.91 [−4.51, −1.72] All and −2.72 [−5.31, −0.04] HM; val margin ≥ 1 / ≥ 3 px on every seed → Gate A PASS, with the caveat that the test-split HM margin is 2.72 px and the HM gain over SiMLPe-kp (same recipe, 50.26 / 131.22) is nil (+0.41 [−1.48, +2.26]). Every learned keypoint-only model is far below Driver-WM's published 71.47 / 138.03. Details and ablations: docs/decisions.md rows of 2026-09-10.

6. Driver-WM reimplementation assumptions (running list)

The paper releases no code. Wherever we had to choose, the choice is listed here and will be carried into driverwm_repro.py and the tables ("paper number" and "our reproduction" are always shown side by side; SOTA claims are made against the paper numbers).

Protocol / data

1. Driver identity in multi-person frames = first detection of pose_list[t].result (Sec. 2). The paper's Table 1 is consistent only with this rule; the semantically better largest-box rule is a sensitivity variant.
2. Frames without a detection (1 frame in the whole set) are carried over from the previous frame.
3. Uniform sampling of 10 frames uses the floor grid (Sec. 2); the 13 long clips are sampled over their full length.
4. HM uses the 4 future transitions only (the 5-transition variant incl. frame 4→5 is a sensitivity row).
5. PCK normaliser = max(W, H); d-n normaliser = image diagonal.
6. Label strings are normalised (happiness→Happiness, weariness→Weariness, Forward→Forward Moving); the seven-class DBR order and the other class orders follow the paper's tables.

Visual features (Driver-WM App. B.2) 7. Backbone Qwen3-VL-2B-Instruct (local weights), bf16, sdpa attention, frozen. 8. Prompt = the paper's App. B.2 text verbatim (protocol.QWEN_PROMPT). 9. Feature = mean over the full last-layer hidden sequence (model.model(...).last_hidden_state, post-norm), 2048-d; a "visual tokens only" mean is stored as a variant. The paper does not say which tokens are pooled. 10. Images at native 1920×1080 (≈ 2,110 visual tokens/frame); one frame per (clip, sampled index, view), i.e. 10 × 4 frames per clip, stored fp16.

Model (to be used in driverwm_repro.py, not yet run) 11. Identity latent interface on the 2048-d features + view embedding; 1 causal temporal self-attention layer with 4 heads; Perceiver-style context module with 4 latent queries; gate = σ(MLP). 12. Deterministic 5-step mean rollout (closed loop) for the future latents. 13. ST-GCN skeleton decoder with 2 layers and sigmoid output scaled to (W, H); keypoints with conf < 0.1 masked in the training loss (evaluation always uses all joints). 14. Losses L_lat (velocity) + L_skel + L_aux + L_phys with λ_lat = λ_phys = 0.1; AdamW 5e-5, bs 32, 100 epochs, seeds 42/123/456. 15. The in-cabin view is the seat ROI crop only if the paper's Fig./B.2 is read that way; we start with the full frame and record the alternative.

Model / training, continued (decided for aide/models/driverwm_repro.py, Stage C) C10. Skeleton decoder outputs ABSOLUTE normalised coordinates through a sigmoid (Eq. 8), no residual anchor (dm_bench's "residual convention" is not used): the reproduction therefore does not start at zero-velocity, unlike our models. C11. ST-GCN = Linear(2048 → 136 × 32) followed by 2 blocks of {spatial graph conv over the normalised HALPE-136 adjacency of protocol.EDGES_ALL (+ self loops), temporal conv k = 3 over the 5 rolled-out steps, residual, LayerNorm} and Linear(32 → 2). C12. Perceiver context (App. B.6): mean-pooled internal history → MLP → 4 latent queries in a 256-d bottleneck; keys/values = 256-d projections of the external prefix; output → Linear → 2048 added to the last internal state (= m_t). Gate input = the current (rolled-out) pooled external latent ê_t (Eq. 18). f_int / f_ext = residual MLPs 2048 → 4096 → 2048; internal and external pre-encoders = one pre-norm TransformerEncoderLayer(2048, 4 heads, d_ff 4096) each with a causal mask and learned step embeddings. C13. Losses exactly as written: L_skel Eq. 23 (per-frame conf ≥ 0.1 mask, squared error in normalised coordinates, summed over the 5 steps), L_lat^vel Eq. 14 (squared norm over the 2048 dims, summed over the 5 velocities from the last observed latent; targets = the same pre-encoder run causally over the 10-frame in-cabin feature sequence, detached), L_bone Eq. 25 (body + hand edges), L_smooth Eq. 26 over [GT last observed; predicted 1..5]; total = 0.1 L_lat + L_skel + L_aux + 0.1 (L_bone + L_smooth). The literal sums make the latent term dominate the transition core; --repro-lat-norm mean (÷ 2048) is run as a declared port-sensitivity row, not as the paper. C14. Training: AdamW 5e-5, weight decay 1e-5, cosine to 0 without warm-up, batch 32, 100 epochs (paper: 80), no early stopping, no augmentation, no EMA; protocol frames only; selection on val All-MPJPE (ours). C15. Interventions (Table 3): do(Ext = ∅) zeroes the exterior features of the 5 observed frames at the input; do(Ext = swap_clip) replaces them by another test clip's (fixed derangement, seed 0); do(λ_CA = 0 / 1) clamps the gate. Deviations are pixel distances between intervened and factual rollouts (All / HM / h=5 / Head = face + body joints 0–4, 17, 18 / Hands); GT errors also reported. C16. Parameter count of the reproduction: 123.7 M (dominated by the 2048-d attention/MLP blocks; the paper gives no count).

CWM3-D-vis / semantic choices (ours, Stage B) S1. Visual tokens: per observed frame and view, LayerNorm(2048) → Linear(2048 → 256) + view embedding + step embedding, inserted next to the 4 part tokens (same-frame fully connected, cross-frame causal); modality dropout 0.3 zeroes the whole visual block of a clip during training. S2. Training sub-windows use the feature of the NEAREST protocol frame (features exist for the 10 protocol frames only; offset ≤ 2 frames = 0.13 s); val/test use exact protocol frames. S3. Semantic heads: DBR / DER from mean(driver part tokens over the observed window) + mean(in-cabin visual tokens); TCR / VCR from mean(exterior visual tokens); LayerNorm + Linear; CE with label smoothing 0.1 and inverse-frequency class weights (train split, normalised to mean 1), weight 0.5 per head; observed window only, labels never enter the geometry path. Without vision (variant ii) all four heads read the driver tokens. S4. Optional traffic → driver gated cross-attention (variant iii): last-frame driver tokens attend to the exterior visual tokens of all observed frames; gate = sigmoid(MLP) initialised at 0.018 (near identity). S5. Selection remains val All-MPJPE; F1 / accuracy are reported at that checkpoint (no semantic selection).

S3b. Variant (v): semantic heads read the frozen 2048-d features directly — DBR/DER = Linear([LN(mean in-cabin feature over the observed window), mean driver part token]), TCR/VCR = Linear(LN(mean exterior feature)). S6. Ego-motion token (variant vi): per observed frame, 16 Farneback flow statistics × 3 exterior views (320×180 grayscale, flow between consecutive protocol frames; step 0 copies step 1), standardised with train-split mean/std stored as model buffers, Linear(48 → 256) + step embedding as one extra token per frame; the TCR/VCR direct heads also receive the window-mean of the standardised statistics. Interventions zero / swap it with the exterior features (zero_ext_ego, swap_ext_ego).

Stage B/C outcome (2026-09-10, aide/results/tables/stage_bc.md): CWM3-D vis+sem test All 48.36 ± 0.17 / HM 130.28 ± 0.86 / PCK@0.05 87.14 / HM h=5 161.93 ± 0.84 (Δ vs keypoint-only −0.49 / −0.20 / −0.52 px All per seed); semantic Macro-F1 43.2 / 45.1 / 82.3 / 59.8 → Gate B FAIL on all four heads (direct-feature heads 48.9 / 48.2 / 78.7 / 60.3; + ego-motion token 43.0 / 47.7 / 78.8 / 62.9). Gate C FAIL vs the paper (HM h=5 161.9 vs 155.82), PASS vs our reproduction (194.3). Driver-WM reproduction (C1–C16): All 130.86 ± 3.11 / HM 176.33 / PCK@0.05 54.2 — geometry not reproduced — but semantics reproduced (F1 71.7 / 68.7 / 89.6 / 68.4; Acc 73.6 / 78.2 / 93.0 / 82.3). Interventions and per-horizon curves: stage_bc.md §3–4; decisions: docs/decisions.md rows of 2026-09-10.

S7. Attention-pooling semantic heads (variant vii/viii, --sem-input attn): per stream one pre-norm causal TransformerEncoderLayer(2048, 4 heads, d_ff 4096) over the observed-window features (in-cabin; exterior = view-embedded mean of the 3 outside views), last state → LayerNorm → Linear; DBR/DER heads also receive the driver part-token mean; TCR/VCR the window-mean ego statistics when the ego token is on. Plain CE, λ 1, no class weights, no label smoothing. S8. Dual selection (--dual-select): besides the geometry-selected checkpoint (val All-MPJPE), the checkpoint with the best val mean macro-F1 is saved and evaluated as a separate run <name>__f1sel; its rows are labelled "DUAL SELECTION" and always carry their own geometry. Selection never touches the test split. S9. Stage-C sweep flags: --h-weights 1,1,1,2,3 (horizon weights on the Euclidean + NLL terms, normalised to mean 1), --motion-weight 3 (per-clip weight = min(GT future displacement / batch mean, 3), renormalised), --d 384 --ffn 1536 --head-hidden 768, --layers 6; aide/eval/ensemble.py averages seed predictions.

Follow-up outcome (2026-09-10, stage_bc.md Gate-B blocks and "Stage C late-horizon sweep"): with the repro-recipe heads the single geometry-selected checkpoint reaches F1 66.9 / 58.3 / 88.7 / 66.7 (geometry 48.51 / 130.94, unchanged); the val-F1-selected checkpoint reaches 72.2 / 64.0 / 89.8 / 69.2 at +1.0–1.4 px All; with the ego-motion token VCR 72.0 (a) / 72.8 (b). Gate B FAIL under both readings (DER −8.6 to −14; TCR −0.3 under (b)). Stage-C sweep: HM h=5 160.7 (motion weights) … 162.1; ensemble 161.5; none < 155.82 → Stage C FAIL vs the paper, PASS vs our reproduction (194.3).

S10. Per-view attention + learned view gate (variant DER-i, --sem-view-gate): all four views pass through the shared in-cabin causal encoder (+ view embedding); per head a learned softmax over the 4 last states (logits initialised 0, reported in the run json model_report); DBR/DER heads also take the driver part-token mean. Learned gates ended nearly uniform (≈ 0.23–0.28), i.e. the gain is multi-view pooling, not view selection. S11. --sem-frame-avg der = frame-level DER logits averaged over the 5 OBSERVED frames (never the future); --sem-task-weights der=2 = per-head CE multiplier; sem_only model = frozen features → S7 heads only, selected on val mean macro-F1 (its pred is the zero-velocity copy so the harness contract holds).

Closing outcome (2026-09-11, stage_bc.md "DER follow-up" + "Final AIDE verdict table"): semantic-only reference DER 64.7 ± 0.2; view-gated heads DER (a) 73.1 ± 1.9 / (b) 74.7 ± 0.4 → full Gate B with this head: (a) single geometry-selected checkpoint FAIL on DBR by 0.5 (67.6 / 73.1 / 91.4 / 69.9), (b) dual selection PASS on all four seed means (71.1 / 74.7 / 90.3 / 70.3; TCR by 0.14, 1/3 seeds) at +1.4 px All. Gate A PASS, Stage C FAIL vs 155.82 (HM h=5 162.8 [119.4, 221.7], 60 clips, ±51 px) / PASS vs our reproduction. Track closed.

CWM3-D-specific choices (ours, documented for reproducibility) 16. Part tokens: 4 per frame (body 0–25, face 26–93, lhand 94–114, rhand 115–135); hands share the tokenizer + a side embedding; per-joint channels (Δx, Δy)/200 to the last observed pose, x/W, y/H, conf. 17. Anchor mixture pred = last + gate ⊙ (Σ_k softmax(logits)_k · a_k − last) + 200·μ over anchors {last, mean(obs), last − 0.1·(last − prev)}; head output layer zero-initialised so gate = μ = logits = 0 and the model is exactly zero-velocity at initialisation (test_cwm3d_zero_init.py, tolerance 1e-6 px). The linear gate is what makes a softmax mixture reducible to ZV at zero init (a pure softmax would start at the anchor mean). 18. Loss = Euclidean px error + 0.1 isotropic Laplace NLL (b₀ = 30 px) + 0.05 bone length over the AlphaPose body + hand edges + 0.01 jerk of [last, pred₁..₅]. Recipe: AdamW 3e-4, wd 0.05, cosine with 10 warm-up epochs, ≤ 200 epochs, bs 32, EMA 0.999, grad-clip 1.0, patience 40. 19. Augmentation (train only): 10-frame sub-windows with gaps in {4,5} (protocol grid with p = 0.5), keypoint noise σ = 2·(1 − conf) px, joint dropout 0.1 (xy → last observed, conf → 0), translate ±20 px and scale ×[0.95, 1.05] about the last-observed centroid (all 10 frames); no flip. 20. The SiMLPe-kp baseline (simlpe_kp_*_v2) is trained with the identical recipe and augmentation (plain Euclidean loss) so the comparison isolates the architecture.

Hardening (roadmap supplement B §B2, 2026-09-12; docs/aide_hardening_report.md, stage_bc.md §1b/1c/4b/8) C17. Shrink-curve analysis and the h=5 trade-off (aide/eval/baselines_analytic.py, tradeoff_all_vs_hm.md, pareto_all_vs_hm.png). Two parameter-free families are evaluated besides the val-fitted ZV-shrink (α = 0.44): (i) mean_pose_shrink_a, pred_h = (1 − a)·last + a·μ with μ = TRAIN-split mean pose of protocol frame 4 (per joint), a ∈ {0.1, 0.2, 0.3} fixed a priori; (ii) zv_shrink_h_val, α_h = a·h/5 towards the observed mean with a = 0.76 fitted on val All. The curves in the figure are evaluated on the test split for description only (nothing is selected on them). Result: HM h=5 and All are anti-correlated along the mean-pose curve on both splits. Driver-WM's All / HM (71.47 / 138.03) sit on the shrinkage curve, but its HM h=5 does not: on TEST a = 0.3 gives 65.78 / 134.09 / 166.00 and the family bottoms out at HM h=5 163.74 (a = 0.50, All 81.53), 8.5 px above 155.82 at the paper's All; the val curve (a = 0.3 → 69.76 / 134.44 / 155.98) does not transfer to test, so shrinkage does NOT reproduce the paper's 155.82. The 5–7 px h=5 gap between the paper and our best (160.7) is unexplained by shrinkage and unresolvable at ±51 px (n = 60 HM clips; single paper seed). Honesty clause: had zv_shrink_h_val beaten CWM3-D-kp on test HM, the Gate-A HM caveat would have become a limitation; it did not (132.34 vs 131.64, 0.7 px) — the caveat stands as a caveat. HM d-nMPJPE = 100·HM/diag is now a reported column (ZV 6.320 = paper 6.32; test_metrics.py). C18. Full Driver-WM Table-3 mirror (aide/eval/interventions.py, stage_bc.md §4b). Operators on the OBSERVED window only: swap (fixed derangement), remove ext (zero), shift large / small (exterior features delayed 2 / 1 protocol steps, edge-padded — frame t receives frame max(t − k, 0)), drop view (one exterior view zeroed; mean and max over the three views reported), ext_scale λ ∈ {0, 2} (exterior visual TOKENS scaled before the trunk; λ = 1 is bit-identical to the factual pass, unit-tested), gate_override λ_CA ∈ {0, 1, 2} for the traffic-gate variant (iii; seeds 123 / 456 trained so it is a 3-seed row) and the reproduction's gate. Deviations are reported in px and as Δ/All (÷ the model's factual GT All-MPJPE) because pixel deviations are not comparable at All 48 vs 71; the GT error of every intervened rollout and its paired difference to the factual rollout are reported (2k clip bootstrap). CWM3-D has no gate: "disable pathway" and "over-injection" are mirrored by λ = 0 / 2 on the exterior tokens (analogue), "force" is undefined for it. C19. Fair keypoint baselines on the shared recipe (hardening D; aide/models/motionbert_kp.py, aide/models/stgcn_kp.py; 3 seeds each, Stage-A recipe + augmentation, plain Euclidean loss, val-selected EMA weights). MotionBERT-kp = DSTformer-style alternating spatial (over the 136 joints of a frame) / temporal (over the 5 observed frames of a joint) pre-norm attention, 3 blocks, d 128, 4 heads, temporal Linear(5 → 5) read-out, zero-initialised Linear(d → 2) residual over the last observed pose (813,088 params). ST-GCN-kp = Linear(5 → 128) per joint + 8 STGCNBlocks taken unchanged from the Driver-WM reproduction's skeleton decoder (normalised HALPE-136 adjacency EDGES_ALL, temporal conv k = 3, residual, LayerNorm), same read-out (530,080 params). Both are exactly zero-velocity at initialisation and are pure functions of the observed block (aide/tests/test_hardening.py). Result (test, 3 seeds): MotionBERT-kp 48.38 ± 0.10 / d-n 2.196 / HM 129.91 ± 0.65 / HM d-n 5.897 / PCK@0.05 87.15 / HM h=5 162.93; ST-GCN-kp 49.22 ± 0.18 / 2.234 / 130.15 ± 0.52 / 5.908 / 86.86 / 162.97; vs CWM3-D-kp 48.63 ± 0.15 / 131.64 ± 0.76 and the published MotionBERT 73.51 / 141.53, ST-GCN 110.98 / 158.10. Consequence for the paper: the ≈ 25 px gap between the published motion baselines and our keypoint models is the training recipe (residual-over-last-pose parametrisation, zero-init output, sub-window / noise / joint-drop augmentation, val-selected EMA), which all our keypoint models share; the CWM3-D architecture adds ≈ 0 px over MotionBERT-kp on All (+0.25) and is 1.7 px worse on HM. Geometry SOTA claims are therefore made for the recipe-plus-model family, not for the part-token architecture alone. C20. Hardening variants and their pre-registered kill rules (roadmap B §B2; all decided on the VALIDATION split; test in stage_bc.md §1 / §8; 3 seeds each; recipe and augmentation of the parent row). A1 --patience-f1 40 (view-gate heads, dual selection): early stop only when val All AND val mean macro-F1 have both stalled for 40 epochs. Rule: kill if no seed's best val F1 exceeds its old run's maximum by > 0.3. Result: +0.00 / +0.44 / +0.00 (epochs 110 / 149 / 135 vs 89 / 102 / 95; the shared epochs are bit-identical) → survives on the rule, but the extra epochs change no test number: (a) is the same geometry checkpoint as DER (i) (a); (b) 70.9 / 74.5 / 90.4 / 70.1 vs 71.1 / 74.7 / 90.3 / 70.3 at All 49.83 vs 49.71. Reported as "no effect". A2 --sem-view-gate --egomotion: view-gate heads + the ego-motion token (S6). Rule: kill if val mean-F1 (b) < 78.2 on 2/3 seeds or val All (b) > 57 on 2/3. Result: val F1 (b) 76.57 / 77.97 / 76.62 (3/3 below) → KILLED (test (b) 70.9 / 73.5 / 90.9 / 71.9 at All 50.88, for the record). A3 logit ensemble of the three DER (i) (b) seeds (aide/eval/ensemble.py, mean of logits and coordinates): a separate "ensemble, not a single model" row (test F1 73.4 / 77.3 / 91.2 / 72.3; All 48.82 / HM 131.23). B1 (i) --motion-weight 5 --h-weights 1,1,1,2,3; (ii) --motion-oversample 2 (TRAIN clips drawn with replacement per epoch, p_i ∝ 1 + 2·s_i/mean(s), s_i = GT future displacement on the clip's protocol frames). Rule: kill if val HM h=5 ≥ 144.5 on 2/3 seeds or val All > 56. Result: (i) 143.5 / 145.2 / 144.9, (ii) 145.8 / 144.9 / 146.3 → both KILLED; their test HM h=5 (160.71 ± 0.38 and 161.04 ± 0.47) are the lowest single-model values on record but are not selected on. B2 --dct (μ parametrised by DCT-II coefficients over the 5 horizons, heads.py). Rule: kill if val All worse than the same-seed B2 main run by > 0.3 on 2/3 seeds. Result: +0.23 / +0.29 / +0.14 → survives (test 48.49 / 130.73 / 162.62 vs 48.36 / 130.28 / 161.93: no gain). B3 --refine-late: zero-initialised MLP on [last-frame part tokens, detached (h ≤ 3 prediction − last)/200] adding Δ to h = 4, 5 only (identity at init, bit-identical to the parent; test_hardening.py). Same rule as B2. Result: +0.30 / −0.03 / −0.29 → survives (test 48.34 ± 0.16 / 130.26 ± 0.08 / HM h=5 161.37 ± 0.55 vs 161.93 ± 0.84: within noise). B4 ensembles: 3 motion-weighted seeds (test 48.54 / 129.73 / HM h=5 160.26) and 9 runs mw + hw + dct (48.23 / 129.83 / 161.08); separate rows, never single models. Hybrid cwm3d_kp_spatial (coordinator request): CWM3-D-kp with two DSTformer-style spatial attention blocks (d_s 32, 4 heads) over the 136 joints of each observed frame in front of the part tokenizer (attended features concatenated to the raw 5 channels; anchor mixture kept; 6.74 M params; aide/models/cwm3d_spatial.py). Adoption rule: paired Δ HM vs the SAME-seed CWM3-D-kp with CI excluding 0 on ≥ 2/3 seeds and All not worse. Result: Δ HM −2.79 [−5.09, −0.62] / −1.45 [−3.21, +0.16] / −1.49 [−3.09, +0.08] (1/3), Δ All −0.07 / −0.01 / +0.03 (never worse) → NOT adopted; reported as a row (test 48.61 ± 0.11 / 129.73 ± 0.14 / HM h=5 161.12 ± 0.70). Killed on paper, not run: temperature scaling (argmax-invariant), flip TTA (item 19; asymmetric exterior views, unflipped features, left/right classes), trunk-state semantic heads (falsified), constant-velocity anchor (worse at every gain), wider / deeper / EMA (already run), train + val (breaks val-only selection and the val-fitted analytic bars).

C21. BATON → AIDE in-cabin pose pretraining (reverse transfer; GPU-baselines agent, 2026-09-12; table aide/results/tables/baton_pretrain.md). Corpus (aide/features/build_baton_pose_corpus.py → aide/results/cache/baton_pose_corpus.npz, an index over the route npz): the 440 BATON TRAIN-split routes of the 96 train-only devices (the s12 splits are device-disjoint — 96 train vs 66 val/test devices, 0 overlap, asserted; no val / test route or device enters), HALPE-136 RTMW keypoints at 10 Hz on the causal driver box (results/pose_causal_routes_v2 for the 11 re-extracted train routes, v1 for the rest); 3,410,103 frames, 68.7 % valid (driver detected, finite nose / neck / box, ≥ 80 % of body AND of face joints with score ≥ 0.3, both shoulders ≥ 0.3); 10-frame windows at 0.3 s steps (3 × 10 Hz; AIDE grid 0.267 / 0.333 s, span 2.7 vs 2.93 s — the closest realisable stride), anchors every 0.3 s, contiguous time required → 642,179 windows (< the 1 M cap), 22 held-out routes for the pretraining early stop. Coordinates: per-window isotropic similarity into the AIDE pixel frame (window-median shoulder width → AIDE-train median 309 px, shoulder centre → AIDE-train median (373, 762) px; a first box-height scaling left the shoulders at 201 px and was replaced before any GPU time); HALPE 17 (head-top, absent from RTMW) = nose + 0.65 (nose − neck); RTMW scores quantile-mapped per part to the AIDE-TRAIN score distribution. Residual domain gap: BATON future displacement median 13.3 px vs AIDE 29.1 px (real driving vs behaviour-selected clips). AIDE side: TRAIN-split statistics only; AIDE test never touched except by one evaluation per finished run. Pretraining (aide/train/pretrain_cwm3d_baton.py, run cwm3d_kp_batonpre_pretrain_s42): CWM3-D-kp (5.77 M, same init as cwm3d_kp_s42_v2), AIDE task and loss (Euclidean + 0.1 NLL + 0.05 bone + 0.01 jerk), train_kp augmentation, AdamW 3e-4 / wd 0.05, bs 256, 6 epochs, EMA; selection on the held-out BATON routes (best epoch 4: All 27.19 px); 16 ms/step, 4.0 GPU-min. The AIDE-val zero-shot curve is diagnostic only (58.6 → 60.2 px over the epochs; ZV 61.16). Zero-shot run cwm3d_kp_batonpre_zeroshot_s42_v2 (no AIDE training): test All 52.75 / HM 136.75 / PCK 85.90 / HM h=5 170.96 (ZV 52.93 / 139.23). Fine-tuning: aide/train/train_kp.py --init-ckpt <ckpt_best.pt> --init-parts all (opt-in flag; default off = unchanged trainer) with the Stage-A / B2 recipes verbatim; runs cwm3d_kp_batonpre_s{42,123,456}_v2, cwm3d_vis_sem_batonpre_s{42,123,456}_v2; paired per-clip bootstrap vs cwm3d_kp_s<k>_v2 / cwm3d_vis_sem_s<k>_v2 (n = 609, 10k). **Pre-registered rule (adopt only if Δ All CI excl. 0 on ≥ 2/3 seeds, improving): NOT met.** kp: Δ All −0.31 ± 0.02 px (CIs cover 0 on 3/3 seeds), Δ HM −2.38 ± 0.40 (1/3 CI excl. 0), Δ HM h=5 −1.21 ± 0.16 (INCONCLUSIVE), PCK +0.11 (NOT_WORSE), left hand −1.3 px, right hand +0.4 px; vis+sem: Δ All −0.10 ± 0.25, HM −1.14 ± 1.33, HM h=5 −0.16 ± 0.77 (all INCONCLUSIVE). Reading: the pretrained init converges ~2× faster (best epoch 22–29 vs 44–58) and moves every keypoint-only cell in the improving direction, but never beyond seed / clip noise → reported as "no reliable transfer", not adopted; the B2 main model is unchanged. A trunk-only-init secondary arm (cwm3d_kp_batonpre_trunk_s<k>_v2) is appended to the same table when finished. C21 addendum (12:55 EDT): a POST HOC trunk-only-init arm (--init-parts trunk, head at zero init; cwm3d_kp_batonpre_trunk_s<k>_v2) meets the numeric criterion (Δ All −0.49 ± 0.14 px, 2/3 CIs excl. 0; Δ HM −2.61 ± 0.79, 3/3) but was added after the pre-registered arms' result, so it is exploratory. Pre-registered before running: an independent pretraining seed (123) with the same trunk-only fine-tunes (cwm3d_kp_batonpre_trunkp123_s<k>_v2); the trunk-only pretrained init is adopted iff its Δ All is BEATS again. Every arm keeps val-only selection and one test evaluation per run. C21 outcome (13:00 EDT): the pre-registered replication REPLICATED. Independent pretraining cwm3d_kp_batonpre_pretrain_s123 (best epoch 4, held-out All 27.20 px, 1.8 GPU-min; zero-shot test All 53.56) → trunk-only fine-tunes cwm3d_kp_batonpre_trunkp123_s{42,123,456}_v2 vs cwm3d_kp_s<k>_v2: Δ All −0.79 [−1.29,−0.30] / −0.62 [−1.17,−0.11] / −0.45 [−0.89,−0.02] = **−0.62 ± 0.17 px, 3/3 CIs excl. 0 (BEATS)**; Δ HM −2.95 ± 1.07 (2/3, BEATS); Δ HM h=5 −3.88 ± 0.85 (1/3 CI excl. 0, INCONCLUSIVE); body −0.54 ± 0.26 BEATS; hands −1.0 to −1.2 px and every horizon negative (INCONCLUSIVE). Test All 47.98 / 48.04 / 48.01 (mean 48.01 vs 48.63 seed init; HM 128.7 vs 131.6; HM h=5 159.5 vs 163.4). Rule → the trunk-only BATON-pretrained init of CWM3-D-kp is adopted as a reported model row ("CWM3-D-kp + BATON in-cabin pretraining (trunk)"), NOT as a replacement of the B2 main model (the vis+sem arm was full-init and INCONCLUSIVE; a trunk-only vis+sem arm was not run). Full-init (trunk + horizon head) does not transfer (Δ All −0.31, all CIs cover 0): the pretrained head encodes BATON's 2–3× smaller motion magnitudes; keeping the head at its zero init (exact zero-velocity start) is what makes the trunk features usable. Every arm: AIDE val-only selection, one test evaluation per run; no AIDE test clip and no BATON val / test device in any training. (C21 timestamps: replication registered/launched 12:35 EDT, outcome 12:41 EDT; the vis+sem and view-gate trunk-only arms registered 12:43 EDT and launched 12:44 EDT — the "12:55 / 13:00 / 13:1x" wall-clock stamps written above were mis-read by the agent; run json t_start_utc fields are authoritative.) C21 outcome 2 (13:08 EDT; the two arms registered 12:43 EDT): (i) vis+sem B2 main recipe with the trunk-only BATON init (cwm3d_vis_sem_batonpre_trunkp123_s<k>_v2 vs cwm3d_vis_sem_s<k>_v2): Δ All −0.29 [−0.75,+0.16] / −0.53 [−0.99,−0.07] / −0.23 [−0.69,+0.22] = −0.35 ± 0.16 px (1/3 CI excl. 0) → **INCONCLUSIVE, not adopted**; Δ HM −1.76 ± 0.88, HM h=5 −2.32 ± 1.74 (INCONCLUSIVE); semantic F1 of the geometry checkpoint lower (DBR 36.7–41.3 vs 39.7–45.8). (ii) view-gate attention-head recipe, geometry-selected checkpoint (a) (cwm3d_vis_sema_vg_batonpre_trunkp123_s<k>_v2 vs cwm3d_vis_sema_vg_s<k>_v2): Δ All −0.49 [−0.94,−0.05] / −0.57 [−1.04,−0.09] / −0.70 [−1.13,−0.26] = **−0.59 ± 0.10 px, 3/3 CIs excl. 0 → BEATS, adopted as a reported row**; Δ HM −2.58 ± 0.24 (2/3, BEATS); left hand −1.50 ± 0.90 (BEATS); HM h=5 −2.68 ± 0.65 (INCONCLUSIVE); test All 48.15 / 48.06 / 47.95 vs 48.64 / 48.63 / 48.65. Caveat to print next to that row: the geometry-selected checkpoint comes ~20 epochs earlier with the pretrained trunk (best epoch 33–35 vs 48–61), so its semantic macro-F1 is markedly lower (DBR 55–59 vs 67–68, DER 61–63 vs 72–75, TCR 87–91 vs 91–92, VCR 64–65 vs 70; descriptive, geometry checkpoint (a)); the __f1sel dual-selection companions were written as usual and are the (b) rows for the AIDE agent's tables — the geometry gain does not come for free on the semantic side at the (a) checkpoint. Reading across the four trunk-only arms: the BATON trunk transfers to the geometry of every keypoint path it was tested on (kp ×2 pretraining seeds, view-gate (a)) and is directionally positive but inconclusive on the B2 vis+sem recipe (class-balanced label-smoothed CE heads); the transfer is a trunk (feature) effect and is lost when the horizon head is also copied.

C22. 5-seed extension of the two locked main models — POST HOC, pre-registered reporting rule (2026-09-13, decisions row of the same date written before any run). Seeds 789 and 1011 are added to CWM3-D-kp (cwm3d_kp_s<k>_v2) and CWM3-D vis+sem (cwm3d_vis_sem_s<k>_v2) with the recipes of the locked runs frozen verbatim (Stage-A recipe; vis+sem adds --visual --modality-dropout 0.3 --semantic --lambda-sem 0.5 --label-smoothing 0.1 --class-balanced 1), val-only selection per seed, one test evaluation per run. Reporting rule: the locked 3-seed rows (42 / 123 / 456) remain the headline numbers of stage_a.md / stage_bc.md; the 5-seed per-seed values, mean ± sd and the margin over the best published value per geometry column (recomputed with the 5-seed sd) live in aide/results/tables/stage_a_5seed.md and stage_bc_5seed.md (+ a section appended to stage_bc.md); a worse seed is reported, never dropped. Nothing else in the protocol changes. C22 outcome (02:39 EDT): kp seeds 789 / 1011 test All 48.54 / 48.57 (5-seed 48.60 ± 0.12, HM 131.57 ± 0.57, PCK 87.02 ± 0.06); vis+sem seeds 789 / 1011 All 48.44 / 48.55 — both above the locked 3-seed mean 48.36 (reported, not dropped) — 5-seed 48.42 ± 0.15, HM 130.07 ± 0.71, PCK 87.11 ± 0.08. Every one of the 10 runs beats the best published value in all 5 geometry columns; 5-seed margins kp All −4.29 / HM −2.99 / PCK +1.07, vis+sem −4.47 / −4.49 / +1.16. Tables aide/results/tables/stage_a_5seed.md, stage_bc_5seed.md, stage_bc.md §9. C23. K-v2 programme (CockpitWM-K improvement, aide/models/cwm3d_v2.py, --model cwm3d_v2) — PRE-REGISTERED 2026-09-14, S0 UTC 2026-09-14T22:51:07Z (decisions row of the same date "K-v2 programme — S0 pre-registration", written before any run; aide/results/kv2/S0_UTC.txt). Honesty clause: the design was motivated by TEST diagnostics (arch_attribution.md §3.2), so every K-v2 configuration is fixed on VAL only and MotionBERT-kp gets the same val budget (10 val points each). Nothing in §4 changes: val-only selection on All-MPJPE of the EMA weights, one test evaluation per run by the harness, no flip TTA, ensembles only as labelled rows, Stage-A recipe verbatim, paired per-clip clip-bootstrap 10k seed 0, verdict rule of arch_attribution.md (BEATS / WORSE = CI excl. 0 on ≥ ceil(2n/3) seeds, same sign on all; NOT_WORSE = all CIs cover 0 and |Δ| < 0.25; else INCONCLUSIVE). Decisions are computed by aide/eval/kv2_gate.py from pred_val.npz (UTC-stamped json in aide/results/kv2/), never by hand. S1 (seed 42, n = 405): 9 configs cwm3d_v2[_{nospatial,khead,nohaz,hazaux,plain,q060,q090,lam03}]_s42_v2 vs cwm3d_kp_s42_v2; statistic = paired per-clip Δ val All; SURVIVE iff mean Δ ≤ −0.30 px; ≤ 2 survivors (ties within 0.10 px → fewer components: full/q060/q090/lam03 = 5, nospatial/khead/hazaux/plain = 4, nohaz = 3); HM guard KILL iff Δ val HM (40 clips) mean > +3.0 AND CI excl. 0 — the 40-clip val HM subset is never a ranking statistic (level CI width ≈ 37 px vs 8.3 for val All; seed sd 0.93 vs 0.21; val threshold 83.3 ≠ test 72.05). Test numbers recorded only. S1b: survivors at seed 123; continue to 456 / 789 / 1011 iff Δ val(123) < 0 and the 2-seed mean ≤ −0.25; else "val-killed at seed 123". S2: survivors × 5 seeds without / with BATON trunk-only init (cwm3d_v2_batonpre_pretrain_s123, corpus sha256 465dac1cf0a8265a, bs 256, 6 epochs, held-out-route selection, --zeroshot 0, unless kv2_gate.py pretrain-check shows K's checkpoint loads every trunk tensor with zero shape skips); K's +pre arm completed with cwm3d_kp_batonpre_trunkp123_s{789,1011}_v2. S3 (MB fairness): (i) motionbert_kp_{d192,L4,lr1e-4,lr1e-3,wd0.01,wd0.1,do0.1,d192_L4,d192_L4_do0.1}_s42_v2 + default = 10 val points; MB-best = argmin val All (ties 0.10 → fewer params); MB-best at 5 seeds if ≠ default; (ii) MB-best + trunk-only init at 5 seeds (motionbert_kp_batonpre_trunkp123_s{789,1011}_v2 unconditionally; new pretraining only if MB-best changes d / L). K-v2(best) = the S2 survivor with the lowest mean VAL All over its finished seeds (ties 0.10 → fewer components). PRIMARY (test All, 5 seeds, both sides same pretraining state): P1 = K-v2(best) − MB-best; P2 = K-v2(best)+pre − MB-best+pre. ADOPTION: A1 = K-v2(best) − K BEATS on All over 5 seeds → main-table row (K's row unchanged), else ablation row; A2 = K-v2+pre − K+pre BEATS → "+ BATON trunk" row. SECONDARY (descriptive, CIs, never an adoption basis): d-n, HM, PCK@0.05, HM h=5, per-part, per-horizon, haz_P AUC vs HM membership. Wording consequences: P1 ∧ P2 → architecture advantage may be written for K-v2 (K's attribution sentence unchanged); P1 ∧ ¬P2 → "advantage without pretraining that the BATON trunk equalises"; ¬P1 ∧ P2 → "advantage conditional on the pretrained trunk"; neither → wording unchanged, K-v2 = ablation family + NEGATIVES entry; any WORSE reported as is. Multiplicity: ≤ ~49 new single test evaluations, none used for selection; confirmatory claims exactly P1, P2, A1, A2. Published numbers untouched; results only in aide/results/tables/kv2_program.md (+ .json, aide/eval/report_kv2_program.py, append-only, machine-readable **ADOPTION A1: ADOPTED|NOT ADOPTED**). Launcher aide/train/run_kv2_program.sh (project GPU gate, finished runs never re-run, pids aide/results/kv2/pids.txt); every programme run json must have t_start_utc later than the S0 UTC above. C23 outcome (2026-09-15; report = last block of aide/results/tables/kv2_program.md, generated 01:31:47Z from run outputs only; every decision by aide/eval/kv2_gate.py from pred_val.npz with UTC-stamped json in aide/results/kv2/; 42 new runs, 139.4 GPU-min; nothing in §2 / §4 changed). S1 (23:45:48Z, n = 405): survivors nospatial (Δ val All −0.579 [−1.275,+0.058], 4 components, 2,666,906 params) and lam03 (−0.348 [−1.018,+0.310]); the other seven stopped (full −0.197, khead +0.115, nohaz −0.042, hazaux −0.227, plain −0.149, q060 −0.047 and HM guard +3.26 [+0.21,+7.50], q090 −0.239). S3 (23:49:07Z): MB-best = do0.1 (val All 55.215 vs default 55.550, Δ −0.335 [−0.888,+0.129]; no d / L change → existing motionbert_kp_batonpre_pretrain_s123 reused). S1b (00:00:38Z / 00:00:39Z): nospatial CONTINUE (Δ val s123 −0.566 [−1.121,−0.038], 2-seed mean −0.573); lam03 val-killed at seed 123 (−0.070 [−0.678,+0.534], mean −0.209) → no S2 arm, no full-trunk pretraining. pretrain-check nospatial vs K's checkpoint: NEED_V2_PRETRAIN (47 / 54 trunk tensors, 7 missing) → own trunk pretraining cwm3d_v2_nospatial_batonpre_pretrain_s123 (same corpus / bs 256 / 6 epochs / held-out-route selection, best epoch 5, 10.72 GPU-min; 54 / 54 trunk tensors loaded, 0 shape-skipped, 0 missing before any +pre fine-tune). K-v2(best) = nospatial (val All 54.906 over 5 seeds). Test All-MPJPE, 5 seeds, one evaluation per run: K-v2 48.27 / 48.53 / 48.53 / 48.44 / 48.30 (+pre 47.98 / 47.95 / 48.04 / 47.81 / 48.10); K 48.77 / 48.66 / 48.46 / 48.54 / 48.57 (+pre 47.98 / 48.04 / 48.01 / 48.12 / 48.11); MB-best 48.39 / 48.41 / 48.21 / 48.37 / 48.92 (+pre 47.78 / 47.44 / 47.74 / 47.74 / 47.77). Paired Δ (BEATS rule 4/5): P1 K-v2 − MB-best −0.05 ± 0.36 (1/5 CI excl. 0, mixed sign) INCONCLUSIVE; P2 (+pre both) +0.28 ± 0.16 (0/5) INCONCLUSIVE; A1 K-v2 − K −0.19 ± 0.21 (1/5) INCONCLUSIVE; A2 (+pre both) −0.08 ± 0.14 (0/5) INCONCLUSIVE → ADOPTION A1: NOT ADOPTED, ADOPTION A2: NOT ADOPTED; consequence row "neither": paper wording unchanged verbatim, K-v2 = ablation family + NEGATIVES entry; no WORSE on any primary cell. Secondary: HM −1.62 ± 1.36 and HM h=5 −1.84 ± 1.75 vs K INCONCLUSIVE; PCK NOT_WORSE. Descriptive: K+pre − K −0.55 ± 0.15 BEATS; MB-best+pre − MB-best −0.77 ± 0.28 BEATS; K − MB-best +0.14 ± 0.28 and K+pre − MB-best+pre +0.36 ± 0.15 INCONCLUSIVE. Deviations (protocol-neutral; aide/results/kv2/NOTES.md): (a) train_kp.py:630 tensor-output guard fixed 23:11:53Z after all 11 MotionBERT-kp launches died before writing any output (relaunched 23:12:51Z; four dict-output runs ran under the pre-fix file, behaviour identical, code_sha256 per run in the report §6); (b) per-architecture pretraining name with the survivor's own flags instead of the single pre-registered name, whose full-trunk pretraining was never needed; (c) S1b before any pretraining (order of GPU work); (d) lane-B bash aborted after its last run (stale byte offset after a live launcher edit; bookkeeping only, launcher now execs a per-launch snapshot). Decisions row "K-v2 outcome" 2026-09-15 (also records the derived five-seed means ± sd, which the report does not print).