8.6 GB
2,838 files
Updated 16 days ago
Name
Size
aide_protocol
human_eval_299
labels
tables
README.md7.38 kB
xet
README.md

TriDrive — benchmark: labels, human evaluation, AIDE protocol and every result table

Anonymous submission under double-blind review. Every file under tables/ and aide_protocol/ is text (Markdown, JSON, CSV, LaTeX, Python) and was passed through the release pseudonymisation described at the end; no number was changed — only identifiers, paths, hosts and annotator names were rewritten. labels/ and human_eval_299/ have their own READMEs.

Sub-folders

Path Contents
labels/ Warning-probe label tables: the 979 consensus-labelled training anchors (562 manual / 417 engaged) and the gold discrimination set (512-row anchored layout; 509-row headline set) — see its own README
human_eval_299/ The pre-registered 299-clip human evaluation: clip manifest, three-stage labels, frozen thresholds, the pre-registration document (byte-identical, hash included) and the result tables — see its own README
aide_protocol/ AIDE evaluation protocol: official split lists, protocol constants and metric code, cache builder, Gate-0 verification script and its record (details below)
tables/release_tables/ Every result table behind the paper, as frozen for the paper's artefact release: aide/ (143 files: *.md + *.json reports, results.csv, kv2/, mirror_completion/, pretrain_variants/), baton/ (225 files: BATON programme tables, alert timing, DMS coverage / fairness, hold-out stage, and the release-time copies of the two ledgers) and baselines_dms/ (100 files: phase-1 extractor measurements, read-out selections, dry runs, repro report)
tables/paper_tables_iclr/ The 53 LaTeX tables of the manuscript and its appendices, generated by code/paper/tables/gen_tables.py from the markdown tables above (every number copied verbatim; the source table is named in a comment at the top of each file)
tables/ledgers/ The current versions of the two append-only run ledgers of the BATON programme: paired_compare.csv (56,990 rows — every paired device-cluster bootstrap comparison, written by code/eval/paired_compare.py) and cwm3_quick_metrics.csv (158,967 rows — quick metrics of every training run); release_tables/baton/ holds the copies frozen for the artefact release
tables/aide_extra/ AIDE material not in the release tables: MotionBERT official-resolution quantiles, the EMA confound check, the Driver-WM quantile sweep, head-to-head tables, the Driver-WM v2 audit and the Fig. 3b readings

Sub-folder file counts: release_tables 464, paper_tables_iclr 53, ledgers 2, aide_extra 9, aide_protocol 12.

aide_protocol/

File Origin in code/aide/ What it is
splits_training.csv, splits_validation.csv, splits_testing.csv data/ The official AIDE split lists (1884 / 405 / 609 clip rows, header excluded)
protocol.py data/ Protocol constants (driver selection rule, sample grid, horizons, PCK threshold); AIDE_ROOT is read from the environment
build_cache.py data/ Builds the AIDE keypoint cache from the official annotations (python aide/data/build_cache.py [--workers 8])
verify_protocol.py, verify_protocol.json data/, results/cache/ Gate 0: reproduces the published zero-velocity numbers on the official test split; the JSON is the recorded verification
cache_summary.json results/cache/ Summary of the built cache: clips per split, detection statistics, label histogram, the locked driver rule and sample grid
metrics.py, harness.py eval/ Protocol metrics (All-MPJPE, PCK, high-motion subsets) and the evaluation harness (pred npz → metrics.json + rows of results.csv)
protocol.md, driverwm_kp_port.md docs/ Protocol documentation, and the notes of the port of the Driver-WM reproduction to keypoint part tokens (architecture-attribution experiment)

The files are copied flat here for reading; the importable versions live at the paths given in the second column.

tables/ledgers/ columns

  • paired_compare.csv: run_stamp,tag,regime,split,cell,metric,a,b,value_a,ci_a_lo,ci_a_hi,value_b,ci_b_lo,ci_b_hi,delta,delta_ci_lo,delta_ci_hi,frac_boot_d_gt0,n,n_pos,n_dev,n_boot
  • cwm3_quick_metrics.csv: run_stamp,stage,variant,seed,regime,split,branch,chain,metric,value,base_rate,n,n_rows_predicted,best_epoch,val_objective,run_dir (run_dir is the pseudonymised run directory)

tables/paper_tables_iclr/

tab_aide_5seed.tex, tab_aide_analytic.tex, tab_aide_geometry.tex, tab_aide_interventions.tex, tab_aide_semantic.tex, tab_app_identity.tex, tab_app_power.tex, tab_app_transfer.tex, tab_baton_exp3P.tex, tab_baton_k4.tex, tab_baton_main.tex, tab_branch_arch.tex, tab_driverwm_quantile.tex, tab_e4.tex, tab_ecan.tex, tab_face.tex, tab_future_state.tex, tab_gzfc.tex, tab_id_map.tex, tab_input_ablation.tex, tab_label_check.tex, tab_main_aide.tex, tab_main_baton_interaction.tex, tab_main_baton_interaction_ecan.tex, tab_main_kpf_gain.tex, tab_main_transfer.tex, tab_main_warning.tex, tab_modality_ablation.tex, tab_negative_summary.tex, tab_operating_points.tex, tab_road_dynamics.tex, tab_robustness.tex, tab_rssm.tex, tab_supp_arch_attribution.tex, tab_supp_baton_interaction_visual.tex, tab_supp_calibration.tex, tab_supp_driverwm_mirror.tex, tab_supp_driverwm_quantile.tex, tab_supp_formulation_abs.tex, tab_supp_hm_h5_arms.tex, tab_supp_hm_subsets.tex, tab_supp_holdout.tex, tab_supp_kpf_forecaster.tex, tab_supp_kv2.tex, tab_supp_master_aide.tex, tab_supp_motionbert_faithful.tex, tab_supp_oracle_diag.tex, tab_supp_pretrain_variants.tex, tab_supp_risk_ranking.tex, tab_supp_transfer.tex, tab_verified_mask.tex, tab_warning_head.tex, tab_warning_rmv.tex

Pseudonym scheme

BATON devices and routes appear only as pseudonyms, consistent across this bucket and with the public BATON release: driver_N for a device (the four real-name device folders of the corpus are mapped the same way), and driver_N__route_M__route_a-b for a route segment group (route_a-b = the segment range of the original recording). A recording id that occurs outside a complete device/route/segment token is replaced by route_M when the private map resolves it and otherwise by a stable date-free pseudonym rid_<10 hex>; no raw recording id or recording timestamp remains. File-system paths were rewritten to /home/USER/..., hosts and IP addresses to REDACTED / REDACTED_IP, annotator names to [annotator], e-mail addresses to [email].

Not included

  • Per-sample prediction dumps of the joint model and of the baselines (pred_layout_*.npz, ≈ 21 GB): the metrics computed from them are in tables/; available on request after the review period.
  • AIDE-derived caches and per-clip predictions (aide_kp.npz, aide_kin11.npz, baton_pose_corpus.npz, pred_*.npz): AIDE is obtained from its authors; aide_protocol/build_cache.py rebuilds the cache and code/aide/ the predictions.
  • The one figure of the release tables (aide/pareto_all_vs_hm.png, drawn by code/aide/eval/baselines_analytic.py; its table is aide/tradeoff_all_vs_hm.md) and the crop of Fig. 3b of the Driver-WM paper (the readings taken from it are aide_extra/driverwm_v2_fig3b_readings.json).
Total size
8.6 GB
Files
2,838
Last updated
Oct 7
Pre-warmed CDN
US EU US EU

Contributors