benchmark/labels/README.md

TriDrive — warning-probe label tables (benchmark/labels/)

Human labels of cabin moments ("episodes") from the BATON naturalistic-driving corpus, used to train and evaluate the TriDrive current-state warning probe (development name CockpitWM, warning head W). Each row is one labelled episode anchored on one row of the 0.2 s analysis grid of the regime stratum it belongs to. Raters saw the cabin + road video around the onset t0 and answered whether a warning to the driver was warranted at that moment. Nothing in these tables is derived from a driver-monitoring model.

Files

file	rows	strata (manual / engaged)	warranted	drivers	use
train_979.csv	979	562 / 417	182	91	training labels of the warning probe (devices of the TRAIN split only)
gold_512.csv	512	329 / 183	63	47	every anchored clean-test episode; column in_gold_509 marks the headline rows
gold_509.csv	509	327 / 182	31 + 32 = 63	46	headline gold of the paper (Section 4.4 / Appendix D)
counts.json	—	—	—	—	every count measured while building these files, the assertions and the file checksums

Label definitions

• Episode / onset t0. A candidate cabin moment of the annotation tool (route-relative seconds). The consensus protocol used two independent raters plus adjudication; only consensus rows are released (single-rater legacy rows, auto-excluded rows, unsure rows and the rows of two lenient rater groups are dropped).
• label / y. warranted (y = 1): a warning to the driver was warranted at t0 (attention or hands); not_warranted (y = 0). No unsure row is released.
• Anchoring rule (s15 gold_eval rule): the episode is attached to the latest grid row of its route with t in [t0 - 2.0, t0 + 0.2]; routes without usable video are skipped; episodes whose route has no grid row in the window are not released (consensus TRAIN rows before anchoring: 1148; defined clean-test rows: 568).
• Stratum = anchored regime. The grid exists separately for the manual regime (human drives) and the engaged regime (openpilot or OEM assistance active). An episode is looked up in both regimes' packs of its split; the row with the latest anchor wins, ties go to the regime of the episode itself. stratum therefore means "the regime pack whose row the episode is anchored on" (0 training and 0 gold episodes were anchorable in both; the regime at the anchor decides).
• Splits. Devices (drivers) are assigned to train / val / test once for the whole project; train_979.csv contains only TRAIN-split devices, gold_*.csv only TEST-split devices. No device appears in both files.
• The 509 rule. The evaluation grid compares the probe with the openpilot-derived comparators on the same rows: within a stratum it keeps the rows where every core comparator score is finite ("common finite comparator score" rule of the project's evaluation code, warning_gold.py, which is not part of this snapshot). Three of the 512 anchored rows fail it and are excluded from the headline numbers:
  • idx 137 — no legacy (seed-0) comparator row at all -> legacy comparator columns NaN -> outside the common-finite mask;
  • idx 38989 — legacy risk chain not anchored at the anchor row (risk_anchored = False); NaN comparator columns ['cockpit@3s', 'persist@3s', 'risk@3s'] -> outside the common-finite mask;
  • idx 6209 — legacy risk chain not anchored at the anchor row (risk_anchored = False); NaN comparator columns ['cockpit@3s', 'persist@3s', 'risk@3s'] -> outside the common-finite mask. All three are not_warranted (two manual, one engaged); the 512-row file keeps them with in_gold_509 = False.

Columns

column	meaning
idx	episode id of the annotation tool (stable across every file of this bucket, e.g. human_eval_299/manifest_v1.csv:s15_idx)
uid	pseudonymised route-segment group driver_N__route_M__route_a-b (route_a-b = segment range of the original recording)
driver	pseudonymised device driver_N (one device = one driver = one vehicle)
stratum	manual or engaged: the regime pack whose grid row the episode is anchored on (see above)
split	device split of the row: train (train_979) or test (gold_512 / gold_509)
row	0-based row index of the anchor within the 0.2 s anchor grid of (stratum, split) — the row order of the released grid packs under ../../data/ (see ../../data/README.md)
t0	onset of the episode, route-relative seconds
t_anchor	time of the anchored grid row, route-relative seconds (t0 - 2.0 <= t_anchor <= t0 + 0.2)
label	warranted / not_warranted
y	1 if label == warranted else 0
in_gold_509	(gold_512 only) True for the 509 headline rows

Identifiers

uid and driver are pseudonyms shared by every file in this bucket and by the public BATON release; the raw device ids, the recording timestamps and the annotators' identities were removed. Group / protocol / free-text columns of the annotation store are not released.

Licence

Labels and derived data: CC BY-NC 4.0, inherited from the BATON corpus — see ../../data/LICENSE-DATA. Attribution: "TriDrive authors (anonymous for double-blind review)".