Buckets:

AsianPlayer
/

TriDrive

Files

xet

AsianPlayer/TriDrive / benchmark /tables /release_tables /baton /k4_input_effect.md

AsianPlayer

21 days ago

preview code

download

raw

3.89 kB

# Stage K4in: causal vs oldin — paired device-cluster bootstrap (eval/paired_compare_joint.py, n_boot 1000), test split, both regimes
Both models are scored on the CAUSAL targets; A = causal-ROI inputs, B = old route-median inputs (input-isolation control) — the K4 input effect.
Seeds with rows: [np.int64(0), np.int64(1), np.int64(2)] (kill rule needs 3: ≥ 2/3 seeds CI excluding 0, same sign, |mean Δ| > seed sd).  Source: `results/paired_compare.csv` tags `K4in_s<k>_causal_vs_oldin`.  Cells: own endpoints (onset PR-AUC@3s), cross-leg endpoints (C1 / C2, PR-AUC + AUROC), hazard NLL, continuous emissions, nr / cooccur Brier.  Driver-leg cells use the route-median (non-causal) ROI inputs (diagnostic, K4 pending).

### Main comparison: A = causal, B = oldin

| regime | cell | metric | A s0 | A s1 | A s2 | Δ s0 [CI] | Δ s1 [CI] | Δ s2 [CI] | mean Δ ± sd | verdict |
|---|---|---|---|---|---|---|---|---|---|---|
| manual | drv_onset/hand_leave/3s | pr_auc | 0.4238 | 0.4172 | 0.4385 | +0.1695 [+0.1410,+0.2230] | +0.1637 [+0.1340,+0.2164] | +0.2303 [+0.1711,+0.2514] | +0.1878 ± 0.0369 (n=3, pos=36693) | BEATS |
| manual | drv_onset/hand_return/3s | pr_auc | 0.7533 | 0.7294 | 0.7456 | +0.3468 [+0.2355,+0.4473] | +0.2617 [+0.1592,+0.4003] | +0.3378 [+0.2450,+0.4583] | +0.3155 ± 0.0468 (n=3, pos=20868) | BEATS |
| manual | drv_onset/head_turn_onset/3s | pr_auc | 0.4568 | 0.4720 | 0.4761 | +0.1331 [+0.0905,+0.1472] | +0.1244 [+0.0734,+0.1423] | +0.1413 [+0.0643,+0.1672] | +0.1329 ± 0.0085 (n=3, pos=61049) | BEATS |
| manual | hazard_nll/hand_leave | nll (lower better) | 0.1349 | 0.1378 | 0.1384 | -0.0409 [-0.0797,-0.0299] | -0.0375 [-0.0740,-0.0271] | -0.0371 [-0.0727,-0.0271] | -0.0385 ± 0.0021 (n=3, pos=69578) | BEATS |
| manual | hazard_nll/hand_return | nll (lower better) | 0.6260 | 0.6480 | 0.6368 | -0.1583 [-0.2050,-0.0960] | -0.1340 [-0.1711,-0.0810] | -0.1035 [-0.1340,-0.0741] | -0.1319 ± 0.0274 (n=3, pos=68711) | BEATS |
| manual | hazard_nll/head_turn_onset | nll (lower better) | 0.1724 | 0.1766 | 0.1732 | -0.0342 [-0.0584,-0.0262] | -0.0228 [-0.0403,-0.0180] | -0.0189 [-0.0367,-0.0140] | -0.0253 ± 0.0079 (n=3, pos=114694) | BEATS |
| engaged | drv_onset/hand_leave/3s | pr_auc | 0.3955 | 0.4018 | 0.4097 | +0.2478 [+0.2029,+0.2867] | +0.2525 [+0.2015,+0.2926] | +0.2566 [+0.2056,+0.2969] | +0.2523 ± 0.0044 (n=3, pos=21986) | BEATS |
| engaged | drv_onset/hand_return/3s | pr_auc | 0.6963 | 0.6928 | 0.6972 | +0.3999 [+0.3139,+0.4578] | +0.3418 [+0.2553,+0.4146] | +0.3549 [+0.2668,+0.4453] | +0.3655 ± 0.0305 (n=3, pos=16138) | BEATS |
| engaged | drv_onset/head_turn_onset/3s | pr_auc | 0.2987 | 0.3042 | 0.3088 | +0.1452 [+0.1215,+0.1609] | +0.1312 [+0.1113,+0.1465] | +0.1194 [+0.0882,+0.1419] | +0.1319 ± 0.0129 (n=3, pos=47095) | BEATS |
| engaged | hazard_nll/hand_leave | nll (lower better) | 0.1183 | 0.1186 | 0.1219 | -0.0623 [-0.0936,-0.0390] | -0.0614 [-0.0929,-0.0381] | -0.0521 [-0.0850,-0.0290] | -0.0586 ± 0.0057 (n=3, pos=44787) | BEATS |
| engaged | hazard_nll/hand_return | nll (lower better) | 0.5078 | 0.5021 | 0.5056 | -0.1419 [-0.2080,-0.0998] | -0.1247 [-0.1861,-0.0857] | -0.0712 [-0.1026,-0.0504] | -0.1126 ± 0.0368 (n=3, pos=44490) | BEATS |
| engaged | hazard_nll/head_turn_onset | nll (lower better) | 0.1542 | 0.1551 | 0.1535 | -0.0321 [-0.0416,-0.0246] | -0.0282 [-0.0375,-0.0210] | -0.0222 [-0.0306,-0.0153] | -0.0275 ± 0.0050 (n=3, pos=85504) | BEATS |

## K4 (input causality) verdict: **PASS**

- own endpoints WORSE: none
- cross-leg endpoints BEATS: ['manual|drv_onset|hand_leave|3s|pr_auc', 'manual|drv_onset|hand_return|3s|pr_auc', 'manual|drv_onset|head_turn_onset|3s|pr_auc', 'engaged|drv_onset|hand_leave|3s|pr_auc', 'engaged|drv_onset|hand_return|3s|pr_auc', 'engaged|drv_onset|head_turn_onset|3s|pr_auc']
- own endpoints BEATS: none
- INCONCLUSIVE own endpoints (mean Δ): none

Xet Storage Details

Size:: 3.89 kB
Xet hash:: f385fff58fc6bce1ddb40f402b2458e60eeb6b98baebedb498eab3b2e9dbd377

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.

	# Stage K4in: causal vs oldin — paired device-cluster bootstrap (eval/paired_compare_joint.py, n_boot 1000), test split, both regimes

	Both models are scored on the CAUSAL targets; A = causal-ROI inputs, B = old route-median inputs (input-isolation control) — the K4 input effect.

	Seeds with rows: [np.int64(0), np.int64(1), np.int64(2)] (kill rule needs 3: ≥ 2/3 seeds CI excluding 0, same sign, \|mean Δ\| > seed sd). Source: `results/paired_compare.csv` tags `K4in_s<k>_causal_vs_oldin`. Cells: own endpoints (onset PR-AUC@3s), cross-leg endpoints (C1 / C2, PR-AUC + AUROC), hazard NLL, continuous emissions, nr / cooccur Brier. Driver-leg cells use the route-median (non-causal) ROI inputs (diagnostic, K4 pending).

	### Main comparison: A = causal, B = oldin

	\| regime \| cell \| metric \| A s0 \| A s1 \| A s2 \| Δ s0 [CI] \| Δ s1 [CI] \| Δ s2 [CI] \| mean Δ ± sd \| verdict \|
	\|---\|---\|---\|---\|---\|---\|---\|---\|---\|---\|---\|
	\| manual \| drv_onset/hand_leave/3s \| pr_auc \| 0.4238 \| 0.4172 \| 0.4385 \| +0.1695 [+0.1410,+0.2230] \| +0.1637 [+0.1340,+0.2164] \| +0.2303 [+0.1711,+0.2514] \| +0.1878 ± 0.0369 (n=3, pos=36693) \| BEATS \|
	\| manual \| drv_onset/hand_return/3s \| pr_auc \| 0.7533 \| 0.7294 \| 0.7456 \| +0.3468 [+0.2355,+0.4473] \| +0.2617 [+0.1592,+0.4003] \| +0.3378 [+0.2450,+0.4583] \| +0.3155 ± 0.0468 (n=3, pos=20868) \| BEATS \|
	\| manual \| drv_onset/head_turn_onset/3s \| pr_auc \| 0.4568 \| 0.4720 \| 0.4761 \| +0.1331 [+0.0905,+0.1472] \| +0.1244 [+0.0734,+0.1423] \| +0.1413 [+0.0643,+0.1672] \| +0.1329 ± 0.0085 (n=3, pos=61049) \| BEATS \|
	\| manual \| hazard_nll/hand_leave \| nll (lower better) \| 0.1349 \| 0.1378 \| 0.1384 \| -0.0409 [-0.0797,-0.0299] \| -0.0375 [-0.0740,-0.0271] \| -0.0371 [-0.0727,-0.0271] \| -0.0385 ± 0.0021 (n=3, pos=69578) \| BEATS \|
	\| manual \| hazard_nll/hand_return \| nll (lower better) \| 0.6260 \| 0.6480 \| 0.6368 \| -0.1583 [-0.2050,-0.0960] \| -0.1340 [-0.1711,-0.0810] \| -0.1035 [-0.1340,-0.0741] \| -0.1319 ± 0.0274 (n=3, pos=68711) \| BEATS \|
	\| manual \| hazard_nll/head_turn_onset \| nll (lower better) \| 0.1724 \| 0.1766 \| 0.1732 \| -0.0342 [-0.0584,-0.0262] \| -0.0228 [-0.0403,-0.0180] \| -0.0189 [-0.0367,-0.0140] \| -0.0253 ± 0.0079 (n=3, pos=114694) \| BEATS \|
	\| engaged \| drv_onset/hand_leave/3s \| pr_auc \| 0.3955 \| 0.4018 \| 0.4097 \| +0.2478 [+0.2029,+0.2867] \| +0.2525 [+0.2015,+0.2926] \| +0.2566 [+0.2056,+0.2969] \| +0.2523 ± 0.0044 (n=3, pos=21986) \| BEATS \|
	\| engaged \| drv_onset/hand_return/3s \| pr_auc \| 0.6963 \| 0.6928 \| 0.6972 \| +0.3999 [+0.3139,+0.4578] \| +0.3418 [+0.2553,+0.4146] \| +0.3549 [+0.2668,+0.4453] \| +0.3655 ± 0.0305 (n=3, pos=16138) \| BEATS \|
	\| engaged \| drv_onset/head_turn_onset/3s \| pr_auc \| 0.2987 \| 0.3042 \| 0.3088 \| +0.1452 [+0.1215,+0.1609] \| +0.1312 [+0.1113,+0.1465] \| +0.1194 [+0.0882,+0.1419] \| +0.1319 ± 0.0129 (n=3, pos=47095) \| BEATS \|
	\| engaged \| hazard_nll/hand_leave \| nll (lower better) \| 0.1183 \| 0.1186 \| 0.1219 \| -0.0623 [-0.0936,-0.0390] \| -0.0614 [-0.0929,-0.0381] \| -0.0521 [-0.0850,-0.0290] \| -0.0586 ± 0.0057 (n=3, pos=44787) \| BEATS \|
	\| engaged \| hazard_nll/hand_return \| nll (lower better) \| 0.5078 \| 0.5021 \| 0.5056 \| -0.1419 [-0.2080,-0.0998] \| -0.1247 [-0.1861,-0.0857] \| -0.0712 [-0.1026,-0.0504] \| -0.1126 ± 0.0368 (n=3, pos=44490) \| BEATS \|
	\| engaged \| hazard_nll/head_turn_onset \| nll (lower better) \| 0.1542 \| 0.1551 \| 0.1535 \| -0.0321 [-0.0416,-0.0246] \| -0.0282 [-0.0375,-0.0210] \| -0.0222 [-0.0306,-0.0153] \| -0.0275 ± 0.0050 (n=3, pos=85504) \| BEATS \|

	## K4 (input causality) verdict: PASS

	- own endpoints WORSE: none
	- cross-leg endpoints BEATS: ['manual\|drv_onset\|hand_leave\|3s\|pr_auc', 'manual\|drv_onset\|hand_return\|3s\|pr_auc', 'manual\|drv_onset\|head_turn_onset\|3s\|pr_auc', 'engaged\|drv_onset\|hand_leave\|3s\|pr_auc', 'engaged\|drv_onset\|hand_return\|3s\|pr_auc', 'engaged\|drv_onset\|head_turn_onset\|3s\|pr_auc']
	- own endpoints BEATS: none
	- INCONCLUSIVE own endpoints (mean Δ): none