Spaces:
Running
Running
| {"generated_by": "backend/data/corpus_finding.py", "question": "Does the Driver State Index separate races that were genuinely different to drive?", "why_it_matters": "This is the strongest available test of whether DSI measures anything real. The slate was chosen for contrast before any of it was analysed: a soaking-wet scramble, a dry processional afternoon as a control, a race run in punishing heat. If those do not separate, the index is measuring something other than driver state.", "setup": {"races": 6, "messages_analysed": 1050, "messages_scored": "engineer-attributed transmissions excluded", "n_scored": 762, "calibration": "pooled across all 12 races in the corpus - per-race calibration centres every race on 50.0 by construction and would have made this test impossible", "control_race": "2023 Italian", "predictions_made_in_advance": {"high": ["2020 Turkish (wet)", "2019 German (wet chaos)", "2023 Monaco (dry to wet)"], "low": ["2023 Italian (dry, processional - the control)"], "fatigue": ["2023 Qatar (extreme heat)"]}}, "results": [{"race_id": "2020_Turkish_Grand_Prix", "race": "2020 Turkish", "season": 2020, "n": 139, "mean_dsi": 55.5, "sd": 17.2, "ci95": [52.7, 58.4], "stressed_pct": 25, "fatigued_pct": 22, "calibration_source": "leave-one-race-out", "messages_total": 190}, {"race_id": "2023_Qatar_Grand_Prix", "race": "2023 Qatar", "season": 2023, "n": 116, "mean_dsi": 51.3, "sd": 19.1, "ci95": [47.8, 54.7], "stressed_pct": 21, "fatigued_pct": 35, "calibration_source": "leave-one-race-out", "messages_total": 148}, {"race_id": "2021_Abu_Dhabi_Grand_Prix", "race": "2021 Abu Dhabi", "season": 2021, "n": 111, "mean_dsi": 51.2, "sd": 16.2, "ci95": [48.2, 54.2], "stressed_pct": 19, "fatigued_pct": 30, "calibration_source": "leave-one-race-out", "messages_total": 168}, {"race_id": "2023_Monaco_Grand_Prix", "race": "2023 Monaco", "season": 2023, "n": 165, "mean_dsi": 50.5, "sd": 17.4, "ci95": [47.8, 53.1], "stressed_pct": 18, "fatigued_pct": 38, "calibration_source": "leave-one-race-out", "messages_total": 207}, {"race_id": "2023_Italian_Grand_Prix", "race": "2023 Italian", "season": 2023, "n": 130, "mean_dsi": 47.4, "sd": 16.4, "ci95": [44.6, 50.3], "stressed_pct": 17, "fatigued_pct": 38, "calibration_source": "leave-one-race-out", "messages_total": 180}, {"race_id": "2019_German_Grand_Prix", "race": "2019 German", "season": 2019, "n": 101, "mean_dsi": 47.0, "sd": 18.0, "ci95": [43.5, 50.5], "stressed_pct": 17, "fatigued_pct": 38, "calibration_source": "leave-one-race-out", "messages_total": 157}], "contrasts_vs_dry_control": [{"race": "2020 Turkish", "diff": 8.1, "z": 3.96, "p": 0.0001, "survives_bonferroni": true, "predicted": "high"}, {"race": "2023 Qatar", "diff": 3.9, "z": 1.71, "p": 0.0873, "survives_bonferroni": false, "predicted": "fatigue"}, {"race": "2021 Abu Dhabi", "diff": 3.8, "z": 1.81, "p": 0.0711, "survives_bonferroni": false, "predicted": "not predicted"}, {"race": "2023 Monaco", "diff": 3.1, "z": 1.57, "p": 0.1165, "survives_bonferroni": false, "predicted": "high"}, {"race": "2019 German", "diff": -0.4, "z": -0.17, "p": 0.8616, "survives_bonferroni": false, "predicted": "high"}], "prediction_scorecard": [{"race": "2020 Turkish", "prediction": "high", "rationale": "wet", "held": true, "detail": "2020 Turkish (wet) predicted high: +8.1 vs the control, p=0.0001, survives Bonferroni 0.010."}, {"race": "2019 German", "prediction": "high", "rationale": "wet chaos", "held": false, "detail": "2019 German (wet chaos) predicted high: -0.4 vs the control, p=0.8616, does not survive Bonferroni 0.010."}, {"race": "2023 Monaco", "prediction": "high", "rationale": "dry to wet", "held": false, "detail": "2023 Monaco (dry to wet) predicted high: +3.1 vs the control, p=0.1165, does not survive Bonferroni 0.010."}, {"race": "2023 Italian", "prediction": "low", "rationale": "dry, processional - the control", "held": false, "detail": "2023 Italian (dry, processional - the control) predicted lowest: it is above 2019 German in the slate at 47.4."}, {"race": "2023 Qatar", "prediction": "fatigue", "rationale": "extreme heat", "held": false, "detail": "2023 Qatar (extreme heat) predicted highest fatigue: 21% Stressed and 35% Fatigued, behind 2020 Turkish Stressed share in the slate."}], "bonferroni_note": "5 contrasts were tested, so the corrected threshold is p < 0.010. 2023 Qatar's p = 0.087 does not survive it and is not claimed; 2021 Abu Dhabi's p = 0.071 does not survive it and is not claimed; 2023 Monaco's p = 0.117 does not survive it and is not claimed; 2019 German's p = 0.862 does not survive it and is not claimed.", "verdict": "PARTIAL SUPPORT - 1 of 5 predictions held, 4 clearly failed.", "what_held": ["2020 Turkish (wet) predicted high: +8.1 vs the control, p=0.0001, survives Bonferroni 0.010."], "what_failed": "2019 German (wet chaos) predicted high: -0.4 vs the control, p=0.8616, does not survive Bonferroni 0.010. 2023 Monaco (dry to wet) predicted high: +3.1 vs the control, p=0.1165, does not survive Bonferroni 0.010. 2023 Italian (dry, processional - the control) predicted lowest: it is above 2019 German in the slate at 47.4. 2023 Qatar (extreme heat) predicted highest fatigue: 21% Stressed and 35% Fatigued, behind 2020 Turkish Stressed share in the slate.", "effect_size": "The full spread across the slate is 8.5 DSI points against a within-race sd of about 17, so Cohen's d is roughly 0.49 - a small-to-moderate effect. The index separates these races, but not dramatically.", "not_predicted": {"note": "6 further races were built after the predictions were registered, to test the recording-era confound. They were not predicted in either direction and are reported descriptively only. Folding them into the prediction test would turn a confirmatory result into an exploratory one.", "races": [{"race_id": "2023_Spanish_Grand_Prix", "race": "2023 Spanish", "season": 2023, "n": 128, "mean_dsi": 54.4, "sd": 18.3, "ci95": [51.2, 57.6], "stressed_pct": 23, "fatigued_pct": 34, "calibration_source": "leave-one-race-out", "messages_total": 162}, {"race_id": "2023_United_States_Grand_Prix", "race": "2023 United States", "season": 2023, "n": 121, "mean_dsi": 54.2, "sd": 15.8, "ci95": [51.4, 57.0], "stressed_pct": 25, "fatigued_pct": 28, "calibration_source": "leave-one-race-out", "messages_total": 162}, {"race_id": "2023_Mexico_City_Grand_Prix", "race": "2023 Mexico City", "season": 2023, "n": 128, "mean_dsi": 52.0, "sd": 16.8, "ci95": [49.1, 54.9], "stressed_pct": 19, "fatigued_pct": 33, "calibration_source": "leave-one-race-out", "messages_total": 187}, {"race_id": "2023_Miami_Grand_Prix", "race": "2023 Miami", "season": 2023, "n": 123, "mean_dsi": 50.6, "sd": 17.7, "ci95": [47.5, 53.8], "stressed_pct": 15, "fatigued_pct": 32, "calibration_source": "leave-one-race-out", "messages_total": 158}, {"race_id": "2023_Dutch_Grand_Prix", "race": "2023 Dutch", "season": 2023, "n": 130, "mean_dsi": 48.0, "sd": 14.5, "ci95": [45.5, 50.5], "stressed_pct": 20, "fatigued_pct": 31, "calibration_source": "leave-one-race-out", "messages_total": 172}, {"race_id": "2023_Japanese_Grand_Prix", "race": "2023 Japanese", "season": 2023, "n": 120, "mean_dsi": 47.9, "sd": 20.2, "ci95": [44.3, 51.5], "stressed_pct": 24, "fatigued_pct": 37, "calibration_source": "leave-one-race-out", "messages_total": 151}]}, "confound_we_cannot_rule_out": "RESOLVED - see _era_analysis.json. This previously read as an open confound: the slate spanned several seasons and radio encoding is not constant across them, so some separation could have been recording era. It was then tested directly by building 9 races from 2023 alone. Within that single season races still spread 6.95 DSI points against 8.5 across all eras, 2023 Spanish vs 2023 Italian (both 2023) differs by +7.0 (p=0.0013, surviving Bonferroni), and there is no systematic offset between eras (-0.6 points, p=0.5867). Recording era does not explain the separation.", "confound_status": "resolved", "honest_summary": "DSI separates races in the direction predicted, with the dry processional control correctly at the bottom, and the era confound has now been tested and ruled out on a 9-race within-season slate. It remains evidence that the index tracks race conditions, not proof it tracks driver stress specifically: the effect is modest (Cohen's d about 0.49), 4 predictions still failed."} |