AsianPlayer/TriDrive/questionnaire - Storage Bucket

TriDrive in-car user study — questionnaire forms, data, results

This folder is the questionnaire part of the public release of TriDrive (development code name CockpitWM; the analysis scripts and results/SUMMARY.md still use the development name in places). The paper is under double-blind review and this release is anonymous: no participant identities, name tables, audio/video recordings, cabin imagery or raw vehicle logs are included — only the anonymised forms (P01–P14), the tables derived from them, the scripts and the figures.

Design

• Within-participant paired comparison with two conditions: C = TriDrive driver-monitoring prompts (comma four device with an external GPU running the TriDrive model) and S = stock openpilot driver monitoring. Each participant drove the same route twice in their own openpilot-supported car, once with each system, with two visually identical devices; the driving order was counterbalanced.
• 14 participants, P01–P14. Form forms/driverNN.pdf is participant PNN (driver01 = P01, …, driver14 = P14).
• Recording convention (experimenter): on every form, block 1 ("After the first drive", items 1–5) records the TriDrive condition and block 2 ("After the second drive", items 6–10) records stock openpilot, irrespective of the actual driving order. Hence D1 = C and D2 = S for every respondent in the tables; "first drive" in END1/END5 means TriDrive. The actual order was not written on the forms, so an order effect cannot be assessed (results/SUMMARY.md §5.4).
• 12 forms carry AcroForm field values; two forms were completed by hand (circled answers) and were read from the rendered pages: driver06 (P06) and driver11 (P11) (source = manual_visual in results/questionnaire_wide.csv). P13's END1 answer was corrected from the experimenter's record (source = acroform+override(END1)).

Items (per drive; 7-point Likert, 1 = strongly disagree … 7 = strongly agree, plus "Cannot judge")

id	form item (D1 / D2)	statement
APP1	1 / 6	The driver-monitoring prompts during this drive fit the driving situations at the time.
APP2	2 / 7	The driver-monitoring prompts during this drive were necessary for me.
APP3	3 / 8	Overall, the driver-monitoring prompts during this drive were appropriate. (positively worded)
TIM1	4 / 9	The driver-monitoring prompts during this drive came at the right time.
ANN1	5 / 10	The driver-monitoring prompts during this drive annoyed me. (higher = more annoying)
END1	11	Which drive's driver-monitoring prompts did you prefer? (first drive / second drive / no difference)
END5	12	Free comment on the prompts across the two drives (optional; verbatim in the tables).

Scoring

• EP0 appropriateness = mean(APP1, APP2, APP3) per drive; APP3 is positively worded in the fielded form and is not reverse-scored; the construct needs at least 2 of the 3 items.
• ES1 timeliness = TIM1. ES2 annoyance = ANN1, higher = more annoying, so a positive difference means TriDrive was rated more annoying.
• Paired difference per participant Δ = C − S (TriDrive minus openpilot).
• END1 decoded with the convention above: first → prefer C (TriDrive), second → prefer S (openpilot), no difference → none.
• Statistics (scripts/analysis_questionnaire_v1.py, scripts/paired_endpoints.py): participant-level paired bootstrap of the mean Δ (2,000 resamples, seed 20260922, percentile 95 % CI) and a two-sided Wilcoxon signed-rank test (scipy, zero_method "wilcox"); the verdict (above / below / inconclusive) follows the CI, p is reported beside it. Preference: shares with Wilson 95 % CIs and a participant-level bootstrap CI for share(prefer C) − share(prefer S).

Missing data

"Cannot judge" and unanswered items are missing. Primary analysis (primary_complete_pairs): per endpoint, only participants with both the C and the S construct score (n = 14 appropriateness, 12 timeliness, 12 annoyance). Sensitivity (sensitivity_NP_neutral): a missing construct score is set to the neutral midpoint 4 (n = 14 for all three); no verdict changes.

Headline numbers (results/endpoints.csv, results/preference.csv)

endpoint	pairs	mean C (TriDrive)	mean S (openpilot)	mean Δ (C − S)	95 % CI	Wilcoxon p	verdict
EP0 appropriateness	14	5.27	3.49	+1.79	[+0.86, +2.75]	0.0076	above
ES1 timeliness	12	5.75	3.08	+2.67	[+1.33, +4.08]	0.0078	above
ES2 annoyance	12	2.58	1.42	+1.17	[+0.33, +2.08]	0.0312	above

Sensitivity NP-neutral (n = 14 each): Δ = +1.79 [0.86, 2.75] (p 0.0076) / +2.50 [1.36, 3.71] (p 0.0048) / +1.07 [0.36, 1.93] (p 0.0278); all verdicts unchanged.

Preference (END1, n = 14): prefer TriDrive 7 (0.50, Wilson 95 % CI [0.27, 0.73]); prefer openpilot 4 (0.29, [0.12, 0.55]); no difference 3 (0.21, [0.08, 0.48]) — the share difference is inconclusive at this sample size. Annoyance must be read together with exposure: the stock monitor prompted rarely and late in these drives, so its low annoyance is not evidence of better calibration (results/SUMMARY.md §5.2).

Files

• README.md — this file
• LICENSE — CC BY 4.0 notice
• SHA256SUMS — sha256 of every file in this folder
• forms/README.md — note on the forms (from the supplementary material)
• forms/questionnaire_blank.pdf — the form as fielded (12 items, fillable)
• forms/driver01.pdf … forms/driver14.pdf — the 14 completed forms (P01–P14); 12 with form-field values, driver06 and driver11 hand-filled (circled answers)
• results/questionnaire.csv — long table, one row per participant × item (data-schema layout: participant_id, session_index, condition_code, form, item_id, raw_value, scored_value, free_text, …)
• results/questionnaire_wide.csv — one row per participant: source, D1.* (TriDrive), D2.* (openpilot), END1, END5
• results/per_participant.csv — construct scores C / S / Δ per participant, preference, comment themes, verbatim comment
• results/endpoints.csv — the six endpoint rows (primary complete pairs + NP-neutral sensitivity): n, means, Δ, CI, Wilcoxon p, verdict
• results/endpoints.md — the same results as markdown, with the per-participant table and the exposure note
• results/preference.csv — END1 counts and shares with Wilson 95 % CIs
• results/SUMMARY.md — full written summary: design, raw answers, scores, endpoints, analysis, limitations, verbatim comments
• scripts/analysis_questionnaire_v1.py — endpoint analysis of the fielded short form (writes endpoints.csv/.md, preference.csv, per_participant.csv)
• scripts/parse_questionnaire_pdfs.py — PDF form fields → questionnaire.csv / questionnaire_wide.csv / sessions.csv
• scripts/paired_endpoints.py — pre-specified paired analysis helpers (bootstrap, Wilcoxon, verdict rules) reused by the analysis
• scripts/analysis.py — full protocol analysis (also covers endpoints that were not fielded in the short form)
• scripts/make_user_study_fig_paper.py — figures/user_study_bars.pdf/.png (paper figure, complete pairs)
• scripts/make_user_study_fig_n14.py — figures/user_study_bars_n14.pdf/.png (NP-neutral sensitivity, n = 14)
• scripts/make_user_study_bars.py — teaser chart from the result tables (figures/user_study_bars_bare.*)
• scripts/PATHS_CHANGED.md — the absolute paths rewritten to relative ones in the scripts
• figures/user_study_bars.pdf — paper figure: three paired-bar panels (appropriateness, timeliness, annoyance), complete pairs
• figures/user_study_bars.png — PNG of the same
• figures/user_study_bars_n14.pdf — sensitivity variant, missing scores imputed with 4 (n = 14)
• figures/user_study_bars_n14.png — PNG of the same
• figures/user_study_bars_bare.pdf — teaser version: appropriateness / timeliness bars and the preference split, minimal text
• figures/user_study_bars_bare.png — PNG of the same

Reproducing the tables and figures

Run from the root of this folder. results/sessions.csv is not shipped; it is one row per participant and drive (participant_id,session_index,condition_code,condition with 1,X,C and 2,Y,S for every participant) and is written by scripts/parse_questionnaire_pdfs.py.

python scripts/parse_questionnaire_pdfs.py --pdf-dir forms --exclude questionnaire_blank --ids <outside>/ids.csv \
       --manual <outside>/manual.json --overrides <outside>/overrides.json --out results
python scripts/analysis_questionnaire_v1.py --questionnaire results/questionnaire.csv --sessions results/sessions.csv --out results
python scripts/make_user_study_fig_paper.py; python scripts/make_user_study_fig_n14.py
python scripts/make_user_study_bars.py --out figures/user_study_bars

--ids maps form stems to P-ids (driver01 → P01 … driver14 → P14; created automatically in sorted order); --manual supplies the hand-read values of driver06 / driver11 and --overrides the P13 END1 correction — both are recorded in results/questionnaire_wide.csv (source column), so the shipped tables can be checked line by line against the forms.

Privacy and provenance

The PDFs keep every answer (form-field values and free text) but carry no participant names, no document metadata other than the title (= file stem) and no embedded provenance manifests. The free-text comments were given with the instruction not to include identifying details and are reproduced verbatim. Nothing in this folder links a P-id to a person.

Licence

Everything in this folder is released under CC BY 4.0 (see LICENSE, https://creativecommons.org/licenses/by/4.0/).