| # Model Card · B2 — Will this event require a road closure? |
|
|
| *Generated 2026-09-02 from `reports/` — do not edit by hand.* |
|
|
| ## What this model answers |
|
|
| Will this event require a road closure? |
|
|
| ## Caveats |
|
|
| - 8.727% positive class. Read PR-AUC against that prevalence, not against 1.0 - the best family is a 4.1x lift over it. |
|
|
| ## Results |
|
|
| Ranked by **PR-AUC ↑** (higher is better). Every family that ran is listed, including those that lost — the comparison is the deliverable. Figures are means over 5 chronological folds, ± one standard deviation. |
|
|
| | Model | Family | PR-AUC ↑ | ROC-AUC ↑ | Brier ↓ | Best F1 ↑ | Fit | Beats baseline | |
| |---|---|---:|---:|---:|---:|---:|---| |
| | `catboost` | gbdt | 0.3615 ± 0.071 | 0.7684 | 0.0688 | 0.4323 | 3s | yes | |
| | `random_forest` | trees | 0.3578 ± 0.080 | 0.7723 | 0.0712 | 0.4326 | 1s | yes | |
| | `lightgbm` | gbdt | 0.3471 ± 0.060 | 0.7583 | 0.0746 | 0.4144 | 6s | yes | |
| | `xgboost` | gbdt | 0.3445 ± 0.081 | 0.7565 | 0.0703 | 0.4093 | 1s | yes | |
| | `logistic` | linear | 0.3304 ± 0.065 | 0.7584 | 0.0713 | 0.3818 | 0s | yes | |
| | `hist_gbm` | gbdt | 0.3269 ± 0.061 | 0.7366 | 0.0724 | 0.3855 | 7s | yes | |
| | `extra_trees` | trees | 0.3024 ± 0.083 | 0.7443 | 0.0734 | 0.3883 | 0s | yes | |
| | `prior` | baseline | 0.0873 ± 0.015 | 0.5000 | 0.0798 | 0.1602 | 0s | *baseline* | |
|
|
| **Best learned family: `catboost`** at 0.3615, 4.1x the strongest baseline (`prior` at 0.0873). |
|
|
| ## Reproduction |
|
|
| ```bash |
| make data && make ingest && make features |
| make train-b |
| ``` |
|
|
| Seed `42`. Splits are chronological, never random. |
|
|
