| # Model Card · A3 — Will this field capture be rejected at officer review? |
|
|
| *Generated 2026-09-02 from `reports/` — do not edit by hand.* |
|
|
| ## What this model answers |
|
|
| Will this field capture be rejected at officer review? |
|
|
| ## Caveats |
|
|
| - NOT YET ANSWERABLE (F11). The review process changed regime mid-window: February 2024 rejection jumped to 0.790 from ~0.28 while reviewed volume collapsed sixfold. |
| - Every family is miscalibrated as a result - Brier scores are worse than the prior baseline. This is a property of the label, not the models. |
|
|
| ## Results |
|
|
| Ranked by **PR-AUC ↑** (higher is better). Every family that ran is listed, including those that lost — the comparison is the deliverable. |
|
|
| | Model | Family | PR-AUC ↑ | ROC-AUC ↑ | Brier ↓ | Best F1 ↑ | Fit | Beats baseline | |
| |---|---|---:|---:|---:|---:|---:|---| |
| | `xgboost` | gbdt | 0.3088 | 0.5750 | 0.7122 | 0.3962 | 2s | yes | |
| | `stacked` | ensemble | 0.3084 | 0.5796 | 0.4401 | 0.4023 | 48s | yes | |
| | `catboost` | gbdt | 0.2987 | 0.5630 | 0.6817 | 0.4033 | 4s | yes | |
| | `hist_gbm` | gbdt | 0.2955 | 0.5575 | 0.7166 | 0.3960 | 4s | yes | |
| | `random_forest` | trees | 0.2919 | 0.5683 | 0.5619 | 0.4040 | 2s | yes | |
| | `logistic` | linear | 0.2893 | 0.5642 | 0.7211 | 0.4068 | 0s | yes | |
| | `lightgbm` | gbdt | 0.2838 | 0.5522 | 0.7250 | 0.3954 | 2s | yes | |
| | `extra_trees` | trees | 0.2825 | 0.5544 | 0.4573 | 0.4027 | 1s | yes | |
| | `prior` | baseline | 0.2459 | 0.5000 | 0.1881 | 0.3947 | 0s | *baseline* | |
|
|
| **Best learned family: `xgboost`** at 0.3088, +25.6% against the strongest baseline (`prior` at 0.2459). |
|
|
| ## Reproduction |
|
|
| ```bash |
| make data && make ingest && make features |
| make train |
| ``` |
|
|
| Seed `42`. Splits are chronological, never random. |
|
|
