# Impact pipeline against the gap key

Model `claude-haiku-5-5`; 12 circulars, every one named in [`company/GAP_KEY.md`](https://github.com/nihardamera/RegRadar/blob/main/company/GAP_KEY.md). Generated by `python -m regradar.evaluate_impact score` from `eval/impact/*.json`. API cost of the run: $0.65.

| Measure | Result |
| --- | --- |
| Planted gaps found, strict: the flag states the rule's key fact | **11/12** = 92% [65, 99] (Wilson 95%) |
| Planted gaps found, lenient: a flag cites the right policy section | 12/12 = 100% [76, 100] |
| Traps flagged as gaps (N1-N3; lower is better), strict / lenient | **2/3** / 2/3 |
| Gap or partial flags in all (share of applicable obligations) | 328 (29%) |
| Flags matching no key row (need a person) | 234 |
| Obligations extracted / rated not applicable / not covered | 1492 / 369 / 660 |
| Items in the review queue | 72 |
| Reports failing a code check | 0 |

Per key row (Y = found or, for N rows, flagged):

| Match | G1 | G2 | G3 | G4 | G5 | G6 | G7 | G8 | G9 | G10 | G11 | G12 | N1 | N2 | N3 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| strict | Y | Y | Y | Y | Y | - | Y | Y | Y | Y | Y | Y | Y | - | Y |
| lenient | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | - | Y |

Matching: a gap or partial rating from the key's circular that cites the key's policy in the same section (heading number); strict also requires the finding or action to state the rule's key fact (`KEY_FACT` in `regradar/evaluate_impact.py`, taken from the key's rule text). **Recall is not the whole story:** the pipeline flags many obligations, and precision is not measured. The key lists what was planted, not every shortfall, so unkeyed flags are neither right nor wrong until a person reviews them; they are listed in `impact_<model>.json`.