Spaces:
Sleeping
Sleeping
solidprivacy-nl commited on
Commit ·
c49e63e
1
Parent(s): 76b728e
Update risks for MVP UI cleanup planning
Browse files- RISK_REGISTER.md +31 -129
RISK_REGISTER.md
CHANGED
|
@@ -19,40 +19,14 @@ Risk:
|
|
| 19 |
Sensitive data remains in the scrubbed output and the user wrongly trusts the result.
|
| 20 |
```
|
| 21 |
|
| 22 |
-
Current mitigations
|
| 23 |
-
|
| 24 |
-
- Human review workflow.
|
| 25 |
-
- Review guidance and final review summary.
|
| 26 |
-
- Dutch legal recall gap tests and pattern-fix verification are present.
|
| 27 |
-
- Synthetic gold-label corpus exists and has been expanded.
|
| 28 |
-
- Minimal diagnostic recall benchmark runner exists.
|
| 29 |
-
- Diagnostic report artifact workflow exists.
|
| 30 |
-
- First and cleaned artifact reviews exist.
|
| 31 |
-
- Planning-only threshold policy exists.
|
| 32 |
-
- PERSON-name coverage review classifies the remaining missed `PERSON` labels.
|
| 33 |
-
- PERSON-name false-negative risk has diagnostic test coverage in `tests/test_recall_person_name_coverage_diagnostics.py`.
|
| 34 |
-
- PERSON-name false-negative risk has a planning/specification-only recognition improvement plan in `RECALL_PERSON_NAME_RECOGNIZER_PLAN.md`.
|
| 35 |
-
- PERSON-name false-negative risk has contract test coverage for future recognizer behavior in `tests/test_recall_person_name_recognizer_contracts.py` and `tests/fixtures/person_name_recognizer_contract_cases.json`.
|
| 36 |
-
- PERSON-name false-negative risk is now partially mitigated for contract-backed role/title name cases by `person_name_recognizer_helper.py` and `tests/test_recall_person_name_recognizer_implementation.py`.
|
| 37 |
|
| 38 |
-
|
| 39 |
|
| 40 |
- No accepted production recall/precision threshold exists.
|
| 41 |
- No production benchmark gate exists.
|
| 42 |
- No production safety claim is supported.
|
| 43 |
-
-
|
| 44 |
-
- App-registered recognizer behavior and benchmark artifact impact still need review.
|
| 45 |
-
- Candidate-only weak contexts are not automatic.
|
| 46 |
-
- No candidate scanner change was made.
|
| 47 |
-
- No gate or threshold enforcement was added.
|
| 48 |
-
- Single-surname risk remains bounded by strong-context rules.
|
| 49 |
-
- Remaining non-PERSON gaps include care room/location references, one client-number example and one nested false-positive hit inside a phone-like value.
|
| 50 |
-
- Corpus is synthetic and small.
|
| 51 |
-
|
| 52 |
-
Recommended workpackages:
|
| 53 |
-
|
| 54 |
-
- Next approved package — `WP_RECALL_PERSON_NAME_RECOGNIZER_BENCHMARK_REVIEW`.
|
| 55 |
-
- Other approved packages — `WP_CARE_LOCATION_REFERENCE_CANDIDATE_PLAN`, `WP_CLIENT_REFERENCE_COVERAGE_REVIEW`, `WP_RECALL_BENCHMARK_THRESHOLDS_CONTRACT_TESTS`.
|
| 56 |
|
| 57 |
---
|
| 58 |
|
|
@@ -67,16 +41,7 @@ Risk:
|
|
| 67 |
The Scrub Key is shared, leaked, retained too long, tampered with or mishandled, allowing full re-identification of scrubbed content.
|
| 68 |
```
|
| 69 |
|
| 70 |
-
Current mitigations
|
| 71 |
-
|
| 72 |
-
- UI warnings around Scrub Key reinsert.
|
| 73 |
-
- Local-only/no-AI/no-cloud positioning for reinsert.
|
| 74 |
-
- Scrub Key warning/acknowledgement UI is closed out after Actions/HF/app verification.
|
| 75 |
-
|
| 76 |
-
Gaps:
|
| 77 |
-
|
| 78 |
-
- UI acknowledgements are safety prompts only; they are not encryption, protected storage, automatic deletion or a managed key vault.
|
| 79 |
-
- No approved key recovery model.
|
| 80 |
|
| 81 |
---
|
| 82 |
|
|
@@ -91,16 +56,7 @@ Risk:
|
|
| 91 |
An AI system rewrites, translates, merges or deletes placeholders, causing deterministic reinsert to fail or restore incompletely.
|
| 92 |
```
|
| 93 |
|
| 94 |
-
Current mitigations
|
| 95 |
-
|
| 96 |
-
- Unknown placeholders and not-found placeholders are reported in reinsert audit flows.
|
| 97 |
-
- Placeholder robustness review, future robust format proposal, validation helper, audit helper and synthetic AI-output corruption tests exist.
|
| 98 |
-
|
| 99 |
-
Gaps:
|
| 100 |
-
|
| 101 |
-
- Robust placeholder generation has not been implemented in product flow.
|
| 102 |
-
- No migration or backward-compatibility implementation exists yet.
|
| 103 |
-
- No Scrub Key schema/version support for robust placeholder metadata exists yet.
|
| 104 |
|
| 105 |
---
|
| 106 |
|
|
@@ -115,17 +71,7 @@ Risk:
|
|
| 115 |
DOCX metadata, comments, tracked changes, headers, footers or hidden content contain sensitive data that is not scrubbed or cleaned.
|
| 116 |
```
|
| 117 |
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
- DOCX hidden-content risk review, visibility helpers and report-only hygiene audit are present.
|
| 121 |
-
- Clean DOCX export policy exists.
|
| 122 |
-
- DOCX hygiene audit UI is report-only and does not claim clean DOCX output.
|
| 123 |
-
|
| 124 |
-
Gaps:
|
| 125 |
-
|
| 126 |
-
- Word comments / kantlijncommentaren are still not scrubbed or removed by the current DOCX scrub/reinsert flow.
|
| 127 |
-
- No clean DOCX export implementation exists.
|
| 128 |
-
- No approved export-blocking implementation exists for high-risk hidden content.
|
| 129 |
|
| 130 |
---
|
| 131 |
|
|
@@ -140,29 +86,19 @@ Risk:
|
|
| 140 |
The final product promise is local-first, but the current fast validation surface is the Hugging Face cloud demo.
|
| 141 |
```
|
| 142 |
|
| 143 |
-
|
| 144 |
-
|
| 145 |
-
- Roadmap includes local desktop/offline direction.
|
| 146 |
-
- UI messaging emphasizes local/no-AI/no-cloud where applicable.
|
| 147 |
-
- Local runtime/packaging decision line is completed through packaging deferral.
|
| 148 |
-
|
| 149 |
-
Gaps:
|
| 150 |
-
|
| 151 |
-
- No full offline demonstration.
|
| 152 |
-
- No production security certification is supported.
|
| 153 |
-
- No signed or managed enterprise deployment proof exists.
|
| 154 |
|
| 155 |
---
|
| 156 |
|
| 157 |
-
## R6 — Review UX and
|
| 158 |
|
| 159 |
Status: mitigating
|
| 160 |
-
Impact:
|
| 161 |
|
| 162 |
Risk:
|
| 163 |
|
| 164 |
```text
|
| 165 |
-
The
|
| 166 |
```
|
| 167 |
|
| 168 |
Current mitigations:
|
|
@@ -170,12 +106,21 @@ Current mitigations:
|
|
| 170 |
- Review table remains source of truth and fallback.
|
| 171 |
- Side-by-side review surface, synced scrolling and collapsible review table are live and verified.
|
| 172 |
- Serial review remains available as a non-destructive review aid.
|
|
|
|
| 173 |
|
| 174 |
Gaps:
|
| 175 |
|
| 176 |
-
-
|
| 177 |
-
-
|
| 178 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 179 |
|
| 180 |
---
|
| 181 |
|
|
@@ -190,16 +135,11 @@ Risk:
|
|
| 190 |
Users may assume PDF support means complete restored PDF reinsert or OCR, while the approved scope is text-based extraction to restored TXT only.
|
| 191 |
```
|
| 192 |
|
| 193 |
-
|
| 194 |
-
|
| 195 |
-
- No restored PDF output.
|
| 196 |
-
- No OCR.
|
| 197 |
-
- No PDF-to-DOCX reconstruction.
|
| 198 |
-
- No layout preservation promises.
|
| 199 |
|
| 200 |
---
|
| 201 |
|
| 202 |
-
## R8 — Workflow status
|
| 203 |
|
| 204 |
Status: mitigating
|
| 205 |
Impact: medium
|
|
@@ -207,32 +147,21 @@ Impact: medium
|
|
| 207 |
Risk:
|
| 208 |
|
| 209 |
```text
|
| 210 |
-
Evidence
|
| 211 |
```
|
| 212 |
|
| 213 |
Current mitigations:
|
| 214 |
|
| 215 |
- Coordinator screenshots/evidence are recorded when connector lookup is incomplete.
|
| 216 |
-
- Workpackages record Actions/sync status.
|
| 217 |
- Diagnostic recall benchmark artifact workflow exists.
|
| 218 |
-
-
|
| 219 |
-
-
|
| 220 |
-
- PERSON-name coverage review exists.
|
| 221 |
-
- PERSON-name diagnostic tests exist.
|
| 222 |
-
- PERSON-name recognizer planning exists.
|
| 223 |
-
- PERSON-name contract tests exist.
|
| 224 |
-
- PERSON-name helper implementation tests exist.
|
| 225 |
|
| 226 |
Gaps:
|
| 227 |
|
| 228 |
-
-
|
| 229 |
- No generalized automated status artifact exists yet.
|
| 230 |
|
| 231 |
-
Recommended workpackages:
|
| 232 |
-
|
| 233 |
-
- Next approved benchmark package — `WP_RECALL_PERSON_NAME_RECOGNIZER_BENCHMARK_REVIEW`.
|
| 234 |
-
- Later status package — automated status artifact/check if connector limitations continue to slow closeouts.
|
| 235 |
-
|
| 236 |
---
|
| 237 |
|
| 238 |
## R9 — Dutch legal reference under-detection and role over-masking
|
|
@@ -246,34 +175,7 @@ Risk:
|
|
| 246 |
Dutch legal matter references can be missed or misclassified, while generic legal/care role words can be masked in ways that damage meaning.
|
| 247 |
```
|
| 248 |
|
| 249 |
-
Current mitigations
|
| 250 |
-
|
| 251 |
-
- Human review table remains source of truth and fallback.
|
| 252 |
-
- Recall benchmark spec defines reference classes, context terms to preserve and over-masking traps.
|
| 253 |
-
- Diagnostic runner/report artifacts make benchmark evidence visible.
|
| 254 |
-
- Cleaned artifact has `preserve_term_hit_count = 0`.
|
| 255 |
-
- Planning-only threshold policy exists.
|
| 256 |
-
- PERSON-name coverage review classifies role-linked missed names, including care-role and legal-role examples.
|
| 257 |
-
- PERSON-name diagnostic tests preserve the gap inventory and non-claim boundaries without requiring current recognizers to pass all examples.
|
| 258 |
-
- PERSON-name recognizer planning records a test-first route and value-only role/context boundary before implementation.
|
| 259 |
-
- PERSON-name recognizer contract tests specify future value-only, preserve-term, single-surname and negative-case behavior.
|
| 260 |
-
- PERSON-name helper implementation now enforces value-only role/title matching for contract-backed helper cases.
|
| 261 |
-
|
| 262 |
-
Gaps:
|
| 263 |
-
|
| 264 |
-
- No accepted production threshold exists.
|
| 265 |
-
- No benchmark gate exists.
|
| 266 |
-
- Corpus is synthetic and not exhaustive.
|
| 267 |
-
- PERSON-name false-negative risk is only partially mitigated for contract-backed role/title helper cases.
|
| 268 |
-
- Benchmark artifact impact still needs review.
|
| 269 |
-
- Single-surname examples remain high ambiguity and must stay limited to strong context.
|
| 270 |
-
- Product claim remains blocked.
|
| 271 |
-
|
| 272 |
-
Recommended workpackages:
|
| 273 |
-
|
| 274 |
-
- `WP_RECALL_PERSON_NAME_RECOGNIZER_BENCHMARK_REVIEW`.
|
| 275 |
-
- `WP_CARE_LOCATION_REFERENCE_CANDIDATE_PLAN`.
|
| 276 |
-
- `WP_CLIENT_REFERENCE_COVERAGE_REVIEW`.
|
| 277 |
|
| 278 |
---
|
| 279 |
|
|
@@ -292,6 +194,6 @@ De benchmark bewijst production readiness.
|
|
| 292 |
Allowed wording:
|
| 293 |
|
| 294 |
```text
|
| 295 |
-
|
| 296 |
-
|
| 297 |
```
|
|
|
|
| 19 |
Sensitive data remains in the scrubbed output and the user wrongly trusts the result.
|
| 20 |
```
|
| 21 |
|
| 22 |
+
Current mitigations include human review, review guidance, diagnostic recall benchmark artifacts, PERSON-name diagnostic/contract/helper work and planning-only threshold policy.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
+
Remaining gaps:
|
| 25 |
|
| 26 |
- No accepted production recall/precision threshold exists.
|
| 27 |
- No production benchmark gate exists.
|
| 28 |
- No production safety claim is supported.
|
| 29 |
+
- Human review remains necessary.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
|
| 31 |
---
|
| 32 |
|
|
|
|
| 41 |
The Scrub Key is shared, leaked, retained too long, tampered with or mishandled, allowing full re-identification of scrubbed content.
|
| 42 |
```
|
| 43 |
|
| 44 |
+
Current mitigations include warnings and acknowledgements. UI cleanup must keep Scrub Key clearly separate from ordinary document exports.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
|
| 46 |
---
|
| 47 |
|
|
|
|
| 56 |
An AI system rewrites, translates, merges or deletes placeholders, causing deterministic reinsert to fail or restore incompletely.
|
| 57 |
```
|
| 58 |
|
| 59 |
+
Current mitigations include placeholder robustness helper/test work and reinsert audit reporting.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
|
| 61 |
---
|
| 62 |
|
|
|
|
| 71 |
DOCX metadata, comments, tracked changes, headers, footers or hidden content contain sensitive data that is not scrubbed or cleaned.
|
| 72 |
```
|
| 73 |
|
| 74 |
+
DOCX hygiene audit remains report-only. MVP UI cleanup may make it visually calmer, but audit details must remain available and must not imply a clean-DOCX guarantee.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
|
| 76 |
---
|
| 77 |
|
|
|
|
| 86 |
The final product promise is local-first, but the current fast validation surface is the Hugging Face cloud demo.
|
| 87 |
```
|
| 88 |
|
| 89 |
+
Local/offline installer work remains later. The current focus is making the web prototype workflow credible first.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 90 |
|
| 91 |
---
|
| 92 |
|
| 93 |
+
## R6 — Review UX and interface clarity risk
|
| 94 |
|
| 95 |
Status: mitigating
|
| 96 |
+
Impact: high
|
| 97 |
|
| 98 |
Risk:
|
| 99 |
|
| 100 |
```text
|
| 101 |
+
The interface still feels like a technical prototype, which can reduce confidence and increase review mistakes.
|
| 102 |
```
|
| 103 |
|
| 104 |
Current mitigations:
|
|
|
|
| 106 |
- Review table remains source of truth and fallback.
|
| 107 |
- Side-by-side review surface, synced scrolling and collapsible review table are live and verified.
|
| 108 |
- Serial review remains available as a non-destructive review aid.
|
| 109 |
+
- `MVP_UI_CLEANUP_AND_EXPORT_REDESIGN_PLAN.md` now defines a route to move debug/audit details out of the primary flow and improve export/download hierarchy.
|
| 110 |
|
| 111 |
Gaps:
|
| 112 |
|
| 113 |
+
- Export/download still needs implementation work.
|
| 114 |
+
- Debug-like labels and technical captions still need to be collapsed, renamed or moved.
|
| 115 |
+
- Risk remains open until UI/export implementation and app verification are complete.
|
| 116 |
+
|
| 117 |
+
Recommended workpackages:
|
| 118 |
+
|
| 119 |
+
- `WP_EXPORT_DOWNLOAD_UX_CONTRACT_TESTS`
|
| 120 |
+
- `WP_EXPORT_DOWNLOAD_UX_IMPLEMENTATION`
|
| 121 |
+
- `WP_REVIEW_DEBUG_ELEMENTS_COLLAPSE_PLAN`
|
| 122 |
+
- `WP_REVIEW_DEBUG_ELEMENTS_COLLAPSE_IMPLEMENTATION`
|
| 123 |
+
- `WP_REVIEW_COPY_POLISH_IMPLEMENTATION`
|
| 124 |
|
| 125 |
---
|
| 126 |
|
|
|
|
| 135 |
Users may assume PDF support means complete restored PDF reinsert or OCR, while the approved scope is text-based extraction to restored TXT only.
|
| 136 |
```
|
| 137 |
|
| 138 |
+
PDF limitations must remain clear in export/reinsert copy.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 139 |
|
| 140 |
---
|
| 141 |
|
| 142 |
+
## R8 — Workflow status, audit visibility and evidence clarity
|
| 143 |
|
| 144 |
Status: mitigating
|
| 145 |
Impact: medium
|
|
|
|
| 147 |
Risk:
|
| 148 |
|
| 149 |
```text
|
| 150 |
+
Evidence and audit controls become either too hidden to trust or too technical for normal users.
|
| 151 |
```
|
| 152 |
|
| 153 |
Current mitigations:
|
| 154 |
|
| 155 |
- Coordinator screenshots/evidence are recorded when connector lookup is incomplete.
|
|
|
|
| 156 |
- Diagnostic recall benchmark artifact workflow exists.
|
| 157 |
+
- Audit/report details exist.
|
| 158 |
+
- `MVP_UI_CLEANUP_AND_EXPORT_REDESIGN_PLAN.md` states that technical/audit details must remain available but become secondary.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 159 |
|
| 160 |
Gaps:
|
| 161 |
|
| 162 |
+
- Technical details are not yet consistently grouped under a professional audit/advanced layer.
|
| 163 |
- No generalized automated status artifact exists yet.
|
| 164 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 165 |
---
|
| 166 |
|
| 167 |
## R9 — Dutch legal reference under-detection and role over-masking
|
|
|
|
| 175 |
Dutch legal matter references can be missed or misclassified, while generic legal/care role words can be masked in ways that damage meaning.
|
| 176 |
```
|
| 177 |
|
| 178 |
+
Current mitigations include diagnostic benchmark work, preservation guidance and PERSON-name contract/helper work. Benchmark follow-up is temporarily parked for UI/export work unless a concrete blocker appears.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 179 |
|
| 180 |
---
|
| 181 |
|
|
|
|
| 194 |
Allowed wording:
|
| 195 |
|
| 196 |
```text
|
| 197 |
+
Scrub helpt gevonden gegevens te controleren en exporteren, maar menselijke review blijft noodzakelijk.
|
| 198 |
+
Technische en auditdetails blijven beschikbaar voor controle.
|
| 199 |
```
|