solidprivacy-nl commited on
Commit
c49e63e
·
1 Parent(s): 76b728e

Update risks for MVP UI cleanup planning

Browse files
Files changed (1) hide show
  1. RISK_REGISTER.md +31 -129
RISK_REGISTER.md CHANGED
@@ -19,40 +19,14 @@ Risk:
19
  Sensitive data remains in the scrubbed output and the user wrongly trusts the result.
20
  ```
21
 
22
- Current mitigations:
23
-
24
- - Human review workflow.
25
- - Review guidance and final review summary.
26
- - Dutch legal recall gap tests and pattern-fix verification are present.
27
- - Synthetic gold-label corpus exists and has been expanded.
28
- - Minimal diagnostic recall benchmark runner exists.
29
- - Diagnostic report artifact workflow exists.
30
- - First and cleaned artifact reviews exist.
31
- - Planning-only threshold policy exists.
32
- - PERSON-name coverage review classifies the remaining missed `PERSON` labels.
33
- - PERSON-name false-negative risk has diagnostic test coverage in `tests/test_recall_person_name_coverage_diagnostics.py`.
34
- - PERSON-name false-negative risk has a planning/specification-only recognition improvement plan in `RECALL_PERSON_NAME_RECOGNIZER_PLAN.md`.
35
- - PERSON-name false-negative risk has contract test coverage for future recognizer behavior in `tests/test_recall_person_name_recognizer_contracts.py` and `tests/fixtures/person_name_recognizer_contract_cases.json`.
36
- - PERSON-name false-negative risk is now partially mitigated for contract-backed role/title name cases by `person_name_recognizer_helper.py` and `tests/test_recall_person_name_recognizer_implementation.py`.
37
 
38
- Gaps:
39
 
40
  - No accepted production recall/precision threshold exists.
41
  - No production benchmark gate exists.
42
  - No production safety claim is supported.
43
- - PERSON-name false-negative risk is only partially mitigated for contract-backed role/title helper cases.
44
- - App-registered recognizer behavior and benchmark artifact impact still need review.
45
- - Candidate-only weak contexts are not automatic.
46
- - No candidate scanner change was made.
47
- - No gate or threshold enforcement was added.
48
- - Single-surname risk remains bounded by strong-context rules.
49
- - Remaining non-PERSON gaps include care room/location references, one client-number example and one nested false-positive hit inside a phone-like value.
50
- - Corpus is synthetic and small.
51
-
52
- Recommended workpackages:
53
-
54
- - Next approved package — `WP_RECALL_PERSON_NAME_RECOGNIZER_BENCHMARK_REVIEW`.
55
- - Other approved packages — `WP_CARE_LOCATION_REFERENCE_CANDIDATE_PLAN`, `WP_CLIENT_REFERENCE_COVERAGE_REVIEW`, `WP_RECALL_BENCHMARK_THRESHOLDS_CONTRACT_TESTS`.
56
 
57
  ---
58
 
@@ -67,16 +41,7 @@ Risk:
67
  The Scrub Key is shared, leaked, retained too long, tampered with or mishandled, allowing full re-identification of scrubbed content.
68
  ```
69
 
70
- Current mitigations:
71
-
72
- - UI warnings around Scrub Key reinsert.
73
- - Local-only/no-AI/no-cloud positioning for reinsert.
74
- - Scrub Key warning/acknowledgement UI is closed out after Actions/HF/app verification.
75
-
76
- Gaps:
77
-
78
- - UI acknowledgements are safety prompts only; they are not encryption, protected storage, automatic deletion or a managed key vault.
79
- - No approved key recovery model.
80
 
81
  ---
82
 
@@ -91,16 +56,7 @@ Risk:
91
  An AI system rewrites, translates, merges or deletes placeholders, causing deterministic reinsert to fail or restore incompletely.
92
  ```
93
 
94
- Current mitigations:
95
-
96
- - Unknown placeholders and not-found placeholders are reported in reinsert audit flows.
97
- - Placeholder robustness review, future robust format proposal, validation helper, audit helper and synthetic AI-output corruption tests exist.
98
-
99
- Gaps:
100
-
101
- - Robust placeholder generation has not been implemented in product flow.
102
- - No migration or backward-compatibility implementation exists yet.
103
- - No Scrub Key schema/version support for robust placeholder metadata exists yet.
104
 
105
  ---
106
 
@@ -115,17 +71,7 @@ Risk:
115
  DOCX metadata, comments, tracked changes, headers, footers or hidden content contain sensitive data that is not scrubbed or cleaned.
116
  ```
117
 
118
- Current mitigations:
119
-
120
- - DOCX hidden-content risk review, visibility helpers and report-only hygiene audit are present.
121
- - Clean DOCX export policy exists.
122
- - DOCX hygiene audit UI is report-only and does not claim clean DOCX output.
123
-
124
- Gaps:
125
-
126
- - Word comments / kantlijncommentaren are still not scrubbed or removed by the current DOCX scrub/reinsert flow.
127
- - No clean DOCX export implementation exists.
128
- - No approved export-blocking implementation exists for high-risk hidden content.
129
 
130
  ---
131
 
@@ -140,29 +86,19 @@ Risk:
140
  The final product promise is local-first, but the current fast validation surface is the Hugging Face cloud demo.
141
  ```
142
 
143
- Current mitigations:
144
-
145
- - Roadmap includes local desktop/offline direction.
146
- - UI messaging emphasizes local/no-AI/no-cloud where applicable.
147
- - Local runtime/packaging decision line is completed through packaging deferral.
148
-
149
- Gaps:
150
-
151
- - No full offline demonstration.
152
- - No production security certification is supported.
153
- - No signed or managed enterprise deployment proof exists.
154
 
155
  ---
156
 
157
- ## R6 — Review UX and replace-flow ceiling
158
 
159
  Status: mitigating
160
- Impact: medium
161
 
162
  Risk:
163
 
164
  ```text
165
- The review interface may not support the document-centric review experience needed for high-trust legal/care workflows.
166
  ```
167
 
168
  Current mitigations:
@@ -170,12 +106,21 @@ Current mitigations:
170
  - Review table remains source of truth and fallback.
171
  - Side-by-side review surface, synced scrolling and collapsible review table are live and verified.
172
  - Serial review remains available as a non-destructive review aid.
 
173
 
174
  Gaps:
175
 
176
- - The review table remains the source of truth and fallback.
177
- - Percentage-based synchronized scrolling can be imperfect when source and processed text differ structurally.
178
- - No professional document editor exists.
 
 
 
 
 
 
 
 
179
 
180
  ---
181
 
@@ -190,16 +135,11 @@ Risk:
190
  Users may assume PDF support means complete restored PDF reinsert or OCR, while the approved scope is text-based extraction to restored TXT only.
191
  ```
192
 
193
- Boundaries:
194
-
195
- - No restored PDF output.
196
- - No OCR.
197
- - No PDF-to-DOCX reconstruction.
198
- - No layout preservation promises.
199
 
200
  ---
201
 
202
- ## R8 — Workflow status and benchmark evidence visibility
203
 
204
  Status: mitigating
205
  Impact: medium
@@ -207,32 +147,21 @@ Impact: medium
207
  Risk:
208
 
209
  ```text
210
- Evidence becomes ambiguous because connector status can be incomplete and diagnostic outputs are hard to inspect.
211
  ```
212
 
213
  Current mitigations:
214
 
215
  - Coordinator screenshots/evidence are recorded when connector lookup is incomplete.
216
- - Workpackages record Actions/sync status.
217
  - Diagnostic recall benchmark artifact workflow exists.
218
- - First and cleaned artifact reviews exist.
219
- - Planning-only threshold policy exists.
220
- - PERSON-name coverage review exists.
221
- - PERSON-name diagnostic tests exist.
222
- - PERSON-name recognizer planning exists.
223
- - PERSON-name contract tests exist.
224
- - PERSON-name helper implementation tests exist.
225
 
226
  Gaps:
227
 
228
- - Connector workflow-run lookup can still be incomplete for some push-triggered runs.
229
  - No generalized automated status artifact exists yet.
230
 
231
- Recommended workpackages:
232
-
233
- - Next approved benchmark package — `WP_RECALL_PERSON_NAME_RECOGNIZER_BENCHMARK_REVIEW`.
234
- - Later status package — automated status artifact/check if connector limitations continue to slow closeouts.
235
-
236
  ---
237
 
238
  ## R9 — Dutch legal reference under-detection and role over-masking
@@ -246,34 +175,7 @@ Risk:
246
  Dutch legal matter references can be missed or misclassified, while generic legal/care role words can be masked in ways that damage meaning.
247
  ```
248
 
249
- Current mitigations:
250
-
251
- - Human review table remains source of truth and fallback.
252
- - Recall benchmark spec defines reference classes, context terms to preserve and over-masking traps.
253
- - Diagnostic runner/report artifacts make benchmark evidence visible.
254
- - Cleaned artifact has `preserve_term_hit_count = 0`.
255
- - Planning-only threshold policy exists.
256
- - PERSON-name coverage review classifies role-linked missed names, including care-role and legal-role examples.
257
- - PERSON-name diagnostic tests preserve the gap inventory and non-claim boundaries without requiring current recognizers to pass all examples.
258
- - PERSON-name recognizer planning records a test-first route and value-only role/context boundary before implementation.
259
- - PERSON-name recognizer contract tests specify future value-only, preserve-term, single-surname and negative-case behavior.
260
- - PERSON-name helper implementation now enforces value-only role/title matching for contract-backed helper cases.
261
-
262
- Gaps:
263
-
264
- - No accepted production threshold exists.
265
- - No benchmark gate exists.
266
- - Corpus is synthetic and not exhaustive.
267
- - PERSON-name false-negative risk is only partially mitigated for contract-backed role/title helper cases.
268
- - Benchmark artifact impact still needs review.
269
- - Single-surname examples remain high ambiguity and must stay limited to strong context.
270
- - Product claim remains blocked.
271
-
272
- Recommended workpackages:
273
-
274
- - `WP_RECALL_PERSON_NAME_RECOGNIZER_BENCHMARK_REVIEW`.
275
- - `WP_CARE_LOCATION_REFERENCE_CANDIDATE_PLAN`.
276
- - `WP_CLIENT_REFERENCE_COVERAGE_REVIEW`.
277
 
278
  ---
279
 
@@ -292,6 +194,6 @@ De benchmark bewijst production readiness.
292
  Allowed wording:
293
 
294
  ```text
295
- The PERSON-name helper implements contract-backed value-only role/title matching for synthetic examples.
296
- Human review remains necessary.
297
  ```
 
19
  Sensitive data remains in the scrubbed output and the user wrongly trusts the result.
20
  ```
21
 
22
+ Current mitigations include human review, review guidance, diagnostic recall benchmark artifacts, PERSON-name diagnostic/contract/helper work and planning-only threshold policy.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
23
 
24
+ Remaining gaps:
25
 
26
  - No accepted production recall/precision threshold exists.
27
  - No production benchmark gate exists.
28
  - No production safety claim is supported.
29
+ - Human review remains necessary.
 
 
 
 
 
 
 
 
 
 
 
 
30
 
31
  ---
32
 
 
41
  The Scrub Key is shared, leaked, retained too long, tampered with or mishandled, allowing full re-identification of scrubbed content.
42
  ```
43
 
44
+ Current mitigations include warnings and acknowledgements. UI cleanup must keep Scrub Key clearly separate from ordinary document exports.
 
 
 
 
 
 
 
 
 
45
 
46
  ---
47
 
 
56
  An AI system rewrites, translates, merges or deletes placeholders, causing deterministic reinsert to fail or restore incompletely.
57
  ```
58
 
59
+ Current mitigations include placeholder robustness helper/test work and reinsert audit reporting.
 
 
 
 
 
 
 
 
 
60
 
61
  ---
62
 
 
71
  DOCX metadata, comments, tracked changes, headers, footers or hidden content contain sensitive data that is not scrubbed or cleaned.
72
  ```
73
 
74
+ DOCX hygiene audit remains report-only. MVP UI cleanup may make it visually calmer, but audit details must remain available and must not imply a clean-DOCX guarantee.
 
 
 
 
 
 
 
 
 
 
75
 
76
  ---
77
 
 
86
  The final product promise is local-first, but the current fast validation surface is the Hugging Face cloud demo.
87
  ```
88
 
89
+ Local/offline installer work remains later. The current focus is making the web prototype workflow credible first.
 
 
 
 
 
 
 
 
 
 
90
 
91
  ---
92
 
93
+ ## R6 — Review UX and interface clarity risk
94
 
95
  Status: mitigating
96
+ Impact: high
97
 
98
  Risk:
99
 
100
  ```text
101
+ The interface still feels like a technical prototype, which can reduce confidence and increase review mistakes.
102
  ```
103
 
104
  Current mitigations:
 
106
  - Review table remains source of truth and fallback.
107
  - Side-by-side review surface, synced scrolling and collapsible review table are live and verified.
108
  - Serial review remains available as a non-destructive review aid.
109
+ - `MVP_UI_CLEANUP_AND_EXPORT_REDESIGN_PLAN.md` now defines a route to move debug/audit details out of the primary flow and improve export/download hierarchy.
110
 
111
  Gaps:
112
 
113
+ - Export/download still needs implementation work.
114
+ - Debug-like labels and technical captions still need to be collapsed, renamed or moved.
115
+ - Risk remains open until UI/export implementation and app verification are complete.
116
+
117
+ Recommended workpackages:
118
+
119
+ - `WP_EXPORT_DOWNLOAD_UX_CONTRACT_TESTS`
120
+ - `WP_EXPORT_DOWNLOAD_UX_IMPLEMENTATION`
121
+ - `WP_REVIEW_DEBUG_ELEMENTS_COLLAPSE_PLAN`
122
+ - `WP_REVIEW_DEBUG_ELEMENTS_COLLAPSE_IMPLEMENTATION`
123
+ - `WP_REVIEW_COPY_POLISH_IMPLEMENTATION`
124
 
125
  ---
126
 
 
135
  Users may assume PDF support means complete restored PDF reinsert or OCR, while the approved scope is text-based extraction to restored TXT only.
136
  ```
137
 
138
+ PDF limitations must remain clear in export/reinsert copy.
 
 
 
 
 
139
 
140
  ---
141
 
142
+ ## R8 — Workflow status, audit visibility and evidence clarity
143
 
144
  Status: mitigating
145
  Impact: medium
 
147
  Risk:
148
 
149
  ```text
150
+ Evidence and audit controls become either too hidden to trust or too technical for normal users.
151
  ```
152
 
153
  Current mitigations:
154
 
155
  - Coordinator screenshots/evidence are recorded when connector lookup is incomplete.
 
156
  - Diagnostic recall benchmark artifact workflow exists.
157
+ - Audit/report details exist.
158
+ - `MVP_UI_CLEANUP_AND_EXPORT_REDESIGN_PLAN.md` states that technical/audit details must remain available but become secondary.
 
 
 
 
 
159
 
160
  Gaps:
161
 
162
+ - Technical details are not yet consistently grouped under a professional audit/advanced layer.
163
  - No generalized automated status artifact exists yet.
164
 
 
 
 
 
 
165
  ---
166
 
167
  ## R9 — Dutch legal reference under-detection and role over-masking
 
175
  Dutch legal matter references can be missed or misclassified, while generic legal/care role words can be masked in ways that damage meaning.
176
  ```
177
 
178
+ Current mitigations include diagnostic benchmark work, preservation guidance and PERSON-name contract/helper work. Benchmark follow-up is temporarily parked for UI/export work unless a concrete blocker appears.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
179
 
180
  ---
181
 
 
194
  Allowed wording:
195
 
196
  ```text
197
+ Scrub helpt gevonden gegevens te controleren en exporteren, maar menselijke review blijft noodzakelijk.
198
+ Technische en auditdetails blijven beschikbaar voor controle.
199
  ```