solidprivacy commited on
Commit
255cd61
·
unverified ·
2 Parent(s): 54c73e08028643

Merge pull request #114 from solidprivacy-nl/wp/repository-convergence-bootstrap

Browse files

SCRUB-WP_REPOSITORY_CONVERGENCE_BOOTSTRAP — reset current truth before further development

AGENTS.md CHANGED
@@ -10,7 +10,7 @@ Work only in:
10
  solidprivacy-nl/scrub
11
  ```
12
 
13
- Do not modify unrelated repositories. If the active repository is not `solidprivacy-nl/scrub`, stop and report the mismatch.
14
 
15
  ## Required start sequence
16
 
@@ -21,196 +21,160 @@ Before starting any task, read in order:
21
  3. `WORKPACKAGES.md`
22
  4. `CHANGELOG.md`
23
 
24
- Also read when relevant:
25
 
26
- - `RISK_REGISTER.md`
27
- - `DECISION_LOG.md`
28
- - `STATUS_MONITORING_RUNBOOK.md`
29
- - `RELEASE_NOTES.md`
30
- - feature specs such as `SCRUB_KEY_SPEC.md` and `PDF_TEXT_REINSERT_UI_PLAN.md`
31
 
32
- ## Workpackage claim check
33
 
34
- Before starting implementation or documentation changes for a workpackage, check `workpackage_claims/`.
35
 
36
- If a claim file for the same workpackage already exists:
37
 
38
- - status `in_progress`: stop and report that the package is already claimed;
39
- - status `completed`: stop and report that the package is already done;
40
- - status `blocked`: stop and read the blocking reason;
41
- - status `abandoned`: continue only with coordinator approval.
42
 
43
- If no claim exists, create a new claim file before editing code, tests, UI, export, schema or shared documentation files.
 
 
 
 
 
 
 
 
 
44
 
45
- Use `GitHub.create_file` for the claim. Do not overwrite an existing claim. This is the lightweight lock that prevents two workers from silently starting the same package.
46
 
47
- When the workpackage is done, update the same claim file to `completed` and record the final commit or PR, handover path, tests/checks, validation status, remaining risks and next recommended step.
48
-
49
- ## Product direction
50
-
51
- The product direction is:
52
 
53
  ```text
54
- Scrub → Review → Scrub Key → AI → Reinsert → Export → Audit
 
 
 
 
55
  ```
56
 
57
- The roadmap is risk-driven. Highest risks:
58
-
59
- 1. false negatives / missed sensitive data;
60
- 2. Scrub Key leakage or accidental sharing;
61
- 3. hidden document content and metadata leakage;
62
- 4. cloud-demo trust gap versus local-first promise;
63
- 5. placeholder corruption during AI roundtrip;
64
- 6. review UX limitations;
65
- 7. PDF-scope misunderstandings.
66
-
67
- ## Safety rules
68
-
69
- Do not weaken privacy or review controls.
70
-
71
- Do not silently change export semantics.
72
-
73
- Do not introduce cloud document processing unless explicitly approved.
74
 
75
- Do not store secrets, tokens or real personal data.
76
 
77
- Use synthetic data only.
78
 
79
- Preserve legal/professional context: replace sensitive values, not legal meaning.
80
 
81
- Treat the Scrub Key as sensitive re-identification data.
82
 
83
- ## Workpackage discipline
84
 
85
- Work only on the assigned workpackage.
86
 
87
- Do not broaden scope.
88
-
89
- Do not start another workpackage unless explicitly instructed.
90
-
91
- If the task is specification-only, do not change code, tests, dependencies or UI.
 
92
 
93
- If the task is closeout-only, do not change code, tests, dependencies or UI.
94
 
95
- If the task is implementation, prefer helper modules and tests before UI changes.
96
 
97
- ## Parallelization rules
98
 
99
- Safe to work on in parallel:
 
 
 
100
 
101
- - separate specification documents;
102
- - helper modules in separate files;
103
- - tests that do not touch the same UI flow;
104
- - risk reviews;
105
- - architecture plans;
106
- - non-UI documentation.
107
 
108
- Do not work in parallel without explicit coordination on:
109
 
110
- - `presidio_streamlit.py`
111
- - `fix_streamlit_nested_expanders.py`
112
- - `fix_streamlit_pdf_text_reinsert.py`
113
- - review table UI flow
114
- - export/download UI flow
115
- - shared workflow state
116
- - Docker/runtime startup patch order
117
 
118
- ## Testing and validation
119
 
120
- Run the tests named in the workpackage.
121
 
122
- If no tests are required because the task is documentation-only or specification-only, state that clearly.
123
 
124
- For implementation tasks, run targeted tests first, then related regression tests when practical.
125
 
126
- Do not claim success if tests were not run. State the exact validation status.
127
 
128
- ## GitHub Actions and sync
129
 
130
- Where possible, check GitHub Actions and GitHub-to-Hugging-Face sync status for the relevant commit.
131
 
132
- If connector or environment access prevents status lookup, say so clearly.
133
 
134
- Do not ask for app verification until Actions and sync are green.
 
 
 
 
 
135
 
136
- UI changes require app verification by the coordinator/user.
137
 
138
- ## Documentation updates
139
 
140
- Update these files according to the workpackage:
141
 
142
- - `WORKPACKAGES.md` for status and next queue;
143
- - `CHANGELOG.md` for internal implementation history;
144
- - `RELEASE_NOTES.md` for user-visible product changes;
145
- - `RISK_REGISTER.md` for risk state changes;
146
- - `DECISION_LOG.md` for accepted strategic/architecture decisions.
 
147
 
148
- Do not update `ROADMAP.md` unless strategy, phase order or product direction changes.
149
 
150
- ## Handover process
151
 
152
- Every task must create a handover file in:
 
 
 
 
 
153
 
154
- ```text
155
- handover/workpackages/
156
- ```
157
 
158
- Filename format:
159
 
160
  ```text
161
  handover/workpackages/YYYYMMDD_HHMM_<workpackage_slug>.md
162
  ```
163
 
164
- The handover file must include:
165
-
166
- - repository worked in;
167
- - workpackage title;
168
- - status;
169
- - files added;
170
- - files changed;
171
- - tests added/updated;
172
- - validation status;
173
- - GitHub Actions status;
174
- - Hugging Face sync status;
175
- - app verification status;
176
- - remaining risks;
177
- - next recommended step.
178
-
179
- For Codex or other parallel worker tasks, do not paste the full handover into the coordinator chat when the handover has been committed to the repository.
180
-
181
- Instead, the final worker response should provide only:
182
-
183
- - workpackage title;
184
- - status;
185
- - commit SHA or PR link;
186
- - handover file path;
187
- - short summary of files added/changed;
188
- - tests/checks run;
189
- - remaining risks;
190
  - next recommended step.
191
 
192
- The coordinator can then read the committed handover from GitHub.
193
 
194
- Only paste the full handover into chat if:
 
 
 
 
 
 
 
 
195
 
196
- - the handover could not be committed;
197
- - GitHub access failed;
198
- - there is a conflict or permission issue;
199
- - the coordinator explicitly asks for the full text.
200
-
201
- ## Conflict rule
202
-
203
- Before editing shared documentation files, fetch the latest version.
204
-
205
- If there is a SHA conflict, update conflict or missing permission, stop and report instead of overwriting another worker’s changes.
206
-
207
- ## Final response requirements
208
-
209
- In the final response, summarize:
210
-
211
- - what was changed;
212
- - which files were touched;
213
- - what tests/checks were run;
214
- - what was intentionally not changed;
215
- - remaining risks;
216
- - next recommended step.
 
10
  solidprivacy-nl/scrub
11
  ```
12
 
13
+ Use `solidprivacy-nl/solidprivacy` only when an explicitly shared SolidPrivacy workflow/component is required by the current workpackage. Do not modify unrelated repositories.
14
 
15
  ## Required start sequence
16
 
 
21
  3. `WORKPACKAGES.md`
22
  4. `CHANGELOG.md`
23
 
24
+ For consequential work also read:
25
 
26
+ 1. `control/PROJECT_GOVERNANCE_BOOTSTRAP.md`
27
+ 2. `control/SCRUB_RELEASE_ASSURANCE_CONTRACT_V1.md`
 
 
 
28
 
29
+ Read `RISK_REGISTER.md`, `DECISION_LOG.md` and relevant product contracts for the affected surface.
30
 
31
+ GitHub is the source of truth. If canonical docs conflict with current source/runtime evidence, surface and reconcile the conflict rather than silently selecting one.
32
 
33
+ ## Engineering doctrine
34
 
35
+ Follow the SolidPrivacy Execution & Engineering Constitution:
 
 
 
36
 
37
+ - business outcome first;
38
+ - smallest complete solution;
39
+ - solid but simple;
40
+ - no overengineering;
41
+ - first-principles reasoning;
42
+ - proven/native solutions before custom machinery;
43
+ - one source of truth;
44
+ - remove obsolete/conflicting active paths;
45
+ - verify behavior and security;
46
+ - Done requires implementation, verification, cleanup and documentation alignment.
47
 
48
+ ## Current strategic direction
49
 
50
+ The active line is:
 
 
 
 
51
 
52
  ```text
53
+ Repository Convergence
54
+ → Scrub Private Application
55
+ → Private Service
56
+ → External Product & Service Assurance
57
+ → Pilot
58
  ```
59
 
60
+ Normal new feature development is paused until `SCRUB_REPOSITORY_CONVERGED`.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
61
 
62
+ After convergence, `main` becomes the active Scrub Private line.
63
 
64
+ Hugging Face is a synthetic/approved-test application-validation environment, not the final confidential-production trust environment.
65
 
66
+ Local/offline functionality is preserved/deferred. Do not restart old installer or historical Premium work merely because old files/issues mention it.
67
 
68
+ Do not build a new Evidence Framework; reuse and reconcile the existing validation systems.
69
 
70
+ ## Worker claim rule
71
 
72
+ Before implementation, claim one specific workpackage in `workpackage_claims/` with:
73
 
74
+ - workpackage title;
75
+ - role;
76
+ - issue/branch where applicable;
77
+ - exact starting base;
78
+ - scope and exclusions;
79
+ - required validation.
80
 
81
+ Do not claim multiple overlapping shared-surface packages in one worker.
82
 
83
+ ## Implementation and assurance separation
84
 
85
+ Consequential work uses two separated roles:
86
 
87
+ ```text
88
+ implementation_operations
89
+ governance_release_assurance
90
+ ```
91
 
92
+ Implementation may create a candidate but may not certify it.
 
 
 
 
 
93
 
94
+ Assurance must use a fresh independent worker/session, reconstruct exact source/evidence, and record `PASS`, `FAIL` or `INDETERMINATE`. Before its initial verdict it must not rely on implementation handovers, self-assessments or conclusions. Assurance may not silently repair what it reviews.
95
 
96
+ A changed/repaired exact head requires a fresh assurance pass.
 
 
 
 
 
 
97
 
98
+ ## Workpackage sizing
99
 
100
+ Use small coherent root-cause packages.
101
 
102
+ Do not create one giant convergence PR. Do not fragment trivial administrative edits into meaningless ritual packages.
103
 
104
+ **Do not refactor for aesthetics.**
105
 
106
+ During Repository Convergence, a cleanup/refactor requires a concrete current reason such as duplicate/contradictory/dead behavior, privacy/security risk, obsolete compatibility, meaningful maintenance burden, runtime instability or evidence-authority ambiguity.
107
 
108
+ “A cleaner architecture” alone is not sufficient.
109
 
110
+ ## Shared-surface sequencing
111
 
112
+ Avoid uncoordinated parallel edits to:
113
 
114
+ - `presidio_streamlit.py`;
115
+ - Streamlit/Docker startup behavior;
116
+ - review table/direct-correction flow;
117
+ - processing-generation state;
118
+ - export/download flow;
119
+ - Scrub Key/reinsert paths.
120
 
121
+ Keep shared runtime/UI integration sequential unless explicit coordination proves independence.
122
 
123
+ ## Testing and deployment
124
 
125
+ Where tools allow, workers self-check:
126
 
127
+ 1. relevant focused tests;
128
+ 2. full GitHub Actions on the exact candidate when required;
129
+ 3. failed-job logs;
130
+ 4. exact-main Actions after authorized merge;
131
+ 5. GitHub→Hugging Face sync when applicable;
132
+ 6. deployed/live app verification when visible/runtime behavior changed.
133
 
134
+ Do not treat code inspection alone as proof.
135
 
136
+ ## Documentation ownership
137
 
138
+ - `ROADMAP.md` = strategic stages, not implementation history.
139
+ - `WORKPACKAGES.md` = one current executable queue.
140
+ - `CHANGELOG.md` = current implementation history; exact pre-convergence history is preserved in `history/CHANGELOG_PRE_CONVERGENCE_20260904.md` and Git at the baseline SHA.
141
+ - `DECISION_LOG.md` = accepted current binding decisions.
142
+ - `RISK_REGISTER.md` = current risks.
143
+ - temporary audit/debt ledgers = execution evidence only; they must not become competing permanent status authorities.
144
 
145
+ ## Handover
 
 
146
 
147
+ Every worker writes:
148
 
149
  ```text
150
  handover/workpackages/YYYYMMDD_HHMM_<workpackage_slug>.md
151
  ```
152
 
153
+ Include:
154
+
155
+ - repository;
156
+ - workpackage;
157
+ - role/status;
158
+ - exact candidate/baseline identity where relevant;
159
+ - files added/changed;
160
+ - tests;
161
+ - validation;
162
+ - GitHub Actions;
163
+ - Hugging Face sync;
164
+ - app verification;
165
+ - remaining risks/blockers;
 
 
 
 
 
 
 
 
 
 
 
 
 
166
  - next recommended step.
167
 
168
+ ## Safety
169
 
170
+ - Do not weaken human review/privacy controls.
171
+ - Do not silently change export or Scrub Key semantics.
172
+ - Treat false negatives as product-critical.
173
+ - Treat Scrub Keys as sensitive re-identification material.
174
+ - Use synthetic data only.
175
+ - Do not store secrets/tokens/real personal data.
176
+ - Preserve legal and clinical meaning.
177
+ - Do not introduce external/cloud document processing unless explicitly approved for the relevant product line.
178
+ - Do not claim perfect anonymisation/recall or production readiness from synthetic tests.
179
 
180
+ For Scrub Private, the target is no intentional persistent customer document content, no content-bearing ordinary logs, no document-content backup and no third-party document-processing egress. Minimal non-document control-plane metadata may persist only when justified by a real service requirement.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
CHANGELOG.md CHANGED
The diff for this file is too large to render. See raw diff
 
DECISION_LOG.md CHANGED
@@ -1,739 +1,264 @@
1
- ## 2026-08-08 — D043 — Freeze Premium Standard as a single-page staged document workspace
2
 
3
- Status: accepted product/UX architecture direction; implementation candidate subject to normal independent assurance.
4
 
5
- Decision:
6
 
7
  ```text
8
- For the Standard anonymization core flow, use one persistent document workspace containing Toevoegen → Controleren → Downloaden. Keep all three stage headers visible, allow exactly one expanded/dominant stage at a time, collapse completed stages to compact summaries, keep future stages passive, auto-advance after successful completion, and allow explicit return to an earlier stage with fail-closed downstream invalidation.
9
  ```
10
 
11
- Explicitly rejected for Standard:
12
-
13
- ```text
14
- Toevoegen page → Controleren page → Downloaden page
15
- ```
16
-
17
- The three core stages are states of the same document, not three separate routed screens. Also reject a conventional long Streamlit form rebuilt as a tree of independent or nested expanders.
18
-
19
- First-principles reason:
20
- - the scarce resource is user attention during a privacy-sensitive review task;
21
- - Scrub is iterative: review can legitimately send the user back to input/profile correction;
22
- - the interface must preserve confidence about source identity, completed work and whether downstream state is still current;
23
- - page routing maximizes isolation but removes spatial workflow context and adds state-restoration/routing complexity;
24
- - one staged workspace preserves the cognitive benefit of one task at a time while representing document state directly;
25
- - the model maps naturally to the merged source/processed/review/export generation lineage in `premium_core_flow_state.py`.
26
-
27
- Binding interaction consequences:
28
- - one dominant primary action per active stage;
29
- - no permanent configuration sidebar in Standard;
30
- - completed stage summaries contain orientation/trust information only, not mini-forms;
31
- - future stages show status rather than disabled control forests;
32
- - successful processing opens Controleren automatically;
33
- - explicit review completion opens Downloaden automatically;
34
- - earlier processing-affecting edits invalidate downstream review/export state before those results can be presented as current;
35
- - no nested core-flow accordion hierarchy in Standard;
36
- - stage surfaces should read as application panels even when Streamlit primitives are used internally.
37
-
38
- Current workpackage consequence:
39
- - insert `SCRUB-WP_PREMIUM_STAGED_WORKSPACE_DECISION_FREEZE` before further production integration in `SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION`;
40
- - retain PR #85's compatible pure helper/test work;
41
- - require PR #85 to incorporate the staged-workspace semantics before touching/merging production `presidio_streamlit.py` integration;
42
- - continue sequentially with Input, Review, Export, Expert parity and live app closeout packages.
43
-
44
- Safety boundary:
45
- - this decision changes presentation architecture, not anonymization semantics;
46
- - recognizers, thresholds, review authority, direct masking, export bytes/names/MIME, Scrub Key, reinsert, audit and human-review requirements remain unchanged;
47
- - Standard is lower cognitive load, not lower safety.
48
-
49
- Evidence:
50
- - coordinator/user approval on 2026-08-08 Europe/Amsterdam after explicit first-principles comparison;
51
- - `PREMIUM_STAGED_WORKSPACE_DECISION.md`;
52
- - `PREMIUM_CORE_FLOW_UI_REALIGNMENT_PLAN.md`;
53
- - merged Premium UI contract and state model.
54
 
55
  ---
56
 
57
- ## 2026-08-06 — D042 — Separate implementation from blind independent release assurance
58
 
59
- Status: accepted project-governance and release-control decision
60
 
61
  Decision:
62
 
63
  ```text
64
- Adopt CROSS_PROJECT_TWO_ROLE_GOVERNANCE_V1 for consequential Scrub work. Use one coordinator and two separated roles: implementation_operations and governance_release_assurance. Implementation prepares an identifiable candidate but cannot certify it. Governance reconstructs it independently and cannot silently repair it.
65
- ```
66
-
67
- Scrub-specific strengthening:
68
 
69
- ```text
70
- Before its initial PASS / FAIL / INDETERMINATE, governance may inspect the requested outcome, authoritative control files, candidate source/diff, acceptance criteria and raw machine/deployment evidence, but may not read the implementation handover, implementation self-assessment or implementation conclusions.
 
 
 
 
 
 
 
 
 
71
  ```
72
 
73
- Reason:
74
- - Scrub processes privacy-sensitive professional documents and creates re-identification-sensitive Scrub Keys;
75
- - builder self-certification creates avoidable confirmation bias and weakens release confidence;
76
- - the Weekly ETF donor architecture already establishes a mature coordinator/implementation/assurance split;
77
- - the user's explicit requirement is stronger than ordinary review and requires conclusion isolation before the initial assurance decision.
78
-
79
- Operating consequences:
80
- - consequential implementation and verification are separate workpackages and separate workers/sessions;
81
- - the user continues to give one instruction and receives one consolidated status;
82
- - a failed or indeterminate candidate returns to implementation and requires a fresh assurance pass;
83
- - implementation statuses are limited to `IMPLEMENTATION_IN_PROGRESS`, `IMPLEMENTATION_BLOCKED` and `RELEASE_CANDIDATE_READY`;
84
- - assurance decisions are `PASS`, `FAIL` or `INDETERMINATE`;
85
- - merge/deployment execution and independently confirmed outcome remain distinct statuses.
86
-
87
- Initial maturity:
88
- - `LEVEL_1_CHECKLIST`;
89
- - target `LEVEL_2_MACHINE_EVIDENCE` through a later structured assurance-record package;
90
- - no claim that documentation alone provides hard CI or post-action enforcement.
91
 
92
- Authority:
93
- - canonical standard: `market-predictions/control-plane/control/CROSS_PROJECT_TWO_ROLE_GOVERNANCE_STANDARD_V1.md`;
94
- - project bootstrap: `control/PROJECT_GOVERNANCE_BOOTSTRAP.md`;
95
- - project contract: `control/SCRUB_RELEASE_ASSURANCE_CONTRACT_V1.md`.
 
96
 
97
- ## 2026-08-05 — D041 — Move from review-local decluttering to an application-wide premium core-flow shell
98
 
99
- Status: accepted product, UX and implementation-sequence decision
 
 
 
 
 
 
 
 
 
 
100
 
101
- Decision:
102
 
103
  ```text
104
- Adopt an application-wide Standard/Expert presentation model and a one-active-stage document workspace. Separate Anonymize and Reinsert as top-level workflows. In Standard, present Toevoegen → Controleren → Downloaden with one primary action per stage and progressive disclosure of settings, alternative formats, Scrub Key and audit evidence.
 
 
 
 
105
  ```
106
 
107
- Reason:
108
-
109
- - live app evidence confirms that earlier Basic/Expert and decluttering work improved individual sections but left the whole product as a long form;
110
- - premium and enterprise credibility require a coherent task hierarchy, not merely fewer open expanders;
111
- - input alternatives, profile settings, review machinery, normal downloads, restore material and audit evidence serve different user goals and should not have equal visual weight;
112
- - one active stage at a time reduces cognitive load and makes the current task explicit.
113
-
114
- Presentation boundary:
115
-
116
- - recommended global labels are `Standaard` and `Expert`, subject to contract freeze;
117
- - Standard has no permanent configuration sidebar;
118
- - Expert preserves full inspection, tuning, audit and troubleshooting;
119
- - switching presentation changes visibility and grouping only and must preserve input, replacement decisions and session state.
120
-
121
- Safety boundary:
122
-
123
- - human review remains mandatory;
124
- - the review table remains source of truth and fallback;
125
- - no silent profile or recognizer changes;
126
- - no export-byte, filename, MIME, Scrub Key, reinsert or audit semantic changes;
127
- - no cloud document processing, telemetry or browser persistence;
128
- - an explicit processing action requires a pure stale-state and transition model before UI integration.
129
-
130
- Approved sequence:
131
-
132
- 1. `SCRUB-WP_PROCESSED_TEXT_SELECTION_CROSS_FLOW_REGRESSION`
133
- 2. `SCRUB-WP_PREMIUM_CORE_FLOW_UI_CONTRACT`
134
- 3. `SCRUB-WP_PREMIUM_CORE_FLOW_STATE_MODEL`
135
- 4. `SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION`
136
- 5. `SCRUB-WP_PREMIUM_INPUT_STAGE_SIMPLIFICATION`
137
- 6. `SCRUB-WP_PREMIUM_REVIEW_STAGE_SIMPLIFICATION`
138
- 7. `SCRUB-WP_PREMIUM_EXPORT_STAGE_SIMPLIFICATION`
139
- 8. `SCRUB-WP_PREMIUM_EXPERT_PARITY_REGRESSION`
140
- 9. `SCRUB-WP_PREMIUM_CORE_FLOW_APP_VERIFY_CLOSEOUT`
141
-
142
- Evidence:
143
-
144
- - coordinator/user feedback at 2026-08-05 10:49 Europe/Amsterdam;
145
- - `BASIC_EXPERT_REVIEW_MODE_PLAN.md`;
146
- - `MVP_UI_CLEANUP_AND_EXPORT_REDESIGN_PLAN.md`;
147
- - `PREMIUM_CORE_FLOW_UI_REALIGNMENT_PLAN.md`.
148
-
149
- ## 2026-08-04 — D040 — Use a two-stage server-authoritative protocol for direct masking from processed text
150
-
151
- Status: accepted product, UX and implementation-sequence decision
152
-
153
- Decision:
154
-
155
- ```text
156
- Allow a user to select an unmasked value in Verwerkte tekst, invoke a right-click or visible/keyboard masking action, inspect the server-validated exact-occurrence impact, choose a broad type and create one normal manual replacement row. Version one masks all safe exact occurrences only.
157
- ```
158
-
159
- Protocol:
160
-
161
- ```text
162
- inspect_selection
163
- → server validates UTF-16 offsets, text, document scope, processed hash, collisions and occurrence count
164
- → ready / confirmation_required / blocked inspection result
165
- → commit_manual_mask with a current single-use inspection ID
166
- → server revalidates and creates the bound manual row
167
- ```
168
-
169
- Safety boundaries:
170
- - one to five safe exact occurrences are ready;
171
- - six to twenty require explicit confirmation;
172
- - more than twenty are blocked from the quick path;
173
- - embedded substrings, nested included replacement terms, marked-range intersections, duplicates, stale views and replays fail closed;
174
- - the review table remains source of truth and `Gemiste waarde toevoegen` remains fallback;
175
- - browser code never creates placeholders, mutates the table, writes a Scrub Key or builds an export;
176
- - no external assets, telemetry, browser persistence or cloud processing;
177
- - no occurrence-specific replacement, rich editor or Streamlit upgrade in this line.
178
-
179
- Reason:
180
- - direct correction where a false negative is noticed reduces navigation friction and copying mistakes;
181
- - a server-derived impact step is required because browser-supplied counts and types are untrusted;
182
- - all-exact behavior matches current replacement/export/Scrub Key/reinsert semantics;
183
- - occurrence-specific behavior would require a separate span-aware architecture.
184
-
185
- Approved sequence:
186
- 1. `SCRUB-WP_PROCESSED_TEXT_SELECTION_MASKING_CONTRACT`
187
- 2. `SCRUB-WP_PROCESSED_TEXT_SELECTION_MASKING_ACTION_MODEL`
188
- 3. `SCRUB-WP_PROCESSED_TEXT_SELECTION_COMPONENT_SPIKE`
189
- 4. `SCRUB-WP_PROCESSED_TEXT_SELECTION_TABLE_INTEGRATION`
190
- 5. `SCRUB-WP_PROCESSED_TEXT_SELECTION_CROSS_FLOW_REGRESSION`
191
- 6. `SCRUB-WP_PROCESSED_TEXT_SELECTION_APP_VERIFY`
192
-
193
- Evidence:
194
- - coordinator/user approval at 2026-08-04 00:09 Europe/Amsterdam;
195
- - `PROCESSED_TEXT_SELECTION_MASKING_PLAN.md`;
196
- - `PROCESSED_TEXT_SELECTION_MASKING_CONTRACT.md`;
197
- - `test_cases/processed_text_selection_masking/contract.json`.
198
-
199
- ## 2026-08-03 — D039 — Add Zorgfilter v1 with clinical-context preservation
200
-
201
- Status: accepted product and implementation-sequence decision
202
-
203
- Decision:
204
-
205
- ```text
206
- Add an explicit Dutch Zorg profile. Replace date of birth and patient/client identity by default. Show other exact care dates, provider identity, BIG/AGB, organizations and locations for review and select them by default. Preserve diagnosis, medication, dosage, laboratory results and observations. Surface rare-case indirect re-identification as residual-risk evidence rather than blindly masking clinical meaning.
207
- ```
208
-
209
- Implementation sequence:
210
- - policy and fully synthetic corpus first;
211
- - current-engine baseline and gap triage second;
212
- - recognizer contracts and pure recognizer implementation before UI;
213
- - central profile configuration before adding the fourth Streamlit profile;
214
- - cross-profile regression and live app verification before closeout.
215
-
216
- Reason:
217
- - Care documents contain both direct identifiers and essential clinical meaning.
218
- - A broad medical-word filter would destroy usability and potentially clinical/legal context.
219
- - The current `NL_HEALTHCARE_REFERENCE` category combines values with different privacy policies and must be split through evidence-driven work.
220
- - A fourth UI label without dedicated detection evidence would be cosmetic and unsafe.
221
-
222
- Boundaries:
223
- - synthetic data only;
224
- - human review remains required;
225
- - no production-readiness claim from corpus or benchmark results;
226
- - no silent profile switching;
227
- - no cloud document processing;
228
- - no export, Scrub Key or reinsert semantic change in the foundation package.
229
-
230
- ## 2026-08-03 — D038 — Use AI-first execution with human-controlled signing, security and release gates for Phase 9 desktop packaging
231
-
232
- Status: accepted roadmap and execution-model decision; implementation remains gated
233
-
234
- Decision:
235
-
236
- ```text
237
- When the final local desktop/offline installer phase is explicitly opened, use scoped AI agents for deterministic packaging, CI, synthetic testing and release-candidate preparation. Target a signed Tauri Windows shell with a bundled PyInstaller onedir Python/Presidio sidecar, a low-friction setup.exe and a managed-deployment MSI. Retain human control over publisher identity, production signing, public release, security claims, real-user acceptance and safety-critical semantic changes.
238
- ```
239
-
240
- Planning assumptions:
241
- - 60–70% of first-cycle development/integration labor may be agent-executed;
242
- - 75–90% of repetitive later release work may be automated;
243
- - post-agent development/integration planning range: approximately EUR 8,000–24,000;
244
- - independent security review, test hardware, signing and human release oversight remain retained costs.
245
-
246
- Boundaries:
247
- - Phase 9 remains gated by Phase 6 quality closeout and explicit coordinator approval.
248
- - No installer, runtime, UI, dependency, export, Scrub Key, reinsert or cloud behavior changes in this decision package.
249
- - Synthetic data only.
250
- - No single agent receives unrestricted repository-write, signing-identity and public-release authority.
251
- - Successful packaging alone does not justify a production-readiness or local-only security claim.
252
-
253
- ## 2026-07-28 — D037 — Validate document/key binding before every reinsert replacement
254
-
255
- Status: accepted reinsert-integration decision
256
-
257
- Decision:
258
-
259
- ```text
260
- Validate the complete supported text surface against the supplied Scrub Key before any original value is restored. Permit verified bound matches and explicit legacy-v1.0 compatibility only. All mismatch, mixed-binding, missing-binding, invalid-digest and invalid-bound-key states fail closed with zero replacements and no partial DOCX output.
261
- ```
262
-
263
- Reason:
264
-
265
- - Structural key validity alone cannot prove that a key belongs to a document.
266
- - Applying a wrong but valid key can silently restore incorrect confidential values.
267
- - Validation must happen before mutation so document-level helpers cannot produce partial output.
268
- - Legacy compatibility remains necessary, but it must never be presented as a verified document match.
269
- - The document-first three-step UX remains simpler and does not require new execution gates.
270
 
271
  ---
272
 
273
- ## 2026-07-27 — D036 — Preserve custom replacement text and fail verified key export rather than silently rewriting it
274
 
275
- Status: accepted export-integration decision
276
 
277
  Decision:
278
 
279
  ```text
280
- Generate document-bound placeholders by default. Rebind only recognised placeholder tokens. Never silently replace arbitrary user-chosen replacement text with a placeholder. When any included mapping remains unbound, keep the document export behavior but block schema-1.1 Scrub Key download with a visible validation warning.
281
- ```
282
-
283
- Reason:
284
-
285
- - Review choices remain source of truth.
286
- - Silent rewriting would change legal/readability semantics and user intent.
287
- - A bound key must not claim verified document matching for an unbound mapping.
288
- - Fail-visible key export is safer than a false binding claim.
289
 
290
- ---
291
-
292
- # SolidPrivacy Scrub — Decision Log
293
-
294
- This file records accepted strategic, product and architecture decisions.
295
-
296
- ---
297
-
298
- ## 2026-07-27 — D035 — Keep binding-model implementation pure until sequential export and reinsert integration
299
-
300
- Status: accepted implementation decision
301
-
302
- Decision:
303
-
304
- ```text
305
- Implement document/Scrub-Key binding as a new pure helper module first. Do not alter current placeholder creation, Scrub Key export/import, deterministic replacement or Streamlit behavior in the model package. Export integration creates bound artifacts in the next package; reinsert integration enforces binding only after bound export is proven.
306
  ```
307
 
308
- Reason:
309
-
310
- - Placeholder generation, key export and reinsert are shared safety-critical surfaces.
311
- - Pure helpers can be validated completely against the frozen contract without silently changing current output semantics.
312
- - Sequential integration preserves explicit migration and rollback boundaries.
313
 
314
- Implemented model boundaries:
315
-
316
- - local random/injected binding-ID generation;
317
- - strict bound placeholder build/parse and document-ID extraction;
318
- - canonical SHA-256 mapping digest;
319
- - bound key validation;
320
- - eight stable document/key statuses and six fail-closed statuses;
321
- - explicit legacy-v1.0 unbound compatibility;
322
- - no UI, export or reinsert integration;
323
- - no cloud, AI, file persistence, signing or secret storage.
324
-
325
- Evidence:
326
-
327
- - `scrub_key_binding.py`
328
- - `tests/test_scrub_key_binding_model.py`
329
- - `output/validation/mvp_scrub_key_binding_model_validation.json`
330
 
331
  ---
332
 
333
- ## 2026-07-27 — D034 — Freeze the bound-placeholder and mapping-digest contract before model implementation
334
 
335
- Status: accepted test/specification decision; model implementation may proceed
336
 
337
  Decision:
338
 
339
  ```text
340
- Freeze binding IDs as B plus sixteen uppercase RFC 4648 base32 characters, automatic placeholders as [LABEL_BINDINGID_INDEX], manual placeholders as [LABEL_BINDINGID_HANDMATIG_INDEX], and the bound-key direction as schema version 1.1 with binding version 1 and a canonical SHA-256 mapping digest. Preserve explicit legacy-v1.0 unbound compatibility and require all bound mismatch, mixed-ID, missing-binding, invalid-digest and invalid-bound-key states to fail closed before replacement.
 
 
 
 
 
341
  ```
342
 
343
- Reason:
344
-
345
- - Exact grammar and canonicalization are required before multiple shared placeholder, export and reinsert surfaces change.
346
- - A fixed synthetic digest fixture makes implementation independently testable.
347
- - Bound and legacy statuses must not be conflated.
348
- - UI simplification must survive the security change without new source/key execution gates.
349
-
350
- Boundaries:
351
-
352
- - Contract/tests only in this package; no product behavior change.
353
- - Mapping digest is not authenticity or a signature.
354
- - No automatic placeholder repair or legacy upgrade.
355
- - Preserve the three-step document-first reinsert flow and final confidential-download acknowledgement.
356
- - Model implementation remains pure and Streamlit-free.
357
- - Export and reinsert integration require later sequential packages.
358
- - Human review remains mandatory; no production-readiness claim.
359
-
360
- Evidence:
361
-
362
- - `SCRUB_KEY_BINDING_CONTRACT.md`
363
- - `test_cases/mvp_phase6/scrub_key_binding_contract.json`
364
- - `tests/test_mvp_scrub_key_binding_contracts.py`
365
- - `output/validation/mvp_scrub_key_binding_contract_validation.json`
366
 
367
  ---
368
 
369
- ## 2026-07-27 — D033 — Bind new Scrub Keys through document-specific placeholder namespaces
370
 
371
- Status: accepted planning/architecture decision; implementation requires green contract tests
372
 
373
  Decision:
374
 
375
- ```text
376
- For new bound Scrub Keys, carry one locally generated, non-sensitive document binding ID in every automatic and manual placeholder and in the corresponding Scrub Key. Complement this with a canonical SHA-256 mapping digest for accidental key-corruption detection. Reinsert must fail closed before any replacement when a bound key mismatches the document, the document contains mixed binding IDs, or the mapping digest is invalid.
377
- ```
378
-
379
- Reason:
380
-
381
- - Generic placeholder namespaces allow a wrong valid key to restore wrong originals without an audit mismatch.
382
- - A binding token inside placeholders survives pasted-text, TXT, DOCX and PDF-text roundtrips when the placeholders themselves survive.
383
- - Labels, filenames, metadata, content hashes and placeholder-list hashes are not reliable cross-format AI-roundtrip bindings.
384
- - A digest detects accidental edits but is not authenticity; malicious tampering requires later protected signing-key infrastructure.
385
-
386
- Compatibility and UX boundaries:
387
-
388
- - Introduce an explicit new bound-key contract; do not silently reinterpret legacy v1.0 keys.
389
- - Legacy unbound keys may remain dual-readable with a visible unbound warning.
390
- - Preserve the document-first three-step reinsert flow and the final confidential-download acknowledgement.
391
- - Add no repeated confirmation buttons or checkboxes.
392
- - Preserve unknown, duplicate and missing-placeholder audit reporting.
393
- - No cloud processing, server secret, OCR or restored-PDF behavior.
394
- - Human review remains mandatory; no production-readiness claim.
395
-
396
- Approved sequence:
397
-
398
- 1. `SCRUB-WP_MVP_SCRUB_KEY_BINDING_CONTRACT_TESTS`
399
- 2. `SCRUB-WP_MVP_SCRUB_KEY_BINDING_MODEL_IMPLEMENTATION`
400
- 3. `SCRUB-WP_MVP_SCRUB_KEY_BINDING_EXPORT_INTEGRATION`
401
- 4. `SCRUB-WP_MVP_SCRUB_KEY_BINDING_REINSERT_INTEGRATION`
402
- 5. `SCRUB-WP_MVP_SCRUB_KEY_BINDING_APP_VERIFY`
403
-
404
- Evidence:
405
-
406
- - `MVP_SCRUB_KEY_DOCUMENT_BINDING_GAP_TRIAGE.md`
407
- - `output/validation/mvp_scrub_key_document_binding_gap_triage.json`
408
- - `output/validation/mvp_scrub_key_roundtrip_validation_report.json`
409
 
410
  ---
411
 
412
- ## 2026-07-27 — D032 — Roundtrip evidence requires document/key-binding triage before implementation
413
 
414
- Status: accepted evidence-routing decision
415
 
416
  Decision:
417
 
418
- ```text
419
- Treat the missing document/Scrub-Key binding as a critical Phase 6 finding. Do not implement an implicit heuristic or silently change the Scrub Key schema, export or reinsert semantics inside the validation package. Open a separate triage package to define the binding contract and migration boundaries first.
420
- ```
421
-
422
- Reason:
423
-
424
- - A wrong key with a disjoint placeholder namespace is visibly rejected through unknown/not-found audit evidence.
425
- - A structurally valid wrong or tampered key with the same placeholder names restores wrong original values with no validation issue, unknown placeholder or missing-placeholder signal.
426
- - Document labels are descriptive metadata and are not currently a verified binding mechanism.
427
- - A safe correction may affect key creation, scrubbed output metadata, import compatibility and export semantics.
428
-
429
- Boundaries:
430
 
431
- - Preserve current deterministic local behavior until the triage decision is approved.
432
- - Do not guess whether a key belongs to a document.
433
- - Do not auto-repair malformed placeholders.
434
- - Preserve existing audit evidence and human review.
435
- - Use synthetic data only; no production-readiness claim.
436
-
437
- Evidence:
438
-
439
- - `output/validation/mvp_scrub_key_roundtrip_validation_report.json`
440
- - `test_cases/mvp_phase6/scrub_key_roundtrip_manifest.json`
441
- - `tests/test_mvp_scrub_key_roundtrip_validation.py`
442
 
443
  ---
444
 
445
- ## 2026-07-27 — D031 — Reinsert is document-first with automatic source/key processing
446
 
447
- Status: accepted evidence-driven UX and safety decision
448
 
449
  Decision:
450
 
451
- ```text
452
- Present local reinsert as source document/text → corresponding Scrub Key → restored download. Automatically recognise the source type, structurally validate the supplied Scrub Key and run deterministic local reinsert when both inputs are valid. Keep one explicit confidentiality acknowledgement at the restored-output download boundary instead of repeating acknowledgements and action buttons before processing.
453
- ```
454
-
455
- Reason:
456
-
457
- - Live Phase 6 verification confirmed that DOCX restoration works, but users can reasonably interpret an uploaded and visibly listed file as already accepted.
458
- - Requiring a checkbox and action button after each upload creates hidden completion states and unnecessary form friction.
459
- - The highest-risk user action is obtaining and handling the restored confidential output, so the explicit acknowledgement remains at that boundary.
460
- - Automatic key validation does not weaken structural validation or change Scrub Key semantics.
461
-
462
- Boundaries:
463
-
464
- - Keep warnings about pseudonymisation, key sensitivity and local-only use visible.
465
- - Invalid or ambiguous keys must still fail clearly and must not be used.
466
- - Preserve result/audit warnings for unknown, duplicate and missing placeholders.
467
- - Preserve existing helpers, output bytes, filenames and MIME types.
468
- - Do not add cloud, AI, OCR, restored-PDF or key-storage behavior.
469
- - Human review remains required and no production-readiness claim is created.
470
-
471
- Evidence:
472
-
473
- - User-confirmed restored synthetic DOCX containing body, table, header and footer values.
474
- - `tests/test_reinsert_auto_flow.py`
475
- - `tests/test_reinsert_auto_flow_ui.py`
476
- - `output/validation/mvp_reinsert_auto_flow_validation.json`
477
 
478
  ---
479
 
480
- ## 2026-07-17 — D030 — Restore existing DOCX header and footer text during deterministic reinsert
481
 
482
- Status: accepted implementation decision
483
 
484
- Decision:
485
 
486
- ```text
487
- Extend the existing local deterministic DOCX reinsert helper to process WordprocessingML text nodes in word/header*.xml and word/footer*.xml in addition to word/document.xml.
488
- ```
489
-
490
- Reason:
491
-
492
- - The scrubbed DOCX export already replaces reviewed values in body, table, header and footer paragraphs.
493
- - The Phase 6 matrix showed that reinsert restored body/table values but left header/footer placeholders behind.
494
- - The gap belongs to document fidelity and reinsert scope, not detection or recognizer behavior.
495
-
496
- Boundaries:
497
-
498
- - Process only existing body, header and footer WordprocessingML text nodes.
499
- - Preserve unrelated OOXML package parts byte-for-byte where they are not rewritten.
500
- - Do not claim support for comments, tracked-change-only parts, footnotes/endnotes, text boxes, metadata or placeholders split across text nodes.
501
- - Do not add OCR or restored-PDF behavior.
502
- - Keep processing local, deterministic and Scrub Key driven.
503
-
504
- Evidence:
505
-
506
- - `output/validation/mvp_phase6_document_hygiene_fidelity_hardening_report.json`
507
- - `output/validation/mvp_phase6_false_negative_gap_triage.json`
508
 
509
  ---
510
 
511
- ## 2026-07-17 — D029 — Current Phase 6 matrix does not justify a recognizer fix
512
 
513
- Status: accepted evidence-routing decision
514
 
515
  Decision:
516
 
517
- ```text
518
- Do not open a recognizer or threshold implementation package from the first corrected Phase 6 synthetic matrix.
519
- ```
520
-
521
- Reason:
522
-
523
- - The corrected matrix contains no reproducible detection false negative, misclassification or legal-role over-masking evidence.
524
- - The DOCX result concerns header/footer reinsert fidelity and helper scope.
525
- - The PDF result reflects the approved restored-TXT-only/no-OCR product boundary.
526
- - Treating either item as a recognizer problem would target the wrong layer and weaken evidence discipline.
527
-
528
- Consequences:
529
-
530
- - Route both findings to `SCRUB-WP_MVP_DOCUMENT_HYGIENE_FIDELITY_HARDENING`.
531
- - Preserve current recognizer and threshold behavior.
532
- - Keep the PDF limitation explicit; do not infer OCR or restored-PDF authorization.
533
- - Preserve human review and the no-production-readiness-claim boundary.
534
-
535
- Evidence:
536
-
537
- - `output/validation/mvp_phase6_false_negative_gap_triage.json`
538
- - `output/validation/mvp_phase6_synthetic_validation_report.json`
539
 
540
  ---
541
 
542
- ## 2026-07-17 — D028 — Phase 6 workflow validation becomes the active development line
543
 
544
- Status: accepted product-direction decision
545
 
546
  Decision:
547
 
548
- ```text
549
- Close the verified MVP UI simplification line as the default development focus and activate Phase 6 end-to-end workflow validation and trust hardening.
550
- ```
551
-
552
- Reason:
553
-
554
- ```text
555
- The current import, review, manual correction and export interface has been live-app verified and works as expected. The next material risk reduction comes from proving the supported workflow with synthetic evidence, then fixing only reproducible trust gaps.
556
- ```
557
-
558
- Consequences:
559
-
560
- - Start with `SCRUB-WP_MVP_E2E_SYNTHETIC_VALIDATION_MATRIX`.
561
- - Open recognizer, document-hygiene, Scrub Key/roundtrip or audit fixes only from reproducible evidence.
562
- - Do not start another broad UI package by default.
563
- - Phase 7 pilots remain parked until the Phase 6 quality gate is explicitly approved.
564
- - Local installer/packaging work remains deferred.
565
- - Human review remains mandatory; no production-readiness claim is created by this decision.
566
 
567
  ---
568
 
569
- ## 2026-07-03 — D027 — Basiscontrole / Expertcontrole as review-mode direction
570
-
571
- Status: accepted planning recommendation, implementation pending contract tests
572
-
573
- Decision:
574
-
575
- ```text
576
- Use Basiscontrole / Expertcontrole as the planning names for the two normal review-mode layers.
577
- Basiscontrole is the default MVP path.
578
- Expertcontrole exposes the full inspection/audit machinery.
579
- Mode switching changes visibility only, not processing or export semantics.
580
- ```
581
-
582
- Reason:
583
-
584
- ```text
585
- The MVP interface needs a true less-is-more default path without weakening legal/privacy review controls. Basiscontrole communicates lower cognitive load while keeping the workflow framed as a serious control task.
586
- ```
587
 
588
- Boundaries:
589
 
590
- - Basiscontrole is not weaker review.
591
- - The review table remains source of truth internally.
592
- - Mode switching must not change recognizer behavior, replacement logic, export output, Scrub Key JSON, reinsert behavior or audit generation.
593
- - Implementation requires contract tests first.
594
 
595
  ---
596
 
597
- ## 2026-06-18 — D026 — Temporarily prioritize MVP UI cleanup and export/download redesign
598
-
599
- Status: accepted product-direction decision
600
 
601
- Decision:
602
 
603
- ```text
604
- Pause new recall/benchmark follow-up packages temporarily and prioritize MVP UI cleanup/export redesign.
605
- ```
606
 
607
- Reason:
608
-
609
- ```text
610
- The app must move from a prototype/debug interface toward a professional MVP workflow.
611
- ```
612
-
613
- Consequence:
614
-
615
- ```text
616
- Next packages focus on export/download UX and hiding/collapsing debug details without weakening safety controls.
617
- ```
618
-
619
- Implications:
620
 
621
- - Recall/benchmark follow-up packages are parked unless a concrete blocker appears.
622
- - Export/download UX is now the active next user-visible improvement line.
623
- - Technical/audit details must remain available but move out of the primary flow where appropriate.
624
- - Scrub Key must stay clearly separated and visibly sensitive.
625
- - Export semantics must not change silently.
626
- - The review table remains source of truth and fallback.
627
- - No Streamlit implementation is approved by this planning decision; implementation requires separate workpackages.
628
 
629
  ---
630
 
631
- ## 2026-06-18 — D025 — PERSON-name implementation requires green contract tests first
632
-
633
- Status: accepted tests/specification decision
634
-
635
- Decision:
636
-
637
- ```text
638
- PERSON-name recognizer implementation may only start after contract tests are green.
639
- ```
640
-
641
- Reason:
642
 
643
- ```text
644
- Value-only matching and role preservation are safety-critical.
645
- ```
646
 
647
- Consequence:
648
 
649
- ```text
650
- Implementation package must satisfy the contract fixture before benchmark review.
651
- ```
652
 
653
  ---
654
 
655
- ## 2026-06-18 — D024 — PERSON-name improvement proceeds test-first
656
-
657
- Status: accepted planning/specification decision
658
 
659
- Decision:
660
-
661
- ```text
662
- PERSON-name improvement will proceed test-first.
663
- ```
664
 
665
- Reason:
666
 
667
  ```text
668
- Single-surname and role/context cases are high-risk for over-masking and legal/care meaning damage.
 
 
 
669
  ```
670
 
671
- Consequence:
672
-
673
- ```text
674
- Contract tests are required before recognizer implementation.
675
- ```
676
 
677
  ---
678
 
679
- ## 2026-06-15 — D023 — Synchronized scrolling is default review behavior, not a user-facing technical control
680
-
681
- Status: accepted bounded UX refinement decision
682
-
683
- Decision:
684
-
685
- ```text
686
- In the central side-by-side review surface, synchronized scrolling should be on by default and should not be exposed as a visible checkbox.
687
- ```
688
 
689
- Boundary:
690
 
691
- This decision does not change replacement behavior, export/download behavior, Scrub Key behavior or reinsert behavior.
692
 
693
  ---
694
 
695
- ## 2026-06-15 — D022 — Bounded synchronized side-by-side scrolling approved after prototype review
696
-
697
- Status: accepted bounded UX implementation decision; refined by D023 for visible control behavior
698
-
699
- Decision:
700
 
701
- ```text
702
- After visual review of the isolated synchronized-scroll prototype, bounded synchronized scrolling may be integrated into the existing side-by-side review surface.
703
- ```
704
 
705
- Implementation boundaries:
706
 
707
- - Keep synchronized scrolling bounded to the side-by-side review surface.
708
- - Do not change replacement behavior.
709
- - Do not mutate review table state.
710
- - Do not write or change Scrub Key data.
711
- - Do not change export/download behavior.
712
- - Do not change reinsert behavior.
713
- - Do not use real data.
714
 
715
  ---
716
 
717
- ## 2026-06-14 — D021 — Unified side-by-side review surface is the target review UX
718
-
719
- Status: accepted product/UX direction
720
-
721
- Decision:
722
-
723
- ```text
724
- The review UX should move toward one unified side-by-side main review surface: source text on the left, processed/checked text on the right, with optional highlights integrated in the processed-text pane. The product should not keep adding separate helper panels or duplicate preview expanders for every review feature.
725
- ```
726
-
727
- Implications:
728
-
729
- - Future review UX work should centralize around source-vs-processed comparison.
730
- - The review table remains source of truth and fallback.
731
- - Serial review remains a guided review layer, not a replacement of the table.
732
- - The old replacement decision helper panel must not return as normal user-facing UI.
733
- - Do not start panel removal, click-to-mark, advanced editor, full-document marking, Scrub Key writes, export blocking or reinsert behavior changes without separate approved packages.
734
 
735
- ---
736
 
737
- ## Historical note
738
 
739
- Older decisions remain available in Git history.
 
1
+ # SolidPrivacy Scrub — Current Decision Log
2
 
3
+ This file records **accepted decisions that remain materially binding on current/future product work**.
4
 
5
+ Detailed pre-convergence decision history remains recoverable from exact baseline:
6
 
7
  ```text
8
+ 54c73e0ebf5a3a3ed7039a50596fb57694add3cd
9
  ```
10
 
11
+ Git and `CHANGELOG.md` are the historical provenance layers. This file must not become another implementation-history ledger.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
 
13
  ---
14
 
15
+ ## D044 — 2026-09-04 — Repository Convergence before further normal feature development; Scrub Private becomes the active future production line
16
 
17
+ Status: **accepted strategic/architecture direction; implementation of the bootstrap candidate remains subject to normal independent assurance**
18
 
19
  Decision:
20
 
21
  ```text
22
+ Pause normal new feature development.
 
 
 
23
 
24
+ Preserve exact current source by SHA
25
+ → reconstruct active/reachable current truth
26
+ → classify CANONICAL / RECONCILE / RETIRE / VARIANT-SPECIFIC
27
+ → remove only proven debt
28
+ → reconcile current documentation/issues/evidence authority
29
+ → verify one clean baseline
30
+ → after convergence, continue main as Scrub Private
31
+ → finish Private-oriented application behavior on Hugging Face with synthetic data
32
+ → add production Private Service controls
33
+ → external assurance
34
+ → pilot Legal first, then Zorg based on evidence.
35
  ```
36
 
37
+ Reasons:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
38
 
39
+ - current repository contains mature functionality that must not be rebuilt from stale assumptions;
40
+ - canonical docs/workpackages/issues contain multiple generations of status and local-first deployment assumptions;
41
+ - active technical legacy and valid-but-hosted-incompatible functionality must be distinguished before removal;
42
+ - version control preserves prior/full-feature functionality more safely than an in-repository source clone;
43
+ - the shortest path to the business outcome is to clean current truth first, then adapt only the trust boundary required by Scrub Private.
44
 
45
+ Binding consequences:
46
 
47
+ - exact Git SHA is preservation authority; optional tags are only human-readable markers;
48
+ - no `/app_v2`, `/scrub-new`, duplicate Streamlit main or other source clone;
49
+ - `ROADMAP.md` contains five strategic stages only;
50
+ - `WORKPACKAGES.md` contains one current executable queue only;
51
+ - the temporary convergence debt ledger is not a new permanent source of truth;
52
+ - no cleanup refactor is justified by aesthetics alone;
53
+ - no new Evidence Framework; reconcile/reuse existing synthetic/benchmark/E2E/security evidence;
54
+ - human review and existing privacy/safety semantics remain binding;
55
+ - Hugging Face is a synthetic/approved-test application-validation environment, not confidential-production infrastructure assurance;
56
+ - Local/offline capability remains recoverable/deferred and is not the active delivery line;
57
+ - after `SCRUB_REPOSITORY_CONVERGED`, `main` becomes the active Scrub Private development line.
58
 
59
+ Scrub Private content-plane direction:
60
 
61
  ```text
62
+ no intentional persistent customer document content
63
+ no content-bearing ordinary application logs
64
+ no document-content backup
65
+ no third-party document-processing egress
66
+ minimal non-document control-plane persistence only when a real service requirement justifies it
67
  ```
68
 
69
+ This decision does not itself remove current replacement-memory or external-processing functionality. Those are variant-specific candidates for a later, separately assured Private adaptation after the clean baseline.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
70
 
71
  ---
72
 
73
+ ## D043 — 2026-08-08 — Premium Standard is a single-page staged document workspace
74
 
75
+ Status: accepted and implemented direction; preserve unless current runtime evidence justifies change.
76
 
77
  Decision:
78
 
79
  ```text
80
+ One document. One workspace. Three stages. One active task.
 
 
 
 
 
 
 
 
81
 
82
+ Toevoegen → Controleren → Downloaden
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
83
  ```
84
 
85
+ Binding behavior:
 
 
 
 
86
 
87
+ - all three stages remain represented in one workspace;
88
+ - exactly one Standard stage is dominant at a time;
89
+ - completed stages collapse to compact summaries;
90
+ - future stages are visible but passive;
91
+ - successful completion advances safely;
92
+ - explicit return to earlier stages is allowed;
93
+ - processing-affecting changes invalidate downstream review/export state fail-closed;
94
+ - Standard is lower cognitive load, not lower safety;
95
+ - three routed pages and a nested-expander long form are rejected as the Standard core-flow target.
 
 
 
 
 
 
 
96
 
97
  ---
98
 
99
+ ## D042 — 2026-08-06 — Separate implementation from blind independent release assurance
100
 
101
+ Status: accepted and binding governance decision.
102
 
103
  Decision:
104
 
105
  ```text
106
+ implementation_operations
107
+ → identifiable exact candidate
108
+ → governance_release_assurance fresh blind reconstruction
109
+ → PASS | FAIL | INDETERMINATE
110
+ → authorized action
111
+ → independent post-action confirmation
112
  ```
113
 
114
+ Implementation may not certify its own consequential candidate. Assurance may not repair what it reviews and may not read implementation conclusions before its initial verdict.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
115
 
116
  ---
117
 
118
+ ## D041 — 2026-08-05 — Application-wide Standard/Expert premium core-flow shell
119
 
120
+ Status: accepted and implemented direction.
121
 
122
  Decision:
123
 
124
+ - top-level `Anonimiseren | Terugzetten` workflows;
125
+ - global `Standaard | Expert` presentation;
126
+ - Standard uses the staged `Toevoegen → Controleren → Downloaden` workspace;
127
+ - Expert operates over the same authoritative source/processing/review state;
128
+ - presentation-only switching must not silently change profile/settings, reprocess or reset valid state;
129
+ - human review and export/Scrub Key/reinsert semantics are not weakened by presentation simplification.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
130
 
131
  ---
132
 
133
+ ## D040 — 2026-08-04 — Server-authoritative direct masking from processed text
134
 
135
+ Status: accepted and implemented product decision.
136
 
137
  Decision:
138
 
139
+ A user may select a missed sensitive value in processed text, but browser selection is only an input event. The server validates scope, offsets, exact occurrences, collisions and stale/replay state before creating one normal document-bound manual replacement row.
 
 
 
 
 
 
 
 
 
 
 
140
 
141
+ The review table/authoritative review state remains source of truth. Direct masking may not write Scrub Keys or exports independently.
 
 
 
 
 
 
 
 
 
 
142
 
143
  ---
144
 
145
+ ## D039 — 2026-08-03 — Zorg profile with clinical-context preservation
146
 
147
+ Status: accepted and implemented product decision.
148
 
149
  Decision:
150
 
151
+ - direct patient/client identity and date of birth: replace;
152
+ - other exact care dates/provider identity and dedicated care identifiers: review according to policy, selected by default where defined;
153
+ - diagnosis, medication, dosage, laboratory results, observations and clinically meaningful context: preserve;
154
+ - rare indirect-identification risk: surface for review/audit rather than blindly destroying clinical meaning;
155
+ - synthetic corpus/evidence and human review remain mandatory; no production-safety claim from corpus scores alone.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
156
 
157
  ---
158
 
159
+ ## D038 — 2026-08-03 — Local desktop packaging is a deferred product option, not current execution authority
160
 
161
+ Status: historical decision retained only where still applicable.
162
 
163
+ The previously selected signed Tauri/PyInstaller local packaging direction remains recoverable if Local becomes an active product requirement later.
164
 
165
+ D044 supersedes its roadmap priority. Installer work is not part of the current execution line.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
166
 
167
  ---
168
 
169
+ ## D037 — 2026-07-28 — Validate document/key binding before every reinsert replacement
170
 
171
+ Status: accepted and binding safety decision.
172
 
173
  Decision:
174
 
175
+ Validate the complete supported document text surface against the supplied Scrub Key before restoring any original value. Bound mismatch, mixed binding, missing binding or invalid mapping digest fails closed with zero replacements. Legacy unbound behavior remains explicitly unverified where supported.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
176
 
177
  ---
178
 
179
+ ## D036 — 2026-07-27 — Preserve intentional custom replacement text; fail verified key export rather than silently rewriting it
180
 
181
+ Status: accepted and binding review/export decision.
182
 
183
  Decision:
184
 
185
+ Do not silently convert arbitrary user-chosen replacement text into bound placeholders merely to make Scrub Key export succeed. Preserve the reviewed replacement decision and fail visibly/block verified key export when a fully bound mapping cannot be established.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
186
 
187
  ---
188
 
189
+ ## D035 — 2026-07-27 — Implement Scrub Key binding in pure model first, then integrate sequentially
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
190
 
191
+ Status: accepted implementation/safety pattern; completed historically.
192
 
193
+ Decision remains relevant as a precedent: safety-critical shared placeholder/key/reinsert changes should be isolated in pure/testable helpers before sequential integration into export/reinsert when new changes are genuinely required.
 
 
 
194
 
195
  ---
196
 
197
+ ## D034 — 2026-07-27 — Bound-placeholder and mapping-digest contract
 
 
198
 
199
+ Status: accepted and implemented safety contract.
200
 
201
+ Binding direction:
 
 
202
 
203
+ - document-specific binding ID in automatic/manual bound placeholders;
204
+ - schema-1.1 bound key direction;
205
+ - canonical SHA-256 mapping digest;
206
+ - explicit legacy-unbound compatibility state;
207
+ - mismatch/mixed/missing/digest-invalid bound states fail closed before replacement.
 
 
 
 
 
 
 
 
208
 
209
+ Do not shorten or mutate authoritative bound tokens merely for visual convenience; compact aliases are presentation-only.
 
 
 
 
 
 
210
 
211
  ---
212
 
213
+ ## D033 — 2026-07-27 — Document-specific placeholder namespaces bind new Scrub Keys to their document
 
 
 
 
 
 
 
 
 
 
214
 
215
+ Status: accepted and implemented architecture decision.
 
 
216
 
217
+ Each new bound Scrub Key/document uses one locally generated non-sensitive binding ID carried by automatic/manual placeholders and the corresponding key. A mapping digest complements this for accidental corruption detection. The binding token, not filenames/labels/content heuristics, is the cross-format document/key association.
218
 
219
+ Legacy unbound keys remain explicitly distinguishable and must not be silently reinterpreted as verified bound matches.
 
 
220
 
221
  ---
222
 
223
+ ## D031 — 2026-07-27 — Reinsert is document-first and automatically processes valid source/key inputs
 
 
224
 
225
+ Status: accepted and implemented UX/safety decision.
 
 
 
 
226
 
227
+ Current reinsert direction:
228
 
229
  ```text
230
+ source document/text
231
+ → corresponding Scrub Key
232
+ → deterministic validation/reinsert
233
+ → restored download
234
  ```
235
 
236
+ Source type recognition and structural key validation may run automatically when valid inputs are present. Do not reintroduce redundant hidden confirmation buttons/checkboxes merely to create a sense of safety. Keep the meaningful confidentiality acknowledgement at the restored-output boundary and fail invalid/ambiguous keys visibly.
 
 
 
 
237
 
238
  ---
239
 
240
+ ## D030 — 2026-07-17 — Supported DOCX reinsert includes body, tables, headers and footers
 
 
 
 
 
 
 
 
241
 
242
+ Status: accepted and implemented document-fidelity decision.
243
 
244
+ Restore supported placeholders in existing `word/document.xml`, `word/header*.xml` and `word/footer*.xml` text nodes. Do not imply support for comments, tracked-change-only parts, footnotes/endnotes, text boxes, metadata or split placeholders unless a separate evidence-backed package adds and verifies that support.
245
 
246
  ---
247
 
248
+ ## D021 — 2026-06-14 — Unified side-by-side review is the target review surface
 
 
 
 
249
 
250
+ Status: accepted and still materially binding product/UX decision.
 
 
251
 
252
+ The main review experience centers on source text/document versus processed/checked text. Highlights are a visual aid, not an alternate mutation authority. The authoritative review table/state remains the fallback/source of truth, and future review work should avoid proliferating duplicate helper panels.
253
 
254
+ This direction is implemented within the later D041/D043 Premium staged workspace.
 
 
 
 
 
 
255
 
256
  ---
257
 
258
+ # Current decision discipline
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
259
 
260
+ New decisions belong here only when they remain materially binding on current/future work.
261
 
262
+ Do not copy workpackage status, test runs or implementation narrative into this file.
263
 
264
+ If a decision is superseded, mark the superseding decision explicitly rather than leaving two competing current rules.
PROJECT_PROMPT.md CHANGED
@@ -1,12 +1,12 @@
1
  # SolidPrivacy Scrub — Project Prompt / Worker Instructions
2
 
3
- This file is the full project prompt for workers on SolidPrivacy Scrub.
4
 
5
- For ChatGPT Project Instructions, use the shorter bootstrap prompt in `PROJECT_PROMPT_SHORT.md`. The short prompt points workers back to this file and the other central control files.
6
 
7
  ---
8
 
9
- ## Repository
10
 
11
  Work only in:
12
 
@@ -14,267 +14,298 @@ Work only in:
14
  solidprivacy-nl/scrub
15
  ```
16
 
17
- Do not modify unrelated repositories. If the active repository is not `solidprivacy-nl/scrub`, stop and report the mismatch.
 
 
 
 
18
 
19
  ---
20
 
21
- ## Source of truth
22
 
23
- GitHub is the source of truth.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
24
 
25
- Always start by reading, in this order:
 
 
26
 
27
  1. `PROJECT_PROMPT.md`
28
  2. `ROADMAP.md`
29
  3. `WORKPACKAGES.md`
30
  4. `CHANGELOG.md`
31
 
32
- Then read these when relevant:
33
 
34
- - `RISK_REGISTER.md` for product, privacy and trust risks;
35
- - `DECISION_LOG.md` for strategic decisions and architectural boundaries;
36
- - `RELEASE_NOTES.md` for user-facing product changes;
37
- - `STATUS_MONITORING_RUNBOOK.md` for Actions/sync monitoring procedure;
38
- - relevant specs such as `SCRUB_KEY_SPEC.md` and `PDF_TEXT_REINSERT_UI_PLAN.md`.
 
39
 
40
- Use these files as follows:
41
 
42
- - `PROJECT_PROMPT.md` = full worker rules and operating model.
43
- - `PROJECT_PROMPT_SHORT.md` = short bootstrap prompt suitable for ChatGPT Project Instructions.
44
- - `ROADMAP.md` = product vision, risk-driven strategic direction, phase order, architecture.
45
- - `WORKPACKAGES.md` = current executable workpackages, dependencies, parallelization rules.
46
- - `CHANGELOG.md` = internal implementation history.
47
- - `RELEASE_NOTES.md` = human/user-facing product change summary.
48
- - `RISK_REGISTER.md` = active risk list and mitigation ownership.
49
- - `DECISION_LOG.md` = accepted strategic/product/architecture decisions.
50
 
51
- Do not invent a new direction if it conflicts with these files. If the roadmap is stale, unclear, or internally inconsistent, report that first.
52
 
53
  ---
54
 
55
- ## Implementation-versus-assurance separation
56
 
57
- Consequential work uses the canonical two-role model linked from `control/PROJECT_GOVERNANCE_BOOTSTRAP.md` and the project contract in `control/SCRUB_RELEASE_ASSURANCE_CONTRACT_V1.md`.
58
 
59
  ```text
60
- implementation_operations
61
- governance_release_assurance
62
  ```
63
 
64
- The user gives one instruction and receives one consolidated status. Implementation prepares an identifiable release candidate but may not certify its own completion. Governance independently reconstructs the candidate and issues `PASS`, `FAIL` or `INDETERMINATE`; it may not silently repair what it reviews.
65
 
66
- Before its initial decision, the assurance worker must not read the implementation handover, self-assessment or conclusions. It works from the requested outcome, authoritative project files, candidate source/diff, acceptance criteria and raw machine/deployment evidence. Only after recording the initial decision may it open the implementation handover for disclosure and administrative closeout checks.
67
 
68
- A candidate requiring repair returns to implementation and receives a fresh assurance pass. Consequential implementation and verification must be separate workpackages and separate workers/sessions.
69
-
70
- For consequential work, read after the mandatory start sequence:
71
 
72
- 1. `control/PROJECT_GOVERNANCE_BOOTSTRAP.md`
73
- 2. `control/SCRUB_RELEASE_ASSURANCE_CONTRACT_V1.md`
74
 
75
- ## Current way of working
76
 
77
- Work in small, testable workpackages.
 
 
 
 
 
 
78
 
79
- Scrub is now run as a risk-driven privacy product, not as a simple feature ladder. The highest priority risks are:
80
 
81
- 1. missed sensitive data / false negatives;
82
- 2. Scrub Key leakage or misuse;
83
- 3. hidden document content and metadata;
84
- 4. cloud/demo trust gap versus local-first promise;
85
- 5. placeholder corruption during AI roundtrip;
86
- 6. UI review limitations that prevent reliable human review.
87
 
88
- For each workpackage:
89
 
90
- 1. Confirm the workpackage title and scope.
91
- 2. Check dependencies in `WORKPACKAGES.md`.
92
- 3. Check `RISK_REGISTER.md` when the work touches detection, Scrub Key, export, document parsing, UI review or deployment.
93
- 4. Check `DECISION_LOG.md` when the work touches strategy, architecture or explicit product boundaries.
94
- 5. Prefer helper modules and tests before UI changes.
95
- 6. Avoid parallel edits to the same UI patch area.
96
- 7. Add or update tests where meaningful.
97
- 8. Update `CHANGELOG.md` for implementation history.
98
- 9. Update `RELEASE_NOTES.md` for user-visible product changes.
99
- 10. Update `ROADMAP.md` only when strategy, phase status, risk priority or sequence changes.
100
- 11. Update `WORKPACKAGES.md` when execution status, dependencies or next workpackages change.
101
- 12. End with a handover summary.
102
- 13. Write the handover summary to `handover/workpackages/`.
103
 
104
  ---
105
 
106
- ## Parallelization rule
107
 
108
- Safe to do in parallel:
109
 
110
- - helper modules with separate files;
111
- - tests that do not touch the same UI patch;
112
- - specifications;
113
- - documentation;
114
- - benchmark data design;
115
- - risk reviews;
116
- - non-UI architecture work.
117
-
118
- Do not do in parallel without explicit coordination:
119
-
120
- - `presidio_streamlit.py`;
121
- - `fix_streamlit_nested_expanders.py`;
122
- - any additional Streamlit patch file that edits the same UI flow;
123
- - review table UI flow;
124
- - export/download UI flow;
125
- - shared workflow state;
126
- - Docker/runtime startup patch order.
127
-
128
- When in doubt, keep UI integration sequential.
129
 
130
- ---
131
 
132
- ## Testing, sync and self-monitoring
133
 
134
- After implementation, workers must not rely on coordinator screenshots as the first verification path.
135
 
136
- Workers should self-check, where connector permissions allow:
 
 
 
 
 
137
 
138
- 1. GitHub Actions status for the relevant commit.
139
- 2. GitHub to Hugging Face sync status for the relevant commit.
140
- 3. Failed-job logs when Actions are red.
141
- 4. Whether a fix workpackage is needed before asking for app verification.
142
 
143
- Use `STATUS_MONITORING_RUNBOOK.md` for the monitoring procedure.
144
 
145
- Only ask the coordinator/user for:
 
 
 
 
 
146
 
147
- - app verification when UI behavior changed;
148
- - missing permissions or inaccessible logs;
149
- - subjective UX confirmation;
150
- - explicit approval for gated workpackages.
151
 
152
- Do not claim functional success until either tests prove it or the user confirms it in the app.
153
 
154
  ---
155
 
156
- ## Changelog and release-notes discipline
157
 
158
- Every meaningful internal change must be reflected in `CHANGELOG.md`.
159
 
160
- Include:
161
 
162
- - workpackage/version;
163
- - status;
164
- - purpose;
165
- - files added/changed;
166
- - main changes;
167
- - tests;
168
- - validation status;
169
- - intentionally not changed.
 
 
 
 
 
170
 
171
- For user-visible product changes, also update `RELEASE_NOTES.md` in human language.
172
 
173
- Keep `CHANGELOG.md` as an internal implementation log. Keep `RELEASE_NOTES.md` as the product-facing summary.
174
 
175
  ---
176
 
177
- ## Handover discipline
178
 
179
- Every worker must end with a handover summary.
180
 
181
- The handover must explicitly state:
 
 
 
 
182
 
183
- - repository worked in;
184
- - workpackage title;
185
- - status;
186
- - files added;
187
- - files changed;
188
- - tests added/updated;
189
- - validation status;
190
- - GitHub Actions status if known;
191
- - Hugging Face sync status if known;
192
- - app verification status if known;
193
- - remaining risks or follow-up actions;
194
- - next recommended step.
195
 
196
- Also write the same handover summary to:
 
 
 
 
 
 
197
 
198
- ```text
199
- handover/workpackages/
200
- ```
201
 
202
- Use this filename format:
203
 
204
- ```text
205
- handover/workpackages/YYYYMMDD_HHMM_<workpackage_slug>.md
206
- ```
207
 
208
- ### Handover template
209
 
210
- ```markdown
211
- # Handover — <Workpackage title>
212
 
213
- Repository: solidprivacy-nl/scrub
214
- Status: <completed / implemented / blocked / pending verification>
 
 
 
 
215
 
216
- ## Summary
217
 
218
- <Short explanation of what was done.>
219
 
220
- ## Files added
221
 
222
- - ...
223
 
224
- ## Files changed
225
 
226
- - ...
227
 
228
- ## Tests
229
 
230
- - ...
231
 
232
- ## Validation
233
 
234
- - GitHub Actions: <green/red/unknown>
235
- - Hugging Face sync: <green/red/unknown>
236
- - App verification: <confirmed/pending/not applicable>
237
 
238
- ## Notes / risks
239
 
240
- - ...
241
 
242
- ## Next recommended step
243
 
244
- - ...
 
245
  ```
246
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
247
  ---
248
 
249
- ## Safety and quality rules
250
 
251
- - Do not remove existing functionality unless the workpackage explicitly requires it.
252
- - Do not silently change export semantics.
253
- - Do not weaken privacy or review controls.
254
- - Do not introduce cloud dependencies for document processing unless explicitly approved.
 
 
255
  - Do not store secrets, tokens or real personal data.
256
- - Use synthetic data only.
257
- - Preserve legal context: replace sensitive values, not legal meaning.
258
- - Be honest about uncertainty, failed validation or incomplete implementation.
259
- - Treat the Scrub Key as sensitive data because it can re-identify scrubbed content.
260
- - Treat false negatives as a product-critical risk, not as a cosmetic bug.
261
 
262
- ---
263
 
264
- ## Product direction
265
 
266
- The product direction remains:
267
 
268
- ```text
269
- Scrub → Review → Scrub Key → AI → Reinsert → Export → Audit
270
- ```
 
 
271
 
272
- The first product wedge remains Dutch confidential professional documents, with Legal and Zorg as the most relevant early validation markets.
273
 
274
- The broader direction is a local-first Dutch privacy scrubber for professional confidential documents.
 
 
275
 
276
- The immediate development discipline is now risk-driven:
 
 
277
 
278
  ```text
279
- Stabilize open UI work → Trust/recall benchmark → Scrub Key security → Placeholder robustness → Document hygiene → Document-centric review → Local runtime → Pilot validation → Scale features
280
  ```
 
 
 
1
  # SolidPrivacy Scrub — Project Prompt / Worker Instructions
2
 
3
+ This file contains the stable worker instructions for SolidPrivacy Scrub.
4
 
5
+ For ChatGPT Project Instructions, use the shorter bootstrap prompt in `PROJECT_PROMPT_SHORT.md`. The short prompt points workers back to this file and the other canonical control files.
6
 
7
  ---
8
 
9
+ ## 1. Repository scope
10
 
11
  Work only in:
12
 
 
14
  solidprivacy-nl/scrub
15
  ```
16
 
17
+ Use `solidprivacy-nl/solidprivacy` only for explicitly shared SolidPrivacy privacy/AI workflows, regulatory knowledge, orchestration, evaluation frameworks and reusable privacy components when the workpackage calls for it.
18
+
19
+ Do not modify unrelated repositories.
20
+
21
+ GitHub is the source of truth.
22
 
23
  ---
24
 
25
+ ## 2. Engineering constitution
26
 
27
+ The canonical expanded engineering doctrine is the SolidPrivacy **Execution & Engineering Constitution**.
28
+
29
+ Binding principles include:
30
+
31
+ - business outcome first;
32
+ - smallest complete solution;
33
+ - solid but simple;
34
+ - no overengineering;
35
+ - first-principles reasoning;
36
+ - prefer proven/native solutions;
37
+ - critique and refine material work;
38
+ - keep executing once the objective and constraints are clear;
39
+ - progress reporting must not replace execution;
40
+ - solve root causes and keep one source of truth;
41
+ - verify behavior, security and maintainability;
42
+ - remove obsolete/conflicting active paths;
43
+ - Done means outcome + implementation + verification + cleanup + documentation alignment.
44
+
45
+ Project requirements may strengthen this doctrine, not weaken it.
46
+
47
+ ---
48
 
49
+ ## 3. Required start sequence
50
+
51
+ At the start of every work session, read in this exact order:
52
 
53
  1. `PROJECT_PROMPT.md`
54
  2. `ROADMAP.md`
55
  3. `WORKPACKAGES.md`
56
  4. `CHANGELOG.md`
57
 
58
+ Then read when relevant:
59
 
60
+ - `RISK_REGISTER.md` — current product/privacy/security risks;
61
+ - `DECISION_LOG.md` — accepted strategic and architecture decisions;
62
+ - `RELEASE_NOTES.md` — user-visible product changes;
63
+ - `STATUS_MONITORING_RUNBOOK.md` — Actions/Hugging Face monitoring;
64
+ - `control/PROJECT_GOVERNANCE_BOOTSTRAP.md` and `control/SCRUB_RELEASE_ASSURANCE_CONTRACT_V1.md` for consequential work;
65
+ - relevant specifications such as Scrub Key, reinsert, document-hygiene or domain-profile contracts.
66
 
67
+ Use the canonical files as follows:
68
 
69
+ - `PROJECT_PROMPT.md` — stable worker/governance/safety rules;
70
+ - `ROADMAP.md` — strategic product direction and stage order;
71
+ - `WORKPACKAGES.md` — one current executable queue and gates;
72
+ - `CHANGELOG.md` — historical implementation record;
73
+ - `RELEASE_NOTES.md` — user-facing release summary;
74
+ - `RISK_REGISTER.md` — current risk picture;
75
+ - `DECISION_LOG.md` — accepted current decisions.
 
76
 
77
+ If these files conflict with current source/runtime evidence, treat the conflict as a defect. Do not silently choose whichever statement is convenient.
78
 
79
  ---
80
 
81
+ ## 4. Current product direction
82
 
83
+ Core product workflow:
84
 
85
  ```text
86
+ Scrub → Review → Scrub Key → AI / external use → Reinsert → Export → Audit
 
87
  ```
88
 
89
+ Scrub is a reviewed privacy-processing / pseudonymisation product for confidential professional documents. Legal and Zorg are the priority validation domains. Human review remains required; the product must preserve legal and clinical meaning rather than blindly mask context.
90
 
91
+ ### Current strategic stage
92
 
93
+ ```text
94
+ Stage 1 — Repository Convergence
95
+ ```
96
 
97
+ Normal new feature development is paused until the repository has one verified clean current truth.
 
98
 
99
+ Current strategic sequence:
100
 
101
+ ```text
102
+ Repository Convergence
103
+ → Scrub Private Application
104
+ → Private Service
105
+ → External Product & Service Assurance
106
+ → Pilot
107
+ ```
108
 
109
+ After convergence, `main` becomes the active Scrub Private development line.
110
 
111
+ Hugging Face is a synthetic/approved-test **application validation environment**, not the final confidential-production trust environment.
 
 
 
 
 
112
 
113
+ Local/offline functionality and prior installer work remain recoverable/deferred. They are not the active delivery line and must not be resumed merely because historical plans mention them.
114
 
115
+ Do not build a new Evidence Framework. Reuse and reconcile the existing synthetic corpus, benchmark, gap-triage, E2E, Scrub Key, document and AppTest evidence systems.
 
 
 
 
 
 
 
 
 
 
 
 
116
 
117
  ---
118
 
119
+ ## 5. Implementation versus independent release assurance
120
 
121
+ Consequential work uses the canonical two-role model:
122
 
123
+ ```text
124
+ implementation_operations
125
+ governance_release_assurance
126
+ ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
127
 
128
+ The user remains the coordinator-facing principal.
129
 
130
+ ### Implementation role
131
 
132
+ `implementation_operations`:
133
 
134
+ - implements the workpackage;
135
+ - creates an identifiable exact candidate;
136
+ - adds/updates meaningful tests;
137
+ - records implementation evidence and handover;
138
+ - may only claim an implementation state such as `IMPLEMENTATION_IN_PROGRESS`, `IMPLEMENTATION_BLOCKED` or `RELEASE_CANDIDATE_READY`;
139
+ - may not certify its own consequential candidate.
140
 
141
+ ### Assurance role
 
 
 
142
 
143
+ `governance_release_assurance`:
144
 
145
+ - works independently from a fresh worker/session;
146
+ - reconstructs the requested outcome and exact candidate;
147
+ - may not silently repair the candidate;
148
+ - before its initial verdict may inspect authoritative criteria, exact source/diff and raw machine/deployment evidence, but not implementation handovers, self-assessments or conclusions;
149
+ - records exactly `PASS`, `FAIL` or `INDETERMINATE`;
150
+ - may inspect the implementation handover only after the initial verdict for disclosure/administrative completeness.
151
 
152
+ A repaired candidate receives a fresh independent assurance pass.
 
 
 
153
 
154
+ Consequential implementation and verification are separate workpackages/workers.
155
 
156
  ---
157
 
158
+ ## 6. Workpackage discipline
159
 
160
+ Work in small, testable, coherent root-cause packages.
161
 
162
+ For each workpackage:
163
 
164
+ 1. confirm title, role and scope;
165
+ 2. check dependencies/current queue in `WORKPACKAGES.md`;
166
+ 3. check `RISK_REGISTER.md` for affected product/privacy risks;
167
+ 4. check `DECISION_LOG.md` for relevant boundaries;
168
+ 5. reconstruct current source behavior before changing it;
169
+ 6. prefer pure helpers/tests before risky UI integration when new logic is genuinely required;
170
+ 7. add/update meaningful tests;
171
+ 8. update `CHANGELOG.md` for implementation history;
172
+ 9. update `RELEASE_NOTES.md` only for user-visible behavior changes;
173
+ 10. update `ROADMAP.md` only when strategy/stage order changes;
174
+ 11. update `WORKPACKAGES.md` when current execution status/next work changes;
175
+ 12. independently verify consequential candidates;
176
+ 13. end with a handover and write it under `handover/workpackages/`.
177
 
178
+ Do not create abstractions, services, configuration layers or compatibility mechanisms without a concrete current requirement.
179
 
180
+ During Repository Convergence, no cleanup/refactor is justified by aesthetics alone. Require evidence of duplicate/contradictory/dead behavior, privacy/security risk, obsolete compatibility, meaningful maintenance burden, runtime instability or another concrete current problem.
181
 
182
  ---
183
 
184
+ ## 7. Parallelization and shared-surface rule
185
 
186
+ Safe to parallelize only when changes are genuinely independent, such as:
187
 
188
+ - separate helper modules;
189
+ - independent test/documentation work;
190
+ - benchmark-data analysis;
191
+ - risk review;
192
+ - non-overlapping architecture analysis.
193
 
194
+ Do not make parallel uncoordinated edits to shared state/runtime/UI surfaces, especially:
 
 
 
 
 
 
 
 
 
 
 
195
 
196
+ - `presidio_streamlit.py`;
197
+ - Streamlit startup/patch behavior;
198
+ - review table/direct-correction flow;
199
+ - export/download flow;
200
+ - Scrub Key/reinsert shared paths;
201
+ - processing-generation state;
202
+ - Docker/runtime startup order.
203
 
204
+ When in doubt, keep shared-surface integration sequential.
 
 
205
 
206
+ ---
207
 
208
+ ## 8. Testing, deployment and monitoring
 
 
209
 
210
+ Implementation workers must self-check where connector permissions allow.
211
 
212
+ After meaningful implementation:
 
213
 
214
+ 1. verify relevant local/pure tests where execution is available;
215
+ 2. verify the full GitHub Actions suite for the exact candidate where required;
216
+ 3. inspect failed-job logs rather than inferring causes;
217
+ 4. after authorized merge, verify exact-main Actions;
218
+ 5. verify GitHub→Hugging Face synchronization when applicable;
219
+ 6. request human/live app verification only when UI/runtime behavior changed or subjective UX confirmation is required.
220
 
221
+ Use `STATUS_MONITORING_RUNBOOK.md` for the monitoring procedure.
222
 
223
+ Do not claim functional success merely because code appears correct.
224
 
225
+ For UI behavior, automated tests are necessary but may not replace required deployed/live verification.
226
 
227
+ ---
228
 
229
+ ## 9. Documentation discipline
230
 
231
+ `CHANGELOG.md` is historical implementation provenance. Do not rewrite history simply because old entries describe obsolete states.
232
 
233
+ `ROADMAP.md` must remain strategic and concise.
234
 
235
+ `WORKPACKAGES.md` must contain one real current executable queue. Do not stack multiple historical “current override” sections.
236
 
237
+ `DECISION_LOG.md` records accepted decisions; `RISK_REGISTER.md` records current risk; do not create competing permanent status documents.
238
 
239
+ Temporary audit/debt ledgers may exist during a workpackage, but they are execution evidence, not new canonical authorities.
 
 
240
 
241
+ ---
242
 
243
+ ## 10. Handover discipline
244
 
245
+ Every worker must write a handover to:
246
 
247
+ ```text
248
+ handover/workpackages/YYYYMMDD_HHMM_<workpackage_slug>.md
249
  ```
250
 
251
+ The handover must explicitly state:
252
+
253
+ - repository worked in;
254
+ - workpackage title;
255
+ - role;
256
+ - status;
257
+ - exact candidate/baseline identity where relevant;
258
+ - files added;
259
+ - files changed;
260
+ - tests added/updated;
261
+ - validation status;
262
+ - GitHub Actions status;
263
+ - Hugging Face sync status;
264
+ - app verification status;
265
+ - remaining risks/blockers;
266
+ - next recommended step.
267
+
268
+ Do not label a consequential package `completed` merely because implementation code exists. Distinguish implementation readiness, assurance result, merge/action and independently confirmed outcome.
269
+
270
  ---
271
 
272
+ ## 11. Safety and privacy rules
273
 
274
+ - Do not weaken human review or privacy controls.
275
+ - Do not silently change export semantics, filenames, MIME or Scrub Key semantics.
276
+ - Treat false negatives as product-critical.
277
+ - Treat the Scrub Key as sensitive re-identification material.
278
+ - Do not introduce external/cloud document processing unless explicitly approved for the relevant product line.
279
+ - Use synthetic data only for repository tests and public/reference-environment verification.
280
  - Do not store secrets, tokens or real personal data.
281
+ - Preserve legal and clinical meaning.
282
+ - Do not claim perfect anonymisation, perfect recall or production safety from synthetic benchmarks.
283
+ - Fail visibly/closed where binding/state validation requires it.
 
 
284
 
285
+ ### Scrub Private boundary
286
 
287
+ For the future Private line, customer document content includes uploaded/extracted text, detected values, replacement mappings, processed/restored documents and Scrub Keys.
288
 
289
+ Target policy:
290
 
291
+ - no intentional persistent server-side customer document content;
292
+ - no content-bearing ordinary application logs;
293
+ - no document-content backups;
294
+ - no third-party document-processing egress;
295
+ - minimal non-document control-plane metadata only where a real service requirement justifies persistence.
296
 
297
+ Production infrastructure/security claims must be separately verified; Hugging Face application tests do not prove provider-level retention properties.
298
 
299
+ ---
300
+
301
+ ## 12. Current execution rule
302
 
303
+ Follow `WORKPACKAGES.md`.
304
+
305
+ During Repository Convergence:
306
 
307
  ```text
308
+ Preserve → reconstruct → remove only proven debt → reconcile → verify → freeze
309
  ```
310
+
311
+ Do not resume historical Premium stage packages, installer work, VPS infrastructure, pilots or scale features unless the current roadmap/workpackage gate explicitly releases them.
PROJECT_PROMPT_SHORT.md CHANGED
@@ -12,77 +12,75 @@ Work only in:
12
  solidprivacy-nl/scrub
13
  ```
14
 
15
- GitHub is the source of truth. At the start of every work session, read:
 
 
16
 
17
  1. `PROJECT_PROMPT.md`
18
  2. `ROADMAP.md`
19
  3. `WORKPACKAGES.md`
20
  4. `CHANGELOG.md`
21
 
22
- Apply the project's implementation-versus-release-assurance separation and blind-review boundary for consequential work. Read:
 
 
23
 
24
  1. `control/PROJECT_GOVERNANCE_BOOTSTRAP.md`
25
  2. `control/SCRUB_RELEASE_ASSURANCE_CONTRACT_V1.md`
26
 
27
- Implementation may only prepare a release candidate. A separate `governance_release_assurance` worker/session independently reconstructs and verifies it without reading the implementation handover or conclusions before its initial `PASS`, `FAIL` or `INDETERMINATE`. Governance may not silently repair the candidate it reviews.
28
 
29
- Follow the current workpackage plan in `WORKPACKAGES.md`. Do not invent a new direction if it conflicts with the roadmap.
30
 
31
- Use small, testable workpackages. Prefer helper modules and tests before UI changes. Avoid parallel edits to the same UI flow, especially:
32
 
33
- - `presidio_streamlit.py`
34
- - `fix_streamlit_nested_expanders.py`
35
- - review table flow
36
- - export/download flow
 
 
 
37
 
38
- After meaningful changes:
39
 
40
- - update `CHANGELOG.md`;
41
- - update `ROADMAP.md` only when strategy or phase order changes;
42
- - update `WORKPACKAGES.md` when execution status or next workpackages change;
43
- - verify GitHub Actions tests;
44
- - verify GitHub to Hugging Face sync;
45
- - ask for app verification when UI behavior changed.
46
 
47
- Every worker must end with a handover summary and also write it to:
48
 
49
- ```text
50
- handover/workpackages/
51
- ```
52
 
53
- Use filename format:
 
 
 
 
 
 
 
54
 
55
  ```text
56
  handover/workpackages/YYYYMMDD_HHMM_<workpackage_slug>.md
57
  ```
58
 
59
- The handover must explicitly state:
60
-
61
- - repository worked in;
62
- - workpackage title;
63
- - status;
64
- - files added/changed;
65
- - tests;
66
- - validation status;
67
- - GitHub Actions status;
68
- - Hugging Face sync status;
69
- - app verification status;
70
- - remaining risks;
71
- - next recommended step.
72
 
73
- Safety rules:
74
 
75
- - Do not weaken privacy or review controls.
76
- - Do not silently change export semantics.
77
- - Do not introduce cloud document processing unless explicitly approved.
78
- - Do not store secrets, tokens, or real personal data.
79
- - Use synthetic data only.
80
- - Preserve legal context: replace sensitive values, not legal meaning.
 
 
 
81
 
82
- Product direction:
83
 
84
  ```text
85
- Scrub → Review → Scrub Key → AI → Reinsert → Export → Audit
86
  ```
87
-
88
- Immediate line: v12 Review UX, then v13 Scrub Key / Reinsert.
 
12
  solidprivacy-nl/scrub
13
  ```
14
 
15
+ Use `solidprivacy-nl/solidprivacy` only for explicitly shared SolidPrivacy privacy/AI workflows/components when the current workpackage calls for it. Do not modify unrelated repositories.
16
+
17
+ GitHub is the source of truth. At the start of every work session, read in order:
18
 
19
  1. `PROJECT_PROMPT.md`
20
  2. `ROADMAP.md`
21
  3. `WORKPACKAGES.md`
22
  4. `CHANGELOG.md`
23
 
24
+ Follow the SolidPrivacy Execution & Engineering Constitution: business outcome first, smallest complete/proven solution, solid but simple, no overengineering, one source of truth, verify behavior, remove obsolete/conflicting active paths, and do not call work Done until implementation, verification, cleanup and documentation agree.
25
+
26
+ For consequential work, also read:
27
 
28
  1. `control/PROJECT_GOVERNANCE_BOOTSTRAP.md`
29
  2. `control/SCRUB_RELEASE_ASSURANCE_CONTRACT_V1.md`
30
 
31
+ Implementation may only prepare an exact release candidate. A separate fresh `governance_release_assurance` worker/session independently reconstructs and verifies it before reading implementation conclusions, then records `PASS`, `FAIL` or `INDETERMINATE`. Governance may not silently repair the candidate.
32
 
33
+ Follow the one current execution queue in `WORKPACKAGES.md`. Do not execute historical queues merely because old documentation/issues mention them.
34
 
35
+ Current strategic sequence:
36
 
37
+ ```text
38
+ Repository Convergence
39
+ → Scrub Private Application
40
+ → Private Service
41
+ → External Product & Service Assurance
42
+ → Pilot
43
+ ```
44
 
45
+ Normal new feature work is paused during Repository Convergence. Preserve current capability, remove only proven debt, align current truth, verify, and freeze one exact clean-baseline SHA.
46
 
47
+ Do not build a new Evidence Framework. Reuse/reconcile the existing synthetic corpora, benchmarks, E2E, Scrub Key, document and AppTest evidence.
 
 
 
 
 
48
 
49
+ Hugging Face is a synthetic/approved-test application-validation surface, not the final confidential-production trust environment. Local/offline functionality is preserved/deferred, not the active delivery line.
50
 
51
+ Use small coherent workpackages. Do not refactor for aesthetics. Avoid parallel edits to shared UI/state/runtime surfaces, especially `presidio_streamlit.py`, review, export, Scrub Key/reinsert, processing state and Docker/startup behavior.
52
+
53
+ After meaningful changes:
54
 
55
+ - update `CHANGELOG.md`;
56
+ - update `ROADMAP.md` only for strategy/stage changes;
57
+ - update `WORKPACKAGES.md` when current execution changes;
58
+ - verify GitHub Actions;
59
+ - verify GitHub→Hugging Face sync when applicable;
60
+ - request live app verification when visible/runtime behavior changed.
61
+
62
+ Every worker must write:
63
 
64
  ```text
65
  handover/workpackages/YYYYMMDD_HHMM_<workpackage_slug>.md
66
  ```
67
 
68
+ including repository, workpackage, role/status, exact candidate where relevant, files, tests, validation, Actions, HF sync, app verification, risks/blockers and next step.
 
 
 
 
 
 
 
 
 
 
 
 
69
 
70
+ Safety:
71
 
72
+ - do not weaken privacy or human-review controls;
73
+ - do not silently change export or Scrub Key semantics;
74
+ - false negatives are product-critical;
75
+ - Scrub Keys are sensitive re-identification material;
76
+ - use synthetic data only;
77
+ - do not store secrets/tokens/real personal data;
78
+ - preserve legal and clinical meaning;
79
+ - do not introduce external/cloud document processing unless explicitly approved for the relevant product line;
80
+ - do not claim perfect anonymisation/recall or production safety from synthetic tests.
81
 
82
+ Core workflow:
83
 
84
  ```text
85
+ Scrub → Review → Scrub Key → AI / external use → Reinsert → Export → Audit
86
  ```
 
 
REPOSITORY_CONVERGENCE_DEBT_LEDGER.md ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SolidPrivacy Scrub — Repository Convergence Debt Ledger
2
+
3
+ > **TEMPORARY EXECUTION ARTIFACT — NON-AUTHORITATIVE AFTER CONVERGENCE**
4
+ >
5
+ > This ledger supports `SCRUB-WP_REPOSITORY_CONVERGENCE_BOOTSTRAP` and later convergence work. It is not a new permanent source of product truth. At convergence closeout, accepted decisions belong in `DECISION_LOG.md`, current risks in `RISK_REGISTER.md`, execution state in `WORKPACKAGES.md`, and implementation history in `CHANGELOG.md`.
6
+
7
+ Date: 2026-09-04
8
+ Repository: `solidprivacy-nl/scrub`
9
+ Pre-convergence baseline: `54c73e0ebf5a3a3ed7039a50596fb57694add3cd`
10
+
11
+ ## Classification model
12
+
13
+ - `CANONICAL` — supported current behavior; retain, test and document accurately.
14
+ - `RECONCILE` — valid responsibility whose implementation, tests, configuration, documentation or status does not match current reality.
15
+ - `RETIRE` — superseded, dead, contradictory or unnecessarily active path; remove only after evidence shows it is safe to do so.
16
+ - `VARIANT-SPECIFIC` — potentially valid functionality whose requirements conflict with Scrub Private; preserve recoverability through Git but exclude from the Private line unless explicitly re-approved.
17
+
18
+ Classification applies to capabilities/contracts/runtime paths, not mechanically to whole files.
19
+
20
+ ## Current capability/path ledger
21
+
22
+ Rows describing documentary debt explicitly distinguish the pre-bootstrap finding from the state already resolved in this bootstrap candidate. `RECONCILE — RESOLVED IN BOOTSTRAP CANDIDATE` means no further bootstrap cleanup is authorized for that item unless fresh assurance finds a concrete defect. Unresolved technical, issue-state and validation-hierarchy findings remain future work.
23
+
24
+ | Capability / path | Current role and evidence | Classification | Target | Protecting evidence / tests | Risk | Smallest next action |
25
+ |---|---|---|---|---|---|---|
26
+ | Premium staged workspace and generation-bound state (`premium_core_flow_state.py`, Premium Streamlit helpers, current `presidio_streamlit.py`) | Current main contains `Anonimiseren | Terugzetten`, `Standaard | Expert`, staged `Toevoegen → Controleren → Downloaden`, generation-bound analysis/review state and fail-closed export eligibility. | `CANONICAL` | Retain as current UI/state architecture. | Premium state/AppTest/integration suites. | Rebuilding from stale workpackages could reintroduce already-fixed state defects. | Preserve; do not restart old Input/Review/Export packages without current evidence. |
27
+ | Human review authority, review table and direct correction | Review/include state remains authoritative; direct masking/manual additions feed normal replacement rows rather than bypassing review/export authority. | `CANONICAL` | Retain mandatory human review and authoritative review state. | Review/direct-selection/manual-row/cross-flow tests. | Privacy control could be weakened by UI simplification or alternate mutation paths. | Preserve. |
28
+ | Scrub Key binding and reinsert | Current architecture has bound placeholders, schema-1.1 keys, mapping digest, fail-closed wrong/mixed binding checks and controlled reinsert with explicit legacy handling. | `CANONICAL` | Retain; no redesign without evidence of a concrete flaw. | Scrub Key binding, import/export, TXT/DOCX/PDF-to-TXT reinsert and roundtrip suites. | Re-identification material is safety-critical. | Preserve. |
29
+ | Legal recognition / Dutch deterministic recognition | Current runtime uses deterministic Dutch recognizers plus generic NER and Legal profile behavior. Latest main includes Dutch address-span precision repair. | `CANONICAL` | Retain current behavior; future fixes remain evidence-driven. | Legal/custom-recognizer, recall, address-span and cross-profile tests. | False negatives remain product-critical. | Preserve and verify through canonical validation path. |
30
+ | Zorg profile, policy and recognizers | Dedicated Zorg policy/recognizers, synthetic corpus, baseline, gap triage and cross-profile evidence exist and are integrated. | `CANONICAL` | Retain; preserve clinical meaning and human review. | Care policy/corpus/recognizer/cross-profile/long-form tests. | Under-detection or clinical over-masking remains critical. | Preserve. |
31
+ | TXT/DOCX/text-PDF document handling and DOCX hygiene/fidelity | Supported import/export/reinsert/hygiene paths exist; PDF remains text-based with explicit limitations. | `CANONICAL` | Retain supported boundaries; do not add OCR/restored-PDF merely during cleanup. | Document tools, hygiene, Phase-6 E2E and fidelity tests. | Hidden content and unsupported-format assumptions. | Preserve. |
32
+ | Recognizer-backed recall benchmark (`recall_benchmark_runner.py`, `recall_benchmark_report.py`, `.github/workflows/recall-benchmark-report.yml`, `corpus/**`) | Current workflow executes corpus/runner/report tests and runs the actual recognition stack to produce JSON/Markdown diagnostic artifacts. | `CANONICAL` candidate | Treat as primary current recognizer-backed diagnostic release evidence unless later audit finds a newer equivalent. | Recall benchmark corpus/runner/report tests and workflow. | Competing benchmark generations can confuse release evidence. | Document release role and compare older scorecard/residual-risk helpers before de-authorizing anything. |
33
+ | Older entity-scorecard / residual-risk helper generation (`benchmark/build_entity_scorecard.py`, `benchmark/build_residual_risk_report.py`) | Useful report-only/supplied-prediction diagnostics explicitly state coverage limits and no production gate. | `RECONCILE` | Keep as supplemental diagnostic if still useful; do not present as equivalent to recognizer-backed corpus execution. | Scorecard/residual-risk tests. | Different generations may appear to be competing sources of accuracy truth. | Classify canonical vs supplemental in a narrow validation-hierarchy package; no new framework. |
34
+ | Phase-6 synthetic E2E validation | Exercises TXT/DOCX/PDF workflow, manual additions, Scrub Key, export/reinsert and known limitations. | `CANONICAL` | Retain as workflow-level product evidence complementary to recognizer benchmarks. | `tests/test_mvp_phase6_*` and committed validation artifacts. | Could be mistaken for complete production readiness. | Preserve limitations and human-review boundary. |
35
+ | Persistent replacement memory (`replacement_memory.py` + Expert UI) | Current code deliberately persists original/replacement/entity values to `/data/replacement_memory.json` when persistent storage exists and exposes save/load/clear controls. | `VARIANT-SPECIFIC` | Preserve recoverability for possible Local use; Scrub Private must not persist these mappings server-side. | Existing replacement-memory tests plus future Private no-persistence contract. | Direct conflict with Private content-plane retention boundary. | Do not change in bootstrap. After clean baseline, remove/exclude from Private in a separate consequential package. |
36
+ | Azure AI Language document recognition | Current Expert analyzer list still includes Azure AI Language and current helper can construct Azure recognizer clients. | `VARIANT-SPECIFIC` | Exclude from Scrub Private baseline; Private document processing stays local. | Current analyzer-selection tests; future Private no-egress tests. | Third-party document-content egress conflicts with intended service trust boundary. | Do not change in bootstrap. Remove/exclude after clean baseline. |
37
+ | OpenAI/Azure OpenAI synthesis operator | Current Expert operator includes `synthesize`; helper sends document-bearing prompt to external provider. | `VARIANT-SPECIFIC` | Exclude from Scrub Private baseline unless explicitly approved later. | Current operator/state tests; future Private no-egress tests. | External content processing and unnecessary trust-boundary complexity. | Do not change in bootstrap. Remove/exclude after clean baseline. |
38
+ | Content-bearing prompt print in `create_fake_data()` | Current helper prints the generated synthesis prompt before external call. | `RECONCILE` for current full-feature line; Private blocker | No document text/prompts in ordinary application logs. | Future narrow no-content-log regression. | Sensitive content can enter logs. | Remove with Private external-synthesis cleanup; do not build a logging framework. |
39
+ | Legacy startup patch script invocation in Dockerfile | Docker CMD still runs `fix_streamlit_nested_expanders.py` and `fix_streamlit_pdf_text_reinsert.py`. Current Premium/direct-source markers make both scripts exit without mutating source. | `RETIRE` candidate | Production/current runtime should not invoke obsolete no-op source mutation machinery. | Patch-script contract tests, Premium source markers, full suite and runtime startup tests. | Hidden startup complexity and future accidental mutation. | Prove both current no-op paths are unnecessary, then remove invocation/scripts in isolated package. |
40
+ | `fix_streamlit_nested_expanders.py` historical mutation implementation | Contains large historical patch logic but exits immediately on current Premium source. | `RETIRE` candidate | Remove after startup-path proof; Git history retains implementation. | Existing patch tests and Premium regression suites. | Dead complex code can be accidentally reactivated or mistaken for current UI authority. | Isolated technical package after bootstrap. |
41
+ | `fix_streamlit_pdf_text_reinsert.py` historical mutation implementation | Current direct-source marker makes script exit without mutation. | `RETIRE` candidate | Remove after startup-path proof. | Reinsert direct-source/patch tests. | Dead startup code and duplicate authority. | Isolated technical package after bootstrap. |
42
+ | Docker/runtime production hardening | Current Dockerfile uses Python 3.10 base, build tools, legacy startup patch calls and permissive Streamlit XSRF/CORS flags. This is not itself evidence of a current HF product defect. | `RECONCILE` / later service work | Keep convergence scoped; production immutability, least privilege, egress and secure server config belong primarily to Private Service stage. | Runtime/startup tests and later service security evidence. | Premature hardening work can derail functional convergence; leaving insecure production config later would be unacceptable. | Record only; defer major hardening until Private Service unless a current safety defect is demonstrated. |
43
+ | `openai` and `azure-ai-textanalytics` dependencies | Required only by current variant-specific external processing paths. | `RECONCILE` | Remove when the corresponding Private-incompatible runtime paths are removed; do not leave unused dependencies. | Full dependency/import/runtime suite. | Unused attack/maintenance surface. | Defer to the same Private cleanup package as external processing. |
44
+ | `PROJECT_PROMPT.md` product-direction section | Pre-bootstrap finding: product direction still encoded local-first sequencing. Current bootstrap candidate now preserves stable governance/safety rules while routing execution through Repository Convergence → Scrub Private Application → Private Service → External Assurance → Pilot; HF is synthetic/approved-test validation and Local is deferred. | `RECONCILE — RESOLVED IN BOOTSTRAP CANDIDATE` | Adopt the candidate wording on independent PASS/merge. | Repository-convergence contract tests plus direct document review. | Future drift could again route workers to obsolete strategy. | No further bootstrap action; revisit only if later convergence work changes the strategy. |
45
+ | `AGENTS.md` product-direction section | Pre-bootstrap finding: agent routing still encoded the local-first path. Current bootstrap candidate aligns claim/governance discipline with Repository Convergence and preserves the two-role assurance model. | `RECONCILE — RESOLVED IN BOOTSTRAP CANDIDATE` | Keep agent routing aligned with the canonical roadmap/workpackage queue. | Repository-convergence contract tests plus direct document review. | Agent workers could otherwise restart superseded packages. | No further bootstrap action. |
46
+ | `ROADMAP.md` | Pre-bootstrap finding: strategy, long implementation history and old nine-phase/local-installer path were mixed. Current bootstrap candidate contains the concise five-stage roadmap with Stage 1 Repository Convergence as `CURRENT`. | `RECONCILE — RESOLVED IN BOOTSTRAP CANDIDATE` | On PASS/merge, keep ROADMAP strategic and move detailed execution to WORKPACKAGES/CHANGELOG. | Formal-stage parsing contract and direct roadmap review. | Reintroducing implementation history as current strategy would recreate source-of-truth drift. | No further bootstrap action; Pass B only if later technical convergence changes current truth. |
47
+ | `WORKPACKAGES.md` | Pre-bootstrap finding: multiple stacked `Current execution status override` sections and superseded Premium queues coexisted. Current bootstrap candidate explicitly contains the current executable queue only, with Repository Convergence active and later stages blocked by gates. | `RECONCILE — RESOLVED IN BOOTSTRAP CANDIDATE` | On PASS/merge, maintain one current executable queue. Historical work remains in CHANGELOG/handovers/Git. | Repository-convergence queue contract and direct workpackage review. | Stacked historical queues can misroute future workers. | No further bootstrap action; derive later technical packages only from current evidence. |
48
+ | `CHANGELOG.md` | Pre-bootstrap finding: the root changelog mixed a large historical record with language that could be misread as current state. Current bootstrap candidate starts a post-convergence implementation log and points to `history/CHANGELOG_PRE_CONVERGENCE_20260904.md`; that archive is byte-identical to the baseline changelog blob. | `RECONCILE — RESOLVED IN BOOTSTRAP CANDIDATE` | Keep root CHANGELOG as implementation history from the reset onward while preserving the archived pre-convergence provenance. | Archive blob identity plus convergence changelog contract. | Losing provenance or treating historical candidate status as current authority would both be defects. | No further bootstrap action. |
49
+ | `DECISION_LOG.md` | Pre-bootstrap finding: the strategic pivot was not yet recorded. Current bootstrap candidate contains D044 and retains concise still-binding Premium, review, Zorg, Scrub Key, reinsert and DOCX decisions. | `RECONCILE — RESOLVED IN BOOTSTRAP CANDIDATE` | On PASS/merge, keep only materially binding current/future decisions here; use Git/CHANGELOG for history. | D044/still-binding-decision contracts plus direct review. | Missing or duplicate decisions can create competing architecture authority. | No further bootstrap action unless a later accepted decision supersedes current direction. |
50
+ | `RISK_REGISTER.md` | Pre-bootstrap finding: historical execution narrative obscured live risk and R5 assumed local-first trust. Current bootstrap candidate preserves critical R1/R2/R10 and aligns R5 to Scrub Private retention/egress, with explicit source-of-truth and startup-mutation risks. | `RECONCILE — RESOLVED IN BOOTSTRAP CANDIDATE` | Keep current risks/mitigations visible without turning the register into an execution history. | Risk continuity/current-truth contracts plus direct review. | Stale risk framing can weaken or misroute safety work. | No further bootstrap action; update only as risk reality changes. |
51
+ | Open GitHub issues #112/#109/#107/#106 and earlier Premium/governance cycle issues | Several describe candidates that have since PASSed/merged; open state does not match main. #105/#96 additionally require a final consolidated deployed live retest after their child repairs. | `RECONCILE` | Reconstruct each against current main; close stale completed issues, preserve/consolidate any still-unverified live gate. | Raw PR/Actions/assurance comments + current runtime evidence. | Issue state can misroute future work. | Reconcile after bootstrap candidate is accepted; do not falsely claim unperformed live verification. |
52
+ | Hugging Face role in project docs | Pre-bootstrap finding: HF was described inconsistently against the prior local-first direction. Current bootstrap candidate consistently defines HF as a synthetic/approved-test application-validation surface, not confidential-production infrastructure assurance. | `RECONCILE — RESOLVED IN BOOTSTRAP CANDIDATE` | Preserve this bounded claim unless the deployment strategy changes. | Repository-convergence contracts plus current project-doc review. | Overclaiming provider-level retention/security would create false assurance. | No further bootstrap action. |
53
+ | Local/offline installer programme | Valuable prior planning remains recoverable, but current bootstrap candidate explicitly demotes Local/offline packaging from the active delivery line and records it as a possible future product variant only if business evidence reopens it. | `VARIANT-SPECIFIC — DEFERRED / BOOTSTRAP ROUTING RESOLVED` | Preserve history; do not execute installer work unless a future accepted requirement reopens it. | D038/D044 plus roadmap/workpackage contracts. | Maintaining two active delivery lines would create distraction and duplicate architecture. | No further bootstrap action. |
54
+
55
+ ## Current issue-reconciliation notes
56
+
57
+ These notes are findings, not closure authority by themselves.
58
+
59
+ - #112: independent assurance for PR #111 recorded PASS; PR #111 is merged into current main; exact-main Tests and HF sync are green. Issue is stale as an active candidate gate.
60
+ - #109: independent assurance for PR #108 recorded PASS; PR #108 is merged. Issue is stale as an active candidate gate.
61
+ - #107/#106: implementation issues whose corresponding repairs are merged; active issue state requires reconciliation.
62
+ - #105/#96: their original App Shell state defect was repaired, but live verification later exposed marker/address defects. Those child defects were subsequently repaired by PR #108/#111. Evidence found so far does **not** prove the final consolidated deployed live retest required by the parent issues was completed. Do not close that residual gate as verified without evidence.
63
+ - Earlier #74–#100 Premium/governance repair/assurance issues are historical candidate cycles and should be checked for stale open state, not treated as current implementation direction.
64
+
65
+ ## Current validation-hierarchy hypothesis
66
+
67
+ Subject to a narrow follow-up verification:
68
+
69
+ ### Candidate canonical release/product evidence
70
+
71
+ - full `python -m pytest -q tests` regression;
72
+ - recognizer-backed `recall_benchmark_runner.py` / `recall_benchmark_report.py` corpus workflow;
73
+ - Zorg baseline/recognizer/cross-profile evidence where domain-specific;
74
+ - Phase-6 synthetic E2E workflow validation;
75
+ - Scrub Key security/roundtrip tests;
76
+ - document hygiene/fidelity tests;
77
+ - Premium AppTest/state tests.
78
+
79
+ ### Supplemental diagnostic evidence
80
+
81
+ - entity scorecard and older report-only residual-risk builders when used within their explicit coverage limits.
82
+
83
+ ### Explicit non-claim
84
+
85
+ None of the above alone proves production safety or removes the mandatory human-review requirement.
86
+
87
+ ## Bootstrap boundary
88
+
89
+ This ledger authorizes no runtime change by itself. The bootstrap package may only reset current strategy/execution documentation and contract tests. Technical changes must be split into evidence-backed consequential workpackages and receive normal independent assurance.
RISK_REGISTER.md CHANGED
@@ -1,267 +1,396 @@
1
- # SolidPrivacy Scrub — Risk Register
2
 
3
- This register tracks product, privacy, security and trust risks.
4
 
5
- Status values: `open`, `mitigating`, `accepted`, `closed`.
6
 
 
 
 
 
 
 
 
7
  Impact values: `critical`, `high`, `medium`, `low`.
8
 
9
  ---
10
 
11
  ## R1 — False negatives / missed sensitive data
12
 
13
- Status: mitigating
14
- Impact: critical
15
 
16
  Risk:
17
 
18
  ```text
19
- Sensitive data remains in the scrubbed output and the user wrongly trusts the result.
20
  ```
21
 
22
- Current mitigations include human review, review guidance, diagnostic recall benchmark artifacts, PERSON-name diagnostic/contract/helper work, planning-only threshold policy and a verified simple manual missed-value entry that adds user-supplied values to the existing replacement table. A selection-driven processed-text correction path is approved with an all-exact version-one boundary. Its frozen contract, action model, local component and production integration enforce a two-stage server-authoritative inspect/commit protocol. Accepted commits append one normal document-scoped row, rerun before export and remain visible in the authoritative review table. The static renderer and manual form remain fallbacks. Deployment synchronization and live browser verification are green, including selection, exact-occurrence masking, normal table-row creation and one-step undo. Phase 6 now starts with a synthetic end-to-end validation matrix so new fixes are driven by reproducible false-negative, misclassification and over-masking evidence. The first machine-readable matrix baseline is stored in `output/validation/mvp_phase6_synthetic_validation_report.json` with 3 synthetic cases and 2 recorded evidence gaps or known limitations. Triage confirms that neither remaining item is a detection false negative, so no recognizer fix is justified by this bounded baseline.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
23
 
24
- Remaining gaps:
25
 
26
- - No accepted production recall/precision threshold exists.
27
- - No production benchmark gate exists.
28
- - No production safety claim is supported.
29
- - Human review remains necessary.
30
 
31
  ---
32
 
33
- ## R2 — Scrub Key leakage or accidental sharing
34
 
35
- Status: mitigating
36
- Impact: critical
37
 
38
  Risk:
39
 
40
  ```text
41
- The Scrub Key is shared, leaked, retained too long, tampered with or mishandled, allowing full re-identification of scrubbed content.
42
  ```
43
 
44
- Current mitigations include warnings and acknowledgements. Export/download UX grouping is implemented directly in `presidio_streamlit.py` so the key file is visually separated from normal document exports and shown with a specific warning. Live app verification confirmed the grouped export UI. Reinsert now keeps the key-sensitivity warning visible, automatically applies the existing structural validation to a supplied key, and preserves one explicit confidentiality acknowledgement at the restored-output download boundary. Redundant pre-processing acknowledgements are removed because they obscured workflow state without adding key validation.
45
 
46
- The manual missed-value entry flows through the existing replacement table and existing Scrub Key/export paths without changing key semantics. No key storage, schema or lifecycle behavior is added by the automatic reinsert flow. The Phase 6 adversarial roundtrip matrix exposes a critical unresolved gap: a structurally valid wrong or tampered key that reuses the same placeholder namespace can restore incorrect original values without a detectable mismatch. This is routed to `SCRUB-WP_MVP_SCRUB_KEY_DOCUMENT_BINDING_GAP_TRIAGE`; no schema or export change is authorized by the validation package. Triage recommends a non-sensitive document binding ID inside every automatic/manual placeholder and the corresponding key, plus a canonical mapping digest for accidental corruption. Contract tests now freeze legacy v1.0 unbound behavior, a document-specific base32 binding ID, canonical SHA-256 mapping digest and fail-closed mismatch/mixed-ID/digest-error statuses before model implementation. The pure model implements those contracts, and anonymization/export now creates bound placeholders plus schema-1.1 keys when all selected mappings are bound. Arbitrary custom replacement text is preserved but visibly blocks verified key export. Reinsert now validates the complete supported document text surface before replacement. Correct bound matches are verified; wrong, mixed, missing or digest-invalid bindings fail closed with zero replacements, and DOCX failures return the exact original package bytes. Legacy v1.0 compatibility remains explicitly unverified. The accidental pairing/corruption path is technically mitigated but the risk remains open until deployed app verification passes. Signatures/HMAC remain deferred until protected local signing-key management exists. Display-only placeholder compaction does not shorten the binding ID, alter the mapping digest or change any key/export/reinsert token; full-token roundtrip verification remains required after deployment.
 
 
 
 
 
 
 
 
 
 
 
 
 
47
 
48
  ---
49
 
50
- ## R3 — Placeholder corruption during AI roundtrip
51
 
52
- Status: mitigating
53
- Impact: high
54
 
55
  Risk:
56
 
57
  ```text
58
- An AI system rewrites, translates, merges or deletes placeholders, causing deterministic reinsert to fail or restore incompletely.
59
  ```
60
 
61
- Current mitigations include placeholder robustness helper/test work and reinsert audit reporting. The Phase 6 roundtrip matrix validates translated, merged, unknown, repeated and malformed mutations. Unknown grammar-valid placeholders are visible, while malformed tokens outside the strict grammar are signalled indirectly through expected placeholders not found. This diagnostic limitation is included in the document-binding gap triage. It is not part of the critical binding implementation line; an optional later diagnostic-hardening package may report malformed near-placeholders directly without guessing or repairing values.
 
 
 
 
 
 
 
 
 
 
 
62
 
63
  ---
64
 
65
- ## R4 — Hidden document content and metadata leakage
66
 
67
- Status: mitigating
68
- Impact: high
69
 
70
  Risk:
71
 
72
  ```text
73
- DOCX metadata, comments, tracked changes, headers, footers or hidden content contain sensitive data that is not scrubbed or cleaned.
74
  ```
75
 
76
- DOCX hygiene audit remains report-only. Export grouping keeps audit details available and does not imply a clean-DOCX guarantee. The review debug collapse line explicitly keeps audit details available rather than removing them. The Phase 6 synthetic DOCX case now records header/footer findings and the existing main-document-only reinsert boundary as reproducible evidence for the document-hygiene hardening package. Gap triage classifies this as document fidelity and reinsert scope and routes it to `SCRUB-WP_MVP_DOCUMENT_HYGIENE_FIDELITY_HARDENING`. The hardening package now restores placeholders in existing DOCX header and footer XML parts while retaining hygiene reporting and explicit unsupported-part warnings.
 
 
 
 
 
 
 
 
 
 
 
77
 
78
  ---
79
 
80
- ## R5 — Cloud-demo trust gap and deferred installer risk
81
 
82
- Status: mitigating
83
- Impact: high
84
 
85
  Risk:
86
 
87
  ```text
88
- The final product promise is local-first, but the current fast validation surface is the Hugging Face cloud demo.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
89
  ```
90
 
91
- Local/offline installer work remains later. The current focus is making the web prototype workflow credible first.
 
 
 
 
 
 
 
92
 
93
  ---
94
 
95
- ## R6 — Review UX and interface clarity risk
96
 
97
- Status: mitigating
98
- Impact: high
99
 
100
  Risk:
101
 
102
  ```text
103
- The interface still feels like a technical prototype, which can reduce confidence and increase review mistakes.
104
  ```
105
 
106
- Current mitigations:
107
-
108
- - Review table remains source of truth and fallback.
109
- - Side-by-side review surface, synced scrolling and collapsible review table are live and verified.
110
- - Export/download UX is directly implemented in `presidio_streamlit.py` and live verified.
111
- - `REVIEW_DEBUG_ELEMENTS_COLLAPSE_PLAN.md` narrows review UI cleanup to a small interface pass, not a new review loop.
112
- - `WP_REVIEW_DEBUG_ELEMENTS_COLLAPSE_IMPLEMENTATION` made the existing step-by-step review aid collapsed by default and removed debug/governance wording from the primary UI.
113
- - `WP_MVP_FAST_MANUAL_MASK_ENTRY` adds a verified simple user-facing path to add missed values to the existing replacement table.
114
- - The contract, action model and component spike are now connected through `SCRUB-WP_PROCESSED_TEXT_SELECTION_TABLE_INTEGRATION`. The review table remains source of truth; the component is only an input route into normal bound manual rows.
115
- - Live app evidence on 2026-08-05 confirms that local decluttering is not sufficient: the application still exposes input, settings, review, downloads, Scrub Key and audit as one long form. `PREMIUM_CORE_FLOW_UI_REALIGNMENT_PLAN.md` mitigates this through a global Standard/Expert model, top-level workflow navigation, one active stage and progressive disclosure.
116
 
117
- Gaps:
 
 
 
 
 
 
 
 
118
 
119
- - Live reinsert verification exposed a concrete workflow-state problem: uploaded source and key files still required non-obvious follow-up checkboxes and buttons. `SCRUB-WP_MVP_REINSERT_AUTO_FLOW_SIMPLIFICATION_IMPLEMENTATION` addresses this narrowly with document-first ordering and automatic validation/processing.
120
- - Additional copy polish may still be needed, but it should remain separate and small.
121
- - Production table integration preserves collision/replay/stale guards, hidden-marker protection, immediate rerun and edit-aware undo. Deployment synchronization and live browser/app verification are green. Display-only placeholder compaction is implemented with lossless source tokens, exact UTF-16 mapping and protected compact spans; the underlying 80-bit binding remains unchanged. Focused deployment/app verification of the compact view is required before the remaining high-risk export/Scrub Key/reinsert cross-flow regression.
122
- - Implementation must avoid weakening review controls or hiding audit details; one final confidentiality acknowledgement remains at download.
123
 
124
- Recommended workpackages:
125
-
126
- - The general UI baseline is completed and app-verified. The automatic reinsert-flow package is permitted as a narrow exception because live Phase 6 validation exposed a concrete workflow blocker; broader UI work remains gated.
127
 
128
  ---
129
 
130
  ## R7 — PDF limitations misunderstood by users
131
 
132
- Status: mitigating
133
- Impact: high
134
 
135
  Risk:
136
 
137
  ```text
138
- Users may assume PDF support means complete restored PDF reinsert or OCR, while the approved scope is text-based extraction to restored TXT only.
139
  ```
140
 
141
- PDF limitations must remain clear in export/reinsert copy. The Phase 6 text-based PDF case verifies the current restored-TXT-only path and explicitly records that restored PDF and OCR are unsupported. Gap triage retains this as an explicit product boundary and does not authorize OCR or restored-PDF work. Document-fidelity hardening preserves that boundary unchanged.
 
 
 
 
 
 
 
142
 
143
  ---
144
 
145
- ## R8 — Workflow status, audit visibility and evidence clarity
146
 
147
- Status: mitigating
148
- Impact: medium
149
 
150
  Risk:
151
 
152
  ```text
153
- Evidence and audit controls become either too hidden to trust or too technical for normal users.
154
  ```
155
 
156
- Current mitigations:
157
 
158
- - Coordinator screenshots/evidence are recorded when connector lookup is incomplete.
159
- - Diagnostic recall benchmark artifact workflow exists.
160
- - Audit/report details exist.
161
- - `MVP_UI_CLEANUP_AND_EXPORT_REDESIGN_PLAN.md` states that technical/audit details must remain available but become secondary.
162
- - Export/download UX now groups document downloads, key file, and audit/technical files while keeping audit details available.
163
- - The step-by-step review aid is now secondary by default, while review table and audit controls remain available.
164
- - The verified manual missed-value entry is intentionally placed in the primary review path because it directly supports faster anonymization.
165
- - The approved selection-driven route may reduce navigation friction, but every accepted action must still become a visible normal replacement-table row and must not mutate export or Scrub Key state directly from the browser component.
166
 
167
- Gaps:
168
 
169
- - No generalized automated status artifact exists yet. `SCRUB-WP_MVP_AUDIT_RESIDUAL_RISK_EVIDENCE` is scheduled after the synthetic validation, gap-triage and roundtrip packages.
 
 
 
 
 
 
 
 
 
 
 
 
 
170
 
171
  ---
172
 
173
- ## R9 — Dutch legal reference under-detection and role over-masking
174
 
175
- Status: mitigating
176
- Impact: high
177
 
178
  Risk:
179
 
180
  ```text
181
- Dutch legal matter references can be missed or misclassified, while generic legal/care role words can be masked in ways that damage meaning.
182
  ```
183
 
184
- Current mitigations include diagnostic benchmark work, preservation guidance, PERSON-name contract/helper work and a verified manual missed-value entry path. The Phase 6 synthetic matrix is now the evidence source for `SCRUB-WP_MVP_FALSE_NEGATIVE_GAP_TRIAGE`; only reproducible under-detection, misclassification or role-over-masking findings may open a subsequent fix package. The first triage found none of those categories; recognizer behavior remains unchanged.
 
 
 
 
 
 
 
 
 
185
 
186
  ---
187
 
188
- ## R10 — Care-profile under-detection and clinical over-masking
189
 
190
- Status: mitigating
191
- Impact: critical
192
 
193
  Risk:
194
 
195
  ```text
196
- A care document retains patient or trajectory identifiers, or Scrub removes diagnosis, medication, laboratory values, observations or care context and makes the document misleading or unusable.
197
  ```
198
 
199
- Mitigation direction:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
200
 
201
- - explicit Zorgfilter v1 policy contract;
202
- - fully synthetic corpus across eight care-document families;
203
- - exact replace, review and preserve expectations;
204
- - current-engine baseline before recognizer changes;
205
- - separate care taxonomy and recognizers;
206
- - negative tests for medical numbers, dosages, times, vital signs and laboratory values;
207
- - cross-profile regression before UI promotion;
208
- - human review and residual-risk evidence remain mandatory.
209
 
210
- Approved policy boundary:
 
 
 
211
 
212
  ```text
213
- Patient identity and date of birth: replace.
214
- Other exact care dates and provider identity: review, selected by default.
215
- Clinical meaning: preserve.
216
- Rare-case indirect identification: audit warning, not blind masking.
217
  ```
218
 
219
- The current broad `NL_HEALTHCARE_REFERENCE` category is insufficient because it combines patient numbers, referral references, insurance identifiers and DBC/clinical codes under one behavior.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
220
 
221
- Current bounded baseline evidence:
222
 
223
- - 25/81 expected replace/review values were found as exact spans;
224
- - 14/81 were found under the intended entity type;
225
- - 11 were misclassified and 56 missed;
226
- - only 4/42 review-selected values were found;
227
- - one AGB value collided with BSN recognition;
228
- - no designated clinical preserve phrase was overlapped by the current custom rules.
229
 
230
- Generic NER was excluded, so the PERSON and e-mail findings do not represent complete live-app behavior. This evidence increases confidence that dedicated care patterns and review policy are necessary, but does not establish production quality.
231
 
232
- Gap triage classified all 81 expectations. The largest unresolved family is contextual review recognition (36 values), followed by generic profile dependencies (13), care-specific reclassification (10), dedicated care references (5) and AGB/numeric collision guards (3). The recognizer contract package must freeze these routes before any care pattern implementation.
233
 
234
- The recognizer contract is now frozen with sixteen dedicated entities, 37 positive exact-span cases and 16 negative/collision/preservation cases. Implementation must pass these fixtures before any app registration or UI promotion.
 
235
 
236
- The pure recognizer implementation now passes all frozen fixtures and all 54 dedicated corpus expectations with zero protected-clinical overlaps. Risk R10 remains open because generic NER composition, AGB/BSN cross-recognizer precedence, visible profile policy, cross-profile regression and live app verification are not yet complete.
237
 
238
- The central profile model now freezes Care composition and exact-span precedence without changing the live application. Risk R10 remains open until the current app registers the care recognizers, uses the profile policy, runs cross-profile regression and passes deployed app verification.
 
 
239
 
240
- The current Streamlit integration registers the sixteen care recognizers and applies the central profile policy. Review-selected care detections are selected by default but visibly marked `Controle nodig`; unresolved strongly labelled references remain unchecked candidates. Cross-profile regression, byte-for-byte deployment verification and live app verification are green, including confirmation that clinical meaning remains readable and existing review, export, Scrub Key and reinsert flows remain present. Risk R10 remains mitigating because synthetic and bounded app evidence does not establish production recall, precision or rare-case safety; human review remains mandatory.
241
 
242
- The deterministic cross-profile matrix now passes all hard gates: 108/108 dedicated Care expectations are retained across Care and International, no dedicated Care or Legal entities leak into the wrong profiles, dedicated-type parity holds, and no protected clinical phrase is overlapped. Historical legal metadata remains explicitly recorded as 132/148 deterministic expectations, sixteen gaps and four negative observations. Risk R10 remains mitigating because GitHub-to-Hugging-Face deployment sync, generic-NER behavior and live app verification are still unconfirmed.
 
243
 
244
- Deployment sync is now independently verified: twelve relevant GitHub/Hugging Face files match byte-for-byte, all correctly scoped markers pass and the Space is healthy. Risk R10 remains mitigating only for the remaining human-visible app verification, generic-NER observation and the broader limitation that synthetic evidence does not prove production recall or precision.
 
 
 
 
245
 
246
- The tester-facing care corpus now uses long-form structured variants across all eight approved document families. Each addition supplies substantial clinical and workflow context without adding new names, identifiers, dates, addresses, contact details, organizations, locations or digits. This improves usability and preservation testing but does not change recognizer behavior or establish production recall, precision or rare-case safety.
247
 
248
  ---
249
 
250
- ## Product-claim boundary
251
 
252
- Disallowed claims:
253
 
254
  ```text
255
- Alle persoonsnamen worden altijd gevonden.
256
- Alle persoonsgegevens worden altijd gevonden.
257
- Alle juridische nummers worden altijd herkend.
258
- De app is veilig voor productie zonder menselijke review.
259
- De benchmark bewijst production readiness.
260
  ```
261
 
262
- Allowed wording:
263
 
264
  ```text
265
- Scrub helpt gevonden gegevens te controleren en exporteren, maar menselijke review blijft noodzakelijk.
266
- Technische en auditdetails blijven beschikbaar voor controle.
267
  ```
 
 
 
1
+ # SolidPrivacy Scrub — Current Risk Register
2
 
3
+ This register tracks the **current** product, privacy, security, trust and execution risks for SolidPrivacy Scrub.
4
 
5
+ Detailed pre-convergence risk-history remains recoverable from exact baseline:
6
 
7
+ ```text
8
+ 54c73e0ebf5a3a3ed7039a50596fb57694add3cd
9
+ ```
10
+
11
+ Git/CHANGELOG preserve execution history. This file should show the live risk picture rather than accumulate workpackage narrative.
12
+
13
+ Status values: `open`, `mitigating`, `accepted`, `closed`.
14
  Impact values: `critical`, `high`, `medium`, `low`.
15
 
16
  ---
17
 
18
  ## R1 — False negatives / missed sensitive data
19
 
20
+ Status: `mitigating`
21
+ Impact: `critical`
22
 
23
  Risk:
24
 
25
  ```text
26
+ Sensitive data remains in scrubbed output and the user wrongly trusts the result.
27
  ```
28
 
29
+ Current controls/evidence:
30
+
31
+ - layered deterministic Dutch recognizers plus generic NER;
32
+ - dedicated Legal/Zorg rules and profile policy;
33
+ - recognizer-backed synthetic corpus benchmark/reporting;
34
+ - domain-specific baseline/gap-triage/cross-profile tests;
35
+ - manual missed-value addition and direct processed-text correction;
36
+ - mandatory human review;
37
+ - residual-risk/known-limitation language.
38
+
39
+ Current gaps:
40
+
41
+ - no accepted real-world production recall guarantee exists;
42
+ - synthetic corpora do not prove complete production coverage;
43
+ - human review remains necessary;
44
+ - new recognizer changes must remain evidence-driven so precision fixes do not silently reduce recall.
45
 
46
+ Required principle:
47
 
48
+ > Do not claim perfect recall or production safety from synthetic benchmark results.
 
 
 
49
 
50
  ---
51
 
52
+ ## R2 — Scrub Key leakage, mismatch or misuse
53
 
54
+ Status: `mitigating`
55
+ Impact: `critical`
56
 
57
  Risk:
58
 
59
  ```text
60
+ A Scrub Key leaks, is paired with the wrong document, is corrupted/tampered with, or is otherwise mishandled, enabling incorrect or unauthorized re-identification.
61
  ```
62
 
63
+ Current controls/evidence:
64
 
65
+ - explicit sensitive-key treatment in UI;
66
+ - document-bound placeholders;
67
+ - mapping digest;
68
+ - structural key validation;
69
+ - fail-closed wrong/mixed/missing-binding handling;
70
+ - controlled TXT/DOCX/PDF-to-TXT reinsert paths;
71
+ - adversarial/roundtrip tests;
72
+ - legacy unbound state kept explicitly unverified.
73
+
74
+ Current gaps:
75
+
76
+ - Scrub Key remains inherently sensitive re-identification material;
77
+ - cryptographic authenticity/signing is not implied by the current mapping digest;
78
+ - user/operational handling remains part of the threat model.
79
 
80
  ---
81
 
82
+ ## R3 — Placeholder corruption during AI/external roundtrip
83
 
84
+ Status: `mitigating`
85
+ Impact: `high`
86
 
87
  Risk:
88
 
89
  ```text
90
+ An external AI/process rewrites, translates, merges, deletes or corrupts placeholders, causing deterministic reinsert to fail or restore incompletely.
91
  ```
92
 
93
+ Current controls/evidence:
94
+
95
+ - stable bound placeholder grammar;
96
+ - mapping/document binding;
97
+ - roundtrip/adversarial mutation tests;
98
+ - unknown/missing placeholder diagnostics;
99
+ - fail-visible reinsert behavior.
100
+
101
+ Current gaps:
102
+
103
+ - malformed near-placeholders outside strict grammar may remain a diagnostic limitation;
104
+ - Scrub must not silently guess/repair values where binding cannot be proven.
105
 
106
  ---
107
 
108
+ ## R4 — Hidden document content, metadata and unsupported-part leakage
109
 
110
+ Status: `mitigating`
111
+ Impact: `high`
112
 
113
  Risk:
114
 
115
  ```text
116
+ DOCX metadata, comments, tracked changes, unsupported XML parts, headers/footers or other hidden content retains sensitive information outside the supported scrub/reinsert surface.
117
  ```
118
 
119
+ Current controls/evidence:
120
+
121
+ - DOCX hygiene/audit reporting;
122
+ - body/table/header/footer supported handling;
123
+ - synthetic fidelity/hygiene tests;
124
+ - explicit unsupported-part warnings;
125
+ - PDF support is text-based with explicit product limitations.
126
+
127
+ Current gaps:
128
+
129
+ - unsupported DOCX parts remain a known boundary;
130
+ - hygiene reporting must not be presented as a complete clean-document guarantee where unsupported content may exist.
131
 
132
  ---
133
 
134
+ ## R5 — Scrub Private content-retention / external-egress trust boundary
135
 
136
+ Status: `open`
137
+ Impact: `critical`
138
 
139
  Risk:
140
 
141
  ```text
142
+ A managed Private service retains customer document content, mappings or Scrub Keys, logs content, backs it up, or sends document content to a third party contrary to the intended service promise.
143
+ ```
144
+
145
+ Current evidence on pre-convergence/full-feature application:
146
+
147
+ - `replacement_memory.py` intentionally persists original/replacement/entity values to `/data/replacement_memory.json` when persistent storage exists;
148
+ - current Expert functionality includes Azure AI Language and OpenAI/Azure synthesis paths;
149
+ - synthesis helper currently prints a content-bearing prompt.
150
+
151
+ These are not evidence that the current prototype is defective for every deployment model; they are **variant-specific conflicts** with Scrub Private.
152
+
153
+ Scrub Private target:
154
+
155
+ ```text
156
+ no intentional persistent server-side customer document content
157
+ no content-bearing ordinary application logs
158
+ no document-content backup
159
+ no third-party document-processing egress
160
+ minimal non-document control-plane persistence only when required
161
  ```
162
 
163
+ Mitigation sequence:
164
+
165
+ - first complete Repository Convergence without destroying potentially useful Local/full-feature behavior;
166
+ - then remove/exclude hosted-incompatible persistence/egress from the Private line in separately tested/assured workpackages;
167
+ - prove application-level behavior on HF with synthetic data;
168
+ - prove infrastructure/service retention properties later on controlled production infrastructure.
169
+
170
+ Do not use HF application tests to claim provider-level zero retention.
171
 
172
  ---
173
 
174
+ ## R6 — Review UX, stale state and workflow clarity
175
 
176
+ Status: `mitigating`
177
+ Impact: `high`
178
 
179
  Risk:
180
 
181
  ```text
182
+ The user misses a sensitive value, changes state without realizing downstream output is stale, or the interface obscures whether review is complete/current.
183
  ```
184
 
185
+ Current controls/evidence:
 
 
 
 
 
 
 
 
 
186
 
187
+ - staged Premium workspace;
188
+ - Standard/Expert modes over shared authoritative state;
189
+ - generation-bound analysis/review/export state;
190
+ - fail-closed stale export gating;
191
+ - explicit re-completion after review edits;
192
+ - source-versus-processed review;
193
+ - authoritative replacement table;
194
+ - direct/manual missed-value correction;
195
+ - AppTest and integration coverage.
196
 
197
+ Current gap:
 
 
 
198
 
199
+ - parent Premium live-verification issues contain stale/unfinished closeout state; current deployed behavior must be reconciled against merged marker/address repairs before declaring those historical gates closed.
 
 
200
 
201
  ---
202
 
203
  ## R7 — PDF limitations misunderstood by users
204
 
205
+ Status: `mitigating`
206
+ Impact: `high`
207
 
208
  Risk:
209
 
210
  ```text
211
+ Users interpret text-based PDF support as OCR or complete restored-PDF fidelity.
212
  ```
213
 
214
+ Current boundary:
215
+
216
+ - text-based PDF extraction is supported;
217
+ - OCR is not implied;
218
+ - restored PDF generation is not implied;
219
+ - reinsert output may be restored text rather than a reconstructed PDF.
220
+
221
+ Copy, audit and pilot guidance must keep these limitations explicit.
222
 
223
  ---
224
 
225
+ ## R8 — Audit/evidence clarity and false confidence
226
 
227
+ Status: `mitigating`
228
+ Impact: `high`
229
 
230
  Risk:
231
 
232
  ```text
233
+ Multiple benchmark/report generations are mistaken for equivalent safety evidence, or diagnostic metrics are presented as production-readiness/individual-document safety scores.
234
  ```
235
 
236
+ Current controls:
237
 
238
+ - synthetic-only/report-only warnings in benchmark/residual-risk tooling;
239
+ - known limitations and human-review requirement;
240
+ - separate domain and workflow validation systems.
 
 
 
 
 
241
 
242
+ Current convergence requirement:
243
 
244
+ Classify existing evidence paths as:
245
+
246
+ ```text
247
+ CANONICAL RELEASE VALIDATION
248
+ SUPPLEMENTAL DIAGNOSTIC
249
+ HISTORICAL / SUPERSEDED
250
+ ```
251
+
252
+ Do not create a replacement Evidence Framework merely to make the hierarchy look cleaner.
253
+
254
+ Disallowed product behavior:
255
+
256
+ - per-document “94% safe” or similar false precision;
257
+ - production-safety claim solely from synthetic scores.
258
 
259
  ---
260
 
261
+ ## R9 — Dutch Legal recognition precision/recall and context damage
262
 
263
+ Status: `mitigating`
264
+ Impact: `high`
265
 
266
  Risk:
267
 
268
  ```text
269
+ Legal identifiers/addresses/names are missed or misclassified, or ordinary legal/professional context is over-masked and loses meaning.
270
  ```
271
 
272
+ Current controls/evidence:
273
+
274
+ - Dutch deterministic recognizers and Legal profile;
275
+ - synthetic legal examples/corpus;
276
+ - address-span precision repair on current main;
277
+ - preserve/known-trap evidence;
278
+ - manual/direct review correction;
279
+ - human review.
280
+
281
+ Future fixes remain test-first and must preserve recall/context.
282
 
283
  ---
284
 
285
+ ## R10 — Zorg under-detection and clinical over-masking
286
 
287
+ Status: `mitigating`
288
+ Impact: `critical`
289
 
290
  Risk:
291
 
292
  ```text
293
+ A care document retains patient/trajectory identifiers or Scrub removes clinical meaning and makes the document misleading/unusable.
294
  ```
295
 
296
+ Current controls/evidence:
297
+
298
+ - explicit Zorg profile/policy;
299
+ - eight synthetic care-document families plus long-form variants;
300
+ - current-engine baseline before dedicated rules;
301
+ - gap triage;
302
+ - dedicated care recognizer contracts/implementation;
303
+ - collision/negative tests;
304
+ - cross-profile regression;
305
+ - clinical preserve expectations;
306
+ - human review.
307
+
308
+ Current gap:
309
+
310
+ Synthetic and bounded app evidence does not establish complete production recall/precision or rare-case indirect-identification safety.
311
+
312
+ Required principle:
313
+
314
+ > Preserve diagnosis, medication, dosage, laboratory values, observations and useful care context unless there is a specific evidence-backed reason to treat a value as identifying.
315
 
316
+ ---
317
+
318
+ ## R11 — Repository/source-of-truth drift
 
 
 
 
 
319
 
320
+ Status: `mitigating`
321
+ Impact: `high`
322
+
323
+ Risk:
324
 
325
  ```text
326
+ Workers follow stale ROADMAP/WORKPACKAGES/issues or duplicate already-built functionality because repository documentation and actual main no longer describe one coherent current truth.
 
 
 
327
  ```
328
 
329
+ Observed pre-convergence symptoms:
330
+
331
+ - old nine-phase/local-installer roadmap still active in docs;
332
+ - multiple stacked `Current execution status override` sections in WORKPACKAGES;
333
+ - open issues describing candidate states that already PASSed/merged;
334
+ - historical Premium packages still presented as future work despite current main containing integrated staged behavior;
335
+ - several generations of benchmark/report tooling with unclear release authority.
336
+
337
+ Current mitigation:
338
+
339
+ - Repository Convergence is the active execution line;
340
+ - exact pre-convergence SHA preserved;
341
+ - temporary capability/debt ledger;
342
+ - ROADMAP/WORKPACKAGES/PROJECT_PROMPT reset;
343
+ - stale issue/evidence hierarchy reconciliation before clean-baseline declaration;
344
+ - no new feature work until `SCRUB_REPOSITORY_CONVERGED`.
345
 
346
+ Exit condition:
347
 
348
+ One clean exact SHA whose source, tests, current issues and canonical docs agree materially.
 
 
 
 
 
349
 
350
+ ---
351
 
352
+ ## R12 — Legacy runtime mutation / hidden startup authority
353
 
354
+ Status: `mitigating`
355
+ Impact: `medium`
356
 
357
+ Risk:
358
 
359
+ ```text
360
+ Historical source-patch machinery remains in the startup path and can be mistaken for current product authority or accidentally reactivated.
361
+ ```
362
 
363
+ Current evidence:
364
 
365
+ - Docker still invokes `fix_streamlit_nested_expanders.py` and `fix_streamlit_pdf_text_reinsert.py`;
366
+ - current Premium/direct-source markers cause both to exit without changing source.
367
 
368
+ Mitigation direction:
369
+
370
+ - verify no supported current path depends on the mutation scripts;
371
+ - retire invocation/obsolete scripts in a narrow consequential package if that proof holds;
372
+ - retain product-level behavior tests rather than tests whose only purpose is to keep dead patch machinery alive.
373
 
374
+ Do not combine this with hosted persistence/egress changes.
375
 
376
  ---
377
 
378
+ # Product-claim boundary
379
 
380
+ Disallowed claims include:
381
 
382
  ```text
383
+ Scrub vindt altijd alle persoonsgegevens.
384
+ Scrub garandeert volledige anonymisering.
385
+ De synthetische benchmark bewijst production readiness.
386
+ Hugging Face tests bewijzen zero retention op provider-infrastructuurniveau.
387
+ Een scrubbed document is veilig zonder menselijke controle.
388
  ```
389
 
390
+ Allowed direction:
391
 
392
  ```text
393
+ Scrub helpt gevoelige waarden te detecteren, controleren en pseudonimiseren terwijl context behouden blijft. Menselijke review en zichtbare beperkingen blijven onderdeel van het veiligheidsmodel.
 
394
  ```
395
+
396
+ For Scrub Private, content-retention/security claims must match independently verified application and service behavior exactly.
ROADMAP.md CHANGED
@@ -1,682 +1,370 @@
1
- # SolidPrivacy Scrub — Risk-driven Product & Development Roadmap
2
 
3
- This document is the central roadmap for SolidPrivacy Scrub.
4
 
5
  Use it together with:
6
 
7
- - `WORKPACKAGES.md` for the active execution queue, dependencies and verification gates;
8
- - `CHANGELOG.md` for internal implementation history;
9
- - `RELEASE_NOTES.md` for user-facing product changes;
10
- - `RISK_REGISTER.md` for trust, privacy and product risks;
11
- - `DECISION_LOG.md` for accepted strategic and architecture decisions;
12
- - `PROJECT_PROMPT.md` for worker rules and project governance.
13
 
14
- Last roadmap strategy update: 2026-08-08 — the Premium UI direction is now explicitly frozen as a **single-page staged document workspace**: `Toevoegen → Controleren → Downloaden` remain persistently represented in one workspace, exactly one stage is expanded at a time, completed stages collapse to compact summaries, future stages remain passive, and successful completion auto-advances. Three separate routed pages for these core stages are rejected for Standard. This urgent decision intervenes before production Streamlit integration in PR #85 and preserves its reusable pure helper work.
15
 
16
  ---
17
 
18
- ## Operational governance — implementation and blind release assurance
19
 
20
- Consequential Scrub work follows the canonical cross-project two-role standard used by the Weekly ETF donor architecture:
21
 
22
- ```text
23
- implementation_operations
24
- → identifiable release candidate
25
- → governance_release_assurance blind reconstruction
26
- → PASS / FAIL / INDETERMINATE
27
- → authorized action
28
- → independent post-action confirmation
29
- ```
30
-
31
- The user remains the single coordinator-facing principal. Implementation cannot certify its own candidate, and governance cannot silently repair it. Scrub adds a blind-review boundary: before its initial decision, governance may inspect source, criteria and raw machine evidence but not implementation handovers, self-assessments or conclusions.
32
-
33
- The current maturity is `LEVEL_1_CHECKLIST`, with a later target of `LEVEL_2_MACHINE_EVIDENCE`.
34
-
35
- The governance/cross-flow gate that previously blocked the Premium UI line is closed. Issue #70 is closed, PR #73 is merged, the Premium UI contract is merged through PR #80, and the pure Premium core-flow state model is merged through PR #82. The active line is therefore Premium UI implementation, subject to independent assurance for each consequential candidate.
36
 
37
- Governed by:
38
- - `control/PROJECT_GOVERNANCE_BOOTSTRAP.md`;
39
- - `control/SCRUB_RELEASE_ASSURANCE_CONTRACT_V1.md`;
40
- - `DECISION_LOG.md` D042.
 
 
 
 
41
 
42
- ## 1. Product vision
43
-
44
- Scrub is evolving from a technical Presidio demo into a trustworthy Dutch professional document scrubber for confidential legal and care documents.
45
-
46
- The key promise remains:
47
 
48
  ```text
49
- Sensitive information stays local in the final trust environment.
50
- The user remains in control.
51
- The document stays readable.
52
- Residual risk is visible.
53
  ```
54
 
55
- Current sequencing principle:
56
-
57
- ```text
58
- Make the MVP workflow good before investing further in pilot operations, marketing, pricing, sales, promotion or installer work.
59
- ```
60
-
61
- The active MVP workflow is:
62
-
63
- ```text
64
- Import -> Scrub -> Review -> Replace -> Scrub Key -> Reinsert -> Export -> Audit
65
- ```
66
 
67
  ---
68
 
69
- ## 2. Core product principle
70
-
71
- The product is not just a generic anonymizer.
72
 
73
- The real problem is:
74
 
75
- ```text
76
- How can a professional safely use or share confidential documents without losing context, legal meaning, control or auditability?
77
- ```
78
 
79
- Scrub must optimize for:
80
-
81
- 1. high recall for sensitive data;
82
- 2. context preservation;
83
- 3. human review;
84
- 4. easy and consistent replacement logic;
85
- 5. secure Scrub Key lifecycle;
86
- 6. reliable de-anonymization/reinsert;
87
- 7. document hygiene;
88
- 8. safe export;
89
- 9. audit and residual-risk reporting;
90
- 10. clear interface and workflow confidence.
91
-
92
- Legal and professional context must be preserved. Scrub should replace sensitive values, not legal meaning.
93
-
94
- ---
95
-
96
- ## 3. Strategic workflow
97
-
98
- The product workflow direction remains:
99
-
100
- ```text
101
- Scrub -> Review -> Scrub Key -> AI -> Reinsert -> Export -> Audit
102
- ```
103
-
104
- The roadmap is risk-driven rather than feature-driven.
105
-
106
- Highest priority risks:
107
-
108
- 1. false negatives;
109
- 2. Scrub Key leakage or accidental sharing;
110
  3. hidden document content and metadata leakage;
111
- 4. placeholder corruption during AI roundtrip;
112
- 5. review UX that is not document-centric enough;
113
- 6. confusing import/export/reinsert limitations;
114
- 7. installer/distribution effort before the product behavior is ready.
115
 
116
- ---
117
-
118
- ## 4. Current implementation status
119
-
120
- Recent work completed or recorded:
121
-
122
- ```text
123
- WP19-WP24 — recall/trust foundation.
124
- WP25-WP29C — Scrub Key safety/test line.
125
- WP30-WP34 — placeholder robustness helper/test line.
126
- WP35-WP39 — DOCX hygiene line through clean-DOCX export policy and report-only audit UI.
127
- WP40-WP43 — review UX/frontend decision line.
128
- WP_REPLACE_LOGIC_UI_PRODUCT_ROLLBACK_VERIFY — product-rejected helper panel hidden and verified.
129
- WP_REPLACE_LOGIC_UI_REDESIGN_PLAN — replacement review redesigned around simple user choices.
130
- WP_SIDE_BY_SIDE_REVIEW_ROADMAP_ANCHOR — unified side-by-side review UX direction anchored.
131
- WP45-WP49 — local runtime/packaging decision line; installer deferred.
132
- WP50-WP51 — pilot/ICP thinking artifacts; parked for now.
133
- WP_RECALL_PERSON_NAME_* — diagnostic, contract and helper-level PERSON-name work completed; benchmark follow-up temporarily parked unless a concrete blocker appears.
134
- SCRUB-WP_EXECUTION_INTERFACE_SIMPLIFICATION_* through SCRUB-WP_MANUAL_CORRECTION_PANEL_DENSITY_SIMPLIFICATION_* — the current MVP UI simplification line is completed, synchronized and live-app verified.
135
- SCRUB-WP_MVP_DOCUMENT_HYGIENE_FIDELITY_HARDENING — merged, synchronized and live-app verified for DOCX body, table, header and footer restoration.
136
- SCRUB-WP_MVP_REINSERT_AUTO_FLOW_SIMPLIFICATION_IMPLEMENTATION — completed, synchronized and live-app verified.
137
- SCRUB-WP_MVP_SCRUB_KEY_ROUNDTRIP_VALIDATION — completed with 15/15 cases passing and a critical document/key-binding gap routed to triage.
138
- SCRUB-WP_MVP_SCRUB_KEY_DOCUMENT_BINDING_GAP_TRIAGE — completed with a bound-placeholder plus mapping-digest recommendation and a test-first implementation sequence.
139
- SCRUB-WP_MVP_SCRUB_KEY_BINDING_CONTRACT_TESTS — completed with frozen placeholder, digest, legacy and fail-closed model contracts.
140
- SCRUB-WP_MVP_SCRUB_KEY_BINDING_MODEL_IMPLEMENTATION — completed with pure binding-ID, placeholder, digest, bound-key and document/key validation helpers.
141
- SCRUB-WP_MVP_SCRUB_KEY_BINDING_EXPORT_INTEGRATION — completed with bound placeholders and schema-1.1 key export.
142
- SCRUB-WP_MVP_SCRUB_KEY_BINDING_REINSERT_INTEGRATION — implemented with dual-read import, fail-closed binding enforcement and explicit legacy compatibility; app verification pending.
143
- SCRUB-WP_AI_FIRST_DESKTOP_PACKAGING_ROADMAP_ALIGNMENT — completed as a documentation-only Phase 9 execution-model refinement; installer implementation remains gated.
144
- SCRUB-WP_PROCESSED_TEXT_SELECTION_CROSS_FLOW_REGRESSION — independently assured and governance gate closed through issue #70/#73 recovery.
145
- SCRUB-WP_PREMIUM_CORE_FLOW_UI_CONTRACT — merged through PR #80; contract only, no production UI behavior.
146
- SCRUB-WP_PREMIUM_CORE_FLOW_STATE_MODEL — merged through PR #82; pure state model only, no production UI behavior.
147
- SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION — active in draft PR #85; current candidate contains reusable presentation primitives/tests but production `presidio_streamlit.py` integration has not started.
148
- SCRUB-WP_PREMIUM_STAGED_WORKSPACE_DECISION_FREEZE — urgent architecture/queue intervention approved by the coordinator on 2026-08-08; must be incorporated before PR #85 proceeds to production UI integration.
149
- ```
150
-
151
- Important UX status:
152
-
153
- ```text
154
- The review table remains source of truth and fallback.
155
- The old replacement decision helper panel must not return as normal user-facing UI.
156
- The unified side-by-side review, manual missed-value entry, compact placeholder display and grouped export flow form the verified functional baseline.
157
- Direct live-app evidence confirms that this baseline still presents too much of the workflow as one long form.
158
- The Premium Standard target is one document workspace with three persistent stages and one active task, not another isolated expander-cleanup pass and not three routed core-flow pages.
159
- ```
160
 
161
  ---
162
 
163
- ## 5. MVP product quality gate
164
 
165
- Phase 7 follow-up is parked until the MVP is credible across:
166
 
167
- - import;
168
- - anonymization;
169
- - review;
170
- - replacement logic;
171
- - Scrub Key handling;
172
- - de-anonymization/reinsert;
173
- - export;
174
- - audit/residual-risk reporting;
175
- - interface clarity.
176
 
177
- WP50 and WP51 remain useful early thinking artifacts, but they are not the active execution line now.
 
 
 
 
 
 
 
178
 
179
- Do not start WP52 by default.
180
 
181
  ---
182
 
183
- ## 6. Current phase order
184
-
185
- The current phase order is:
186
-
187
- ```text
188
- Phase 1 — Trust & recall foundation
189
- Phase 2 — Scrub Key security and lifecycle
190
- Phase 3 — Placeholder robustness for AI roundtrip
191
- Phase 4 — Hidden content and document hygiene
192
- Phase 5 — Document-centric review UX
193
- Phase 6 — MVP workflow validation and trust hardening
194
- Phase 7 — Pilot validation: Legal vs Zorg — parked until MVP quality gate passes
195
- Phase 8 — Scale features: profiles, batch, CLI, enterprise
196
- Phase 9 — Final local desktop/offline installer path
197
- ```
198
-
199
- ### Phase 5/6 — Review UX direction
200
-
201
- The review UX target is now:
202
-
203
- ```text
204
- Source text left | Processed/checked text right
205
- | Optional highlights integrated in the processed text
206
- ```
207
-
208
- This is governed by:
209
 
210
- - `DECISION_LOG.md` D021;
211
- - `SIDE_BY_SIDE_REVIEW_UX_DIRECTION.md`;
212
- - `REPLACE_LOGIC_UI_REDESIGN_PLAN.md`;
213
- - `MVP_UI_CLEANUP_AND_EXPORT_REDESIGN_PLAN.md`;
214
- - `PREMIUM_CORE_FLOW_UI_REALIGNMENT_PLAN.md`;
215
- - `PREMIUM_STAGED_WORKSPACE_DECISION.md`.
216
 
217
- UX principles:
218
 
219
- - one main review surface before more helper panels;
220
- - review table remains source of truth and fallback;
221
- - serial review remains a guided review layer, not a table replacement;
222
- - highlights are visual aid, not mutation mechanism;
223
- - no repeated per-highlight `Gemarkeerd` labels as the long-term design;
224
- - no click-to-mark, advanced editor or full-document marking in this phase;
225
- - no Scrub Key/export/reinsert behavior changes from review UX work without separate approval;
226
- - technical/debug-like details should move to secondary audit/advanced layers, not disappear.
227
-
228
- ### Premium core-flow UI realignment — binding staged-workspace decision
229
-
230
- Direct coordinator/user evidence on 2026-08-05 established that the remaining interface problem is structural:
231
-
232
- ```text
233
- The app still behaves visually as one long Streamlit form.
234
- Input, settings, review, corrections, downloads, Scrub Key and audit controls compete on the same page.
235
- ```
236
-
237
- The approved target remains:
238
-
239
- ```text
240
- Top-level workflow: Anonimiseren | Terugzetten
241
- Global presentation: Standaard | Expert
242
- Core stages: Toevoegen → Controleren → Downloaden
243
- One primary action per active stage
244
- Progressive and conditional disclosure for settings, other formats, Scrub Key and audit evidence
245
- ```
246
 
247
- On 2026-08-08 the coordinator explicitly chose how those three core stages must be presented. The Standard anonymization flow is a **single-page staged document workspace**, not three separate routed screens.
248
 
249
- Binding interaction model:
250
 
251
  ```text
252
- One document
253
- → one persistent workspace
254
- → three persistent stage headers
255
- → exactly one expanded/active stage
256
- → completed stages collapse to compact summaries
257
- → future stages remain visible but passive
258
- → successful completion auto-opens the next stage
259
- → explicit return to an earlier stage is allowed
260
- → processing-affecting earlier changes invalidate downstream state fail-closed
261
  ```
262
 
263
- The first-principles reason is that Scrub is an iterative document-review workflow rather than a strictly linear checkout wizard. The interface should represent the **state of one document** rather than force the user to navigate separate pages. A staged workspace keeps the cognitive benefit of one task at a time while preserving context, progress, confidence and easy return from review to input.
264
 
265
- The rejected Standard pattern is:
266
-
267
- ```text
268
- Toevoegen page → Controleren page → Downloaden page
269
- ```
 
 
 
270
 
271
- The rejected implementation shortcut is also:
272
 
273
  ```text
274
- classic long Streamlit form
275
- + multiple independent/nested expanders
276
  ```
277
 
278
- Stage sections must read as application panels, not as a FAQ/settings accordion. There must be no default nested core-flow expander hierarchy in Standard.
279
 
280
- `Standaard` is lower cognitive load, not lower safety. `Expert` preserves full inspection, tuning, audit and troubleshooting. The permanent settings sidebar is not part of the Standard target. Completed stages show only compact orientation/trust summaries rather than full controls.
281
 
282
- Target progression example:
283
 
284
- ```text
285
- Initial:
286
- ▼ 1 Toevoegen
287
- 2 Controleren — beschikbaar na verwerking
288
- 3 Downloaden — beschikbaar na controle
289
-
290
- After processing:
291
- ✓ 1 Toevoegen — contract.docx · Juridisch
292
- ▼ 2 Controleren
293
- 3 Downloaden — beschikbaar na controle
294
-
295
- After review:
296
- ✓ 1 Toevoegen — contract.docx · Juridisch
297
- ✓ 2 Controleren — 14 gecontroleerd · 1 handmatig toegevoegd
298
- ▼ 3 Downloaden
299
- ```
300
 
301
- The implementation remains within Streamlit first, but must approximate a single-task application shell. It must not change recognizers, replacement decisions, export bytes, filenames, MIME types, Scrub Key semantics, reinsert behavior, audit evidence or the human-review requirement.
302
 
303
- The detailed decision, alternative comparison, interaction rules, safety invariants and acceptance criteria are frozen in `PREMIUM_STAGED_WORKSPACE_DECISION.md`.
304
 
305
- ### Premium execution sequence — urgent realignment
306
 
307
- The previous Premium sequence is amended before production UI integration in PR #85:
308
 
309
  ```text
310
- 0. SCRUB-WP_PROCESSED_TEXT_SELECTION_CROSS_FLOW_REGRESSION — completed / gate closed
311
- 1. SCRUB-WP_PREMIUM_CORE_FLOW_UI_CONTRACT — completed / PR #80 merged
312
- 2. SCRUB-WP_PREMIUM_CORE_FLOW_STATE_MODEL — completed / PR #82 merged
313
- 3. SCRUB-WP_PREMIUM_STAGED_WORKSPACE_DECISION_FREEZE — URGENT CURRENT PACKAGE
314
- 4. SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION — active draft PR #85; production integration paused until package 3 is incorporated
315
- 5. SCRUB-WP_PREMIUM_INPUT_STAGE_SIMPLIFICATION
316
- 6. SCRUB-WP_PREMIUM_REVIEW_STAGE_SIMPLIFICATION
317
- 7. SCRUB-WP_PREMIUM_EXPORT_STAGE_SIMPLIFICATION
318
- 8. SCRUB-WP_PREMIUM_EXPERT_PARITY_REGRESSION
319
- 9. SCRUB-WP_PREMIUM_CORE_FLOW_APP_VERIFY_CLOSEOUT
320
  ```
321
 
322
- PR #85 is **amended, not discarded**. Its current pure `premium_app_shell.py` helpers/tests are consistent with the one-active-stage direction and may be retained. Before production `presidio_streamlit.py` integration, the candidate must additionally freeze/test active/completed/future stage presentation, compact completed summaries, passive future stages, explicit return/edit affordances, auto-progression hooks and the no-three-page-routing rule.
323
-
324
- Do not combine input, review and export restructuring into one patch. Do not run shared Streamlit UI packages in parallel. Each consequential production UI candidate receives separate `governance_release_assurance` under the enforced contract.
325
-
326
- ### Phase 6 — MVP workflow validation and trust hardening
327
-
328
- Goal:
329
-
330
- ```text
331
- Use the online/web prototype and GitHub workflow to validate product behavior before pilot expansion or installer investment.
332
- ```
333
 
334
- Focus:
 
 
 
 
335
 
336
- - end-to-end workflow testing;
337
- - import and export confidence;
338
- - Scrub Key warning/acknowledgement verification;
339
- - DOCX hygiene reporting;
340
- - unified side-by-side review direction;
341
- - easier replace/review logic;
342
- - residual-risk and audit reporting;
343
- - MVP interface clarity;
344
- - professional export/download flow;
345
- - Hugging Face app verification after GitHub Actions/sync when UI changes.
346
 
347
- ### Phase 7 — parked pilot validation
348
 
349
- WP50 and WP51 are recorded, but Phase 7 is not the active next line.
350
 
351
- Phase 7 may reopen when the coordinator confirms the MVP product quality gate has been met.
352
 
353
- ---
354
 
355
- ## 7. Active next work direction
 
 
 
 
356
 
357
- The coordinator has explicitly prioritized the Premium interface. The Premium staged-workspace line is therefore the dominant execution queue until its live closeout, unless a privacy/safety blocker requires interruption.
358
 
359
- Current dominant queue:
360
 
361
  ```text
362
- 1. SCRUB-WP_PREMIUM_STAGED_WORKSPACE_DECISION_FREEZE — current; architecture/roadmap/workpackage binding
363
- 2. SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION — draft PR #85; resume production integration only after #1 is incorporated
364
- 3. SCRUB-WP_PREMIUM_INPUT_STAGE_SIMPLIFICATION
365
- 4. SCRUB-WP_PREMIUM_REVIEW_STAGE_SIMPLIFICATION
366
- 5. SCRUB-WP_PREMIUM_EXPORT_STAGE_SIMPLIFICATION
367
- 6. SCRUB-WP_PREMIUM_EXPERT_PARITY_REGRESSION
368
- 7. SCRUB-WP_PREMIUM_CORE_FLOW_APP_VERIFY_CLOSEOUT
369
  ```
370
 
371
- The earlier Phase 6 trust-hardening packages remain part of project history and may be reopened when required by evidence, but they are not allowed to dilute the current Premium UI focus by default.
372
-
373
- Execution principles:
374
 
375
- - use synthetic data only;
376
- - preserve legal meaning and keep human review mandatory;
377
- - the Premium shell is presentation/state orchestration, not a license to change recognition or export semantics;
378
- - one document, one workspace, three persistent stages, one active task;
379
- - no three routed pages for `Toevoegen → Controleren → Downloaden` in Standard;
380
- - no nested-expander recreation of the current long form;
381
- - create recognizer or document-processing fixes only from reproducible safety evidence;
382
- - live UI work must be sequential where it touches shared Streamlit/review/export surfaces;
383
- - do not make production-readiness claims from prototype evidence.
384
 
385
- ### Candidate document-centric manual-correction line
386
 
387
- Direct usability evidence supports correction in the main processed-text pane:
388
 
389
- ```text
390
- select an unmasked value
391
- → right-click
392
- → choose a masking type
393
- → add through the existing manual replacement path
394
- ```
395
 
396
- The first version is bounded to all exact occurrences of the selected value. It creates a normal document-scoped manual row, keeps the replacement table authoritative and preserves current bound export, Scrub Key and reinsert semantics. Occurrence-specific replacement, a rich editor and a combined Streamlit upgrade are excluded.
397
 
398
- The sequence was:
399
 
400
  ```text
401
- 1. SCRUB-WP_PROCESSED_TEXT_SELECTION_MASKING_CONTRACT
402
- 2. SCRUB-WP_PROCESSED_TEXT_SELECTION_MASKING_ACTION_MODEL
403
- 3. SCRUB-WP_PROCESSED_TEXT_SELECTION_COMPONENT_SPIKE
404
- 4. SCRUB-WP_PROCESSED_TEXT_SELECTION_TABLE_INTEGRATION
405
- 5. SCRUB-WP_PROCESSED_TEXT_SELECTION_CROSS_FLOW_REGRESSION
406
- 6. SCRUB-WP_PROCESSED_TEXT_SELECTION_APP_VERIFY
 
 
 
407
  ```
408
 
409
- The coordinator approved the direction and all-exact version-one boundary at 2026-08-04 00:09 Europe/Amsterdam. The contract, action model, component and production table integration are complete. GitHub-to-Hugging-Face synchronization and live app verification are green. Display-only placeholder compaction is deployed and app-verified without changing binding grammar, entropy, export, Scrub Key or reinsert semantics. The cross-flow safety/governance gate is now closed and no longer blocks Premium UI work.
410
-
411
- Recall/benchmark work is reopened only where reproducible safety evidence requires it.
412
-
413
- Do not start local packaging next steps such as `WP48B` or `WP49B` by default. They require explicit coordinator approval.
414
 
415
- Do not start pilot follow-up such as `WP52` by default. It remains gated by `SCRUB-WP_MVP_PHASE6_QUALITY_GATE_CLOSEOUT` and explicit coordinator approval.
416
 
417
- ---
418
-
419
- ## 8. Parallelization strategy
420
-
421
- Safe to do in parallel:
422
-
423
- - helper modules with separate files;
424
- - tests that do not touch the same UI flow;
425
- - specifications;
426
- - documentation;
427
- - benchmark data design;
428
- - risk reviews;
429
- - non-UI architecture work.
430
-
431
- Do not run in parallel:
432
-
433
- - multiple changes to `fix_streamlit_nested_expanders.py` or `presidio_streamlit.py`;
434
- - multiple changes to the same Streamlit patch file;
435
- - `serial_review_panel_ui.py` UI edits without coordination;
436
- - `review_highlight_toggle_panel_ui.py` UI edits without coordination;
437
- - export/download flow changes;
438
- - review table flow changes;
439
- - synchronized scroll implementation;
440
- - custom HTML/component rendering implementation;
441
- - Docker/runtime startup patch-order changes;
442
- - installer/packaging work unless explicitly approved;
443
- - Phase 7 follow-up while the MVP quality gate is not passed.
444
-
445
- Use `workpackage_claims/` before starting a package.
446
-
447
- ---
448
-
449
- ## 9. Product architecture target
450
 
451
- Current prototype architecture:
452
 
453
- ```text
454
- Streamlit app + helper modules + GitHub Actions + Hugging Face Space demo.
455
- ```
 
 
 
 
 
456
 
457
- MVP architecture target:
458
 
459
  ```text
460
- Thin helper-driven Streamlit application shell;
461
- one persistent document workspace;
462
- three core Standard stages with exactly one active at a time;
463
- global Standard/Expert presentation;
464
- tested safety/state boundaries;
465
- local-first direction;
466
- and a clear primary export with secondary restore/audit layers.
467
  ```
468
 
469
- Do not migrate frontend, add a full document editor, introduce cloud document processing, or alter export/Scrub Key/reinsert semantics without a dedicated approved package.
470
-
471
  ---
472
 
473
- ## 10. Phase 9 — AI-first local desktop/offline packaging execution model
474
 
475
- The final trust environment remains a local Windows desktop installation for confidential documents. Hosting migration and local desktop distribution are separate concerns:
476
 
477
- ```text
478
- Hugging Face / web prototype = synthetic and approved non-confidential validation
479
- Local signed desktop runtime = confidential production-oriented trust environment
480
- ```
481
 
482
- Phase 9 remains gated by Phase 6 quality closeout and explicit coordinator approval. This roadmap update authorizes no installer implementation by itself.
483
 
484
- ### Target end-user distribution
 
 
 
 
 
 
485
 
486
- The preferred end-user direction is:
487
 
488
- ```text
489
- one signed setup.exe for low-friction installation
490
- + one signed MSI for managed organizational deployment
491
- + Tauri Windows shell
492
- + bundled local Python/Presidio engine as a PyInstaller onedir sidecar
493
- + all required models and assets available locally
494
- + loopback-only communication on the same PC
495
- + no required cloud calls, runtime downloads or document telemetry
496
- ```
497
 
498
- A portable Python folder remains an internal technical validation path, not the intended premium end-user product. A PyInstaller onefile runtime is not the default because extraction to temporary executable directories increases startup, antivirus, cleanup and diagnostic risk. The user may receive one installer file even when the installed application contains multiple controlled runtime files.
 
 
 
 
 
 
 
 
499
 
500
- ### AI-first execution assumption
501
 
502
- Phase 9 should be implemented AI-first where the work is deterministic, testable and reversible. Planning assumptions:
503
 
504
  ```text
505
- First installer implementation:
506
- - 60–70% of development and integration labor may be agent-executed.
507
- - expected development/integration budget after agent substitution: approximately EUR 8,000–24,000.
508
- - agent/build-compute allowance: approximately EUR 1,000–4,000.
509
-
510
- Subsequent release cycles:
511
- - 75–90% of repetitive build, test, packaging and release-candidate preparation may be automated.
512
  ```
513
 
514
- These are budgeting assumptions, not supplier quotations or production-readiness claims. Independent security review, code-signing identity, final release approval and real user acceptance remain separate costs and responsibilities.
515
-
516
- ### Agent-autonomous scope
517
-
518
- Within isolated workstations or disposable Windows test VMs, scoped agents may autonomously:
519
-
520
- - install approved build SDKs and packaging dependencies;
521
- - inventory and pin Python packages, native libraries and model assets;
522
- - build and debug PyInstaller onedir bundles;
523
- - build the Tauri shell and local sidecar lifecycle;
524
- - create setup.exe and MSI release candidates;
525
- - implement loopback binding, health checks and shutdown cleanup;
526
- - run offline, install, upgrade, uninstall, Defender and synthetic document tests;
527
- - inspect network, process, filesystem, logging and temporary-file evidence;
528
- - maintain CI, checksums, SBOMs, release notes and handovers;
529
- - open pull requests and prepare unsigned release candidates.
530
-
531
- All test documents and Scrub Keys must remain synthetic. Agents must not weaken review, Scrub Key, export or privacy controls to make packaging easier.
532
-
533
- ### Human-controlled gates
534
-
535
- The following responsibilities must not be delegated as an unreviewed autonomous chain:
536
-
537
- - legal publisher identity and long-lived code-signing authority;
538
- - final production signing and public release;
539
- - acceptance of privacy and security claims;
540
- - approval of export, Scrub Key or document-processing semantic changes;
541
- - acceptance testing by non-technical users on real managed Windows environments;
542
- - independent security validation where a strong local-only product claim is made.
543
-
544
- No single agent should simultaneously hold unrestricted repository-write, signing-identity and public-release authority. Use protected environments, least-privilege identities and an explicit human release gate.
545
-
546
- ### Indicative external and retained costs
547
-
548
- Costs that remain materially external or human-controlled include:
549
-
550
- ```text
551
- - managed Windows code signing: approximately EUR 120–300 per year;
552
- - build/CI/test infrastructure: usually limited, depending on runner and artifact use;
553
- - physical or managed Windows test devices: approximately EUR 0–1,500 initially;
554
- - targeted independent desktop/privacy security review: approximately EUR 5,000–15,000;
555
- - broader penetration testing and retest, if required: approximately EUR 15,000–30,000;
556
- - human packaging/security/release oversight: retained even with extensive agent autonomy.
557
- ```
558
 
559
- ### Phase 9 execution sequence
560
 
561
- When the phase is explicitly opened, use small sequential workpackages:
562
 
563
  ```text
564
- 1. Desktop distribution and local-only security contract
565
- 2. Offline dependency, native-library and model inventory
566
- 3. PyInstaller onedir engine-sidecar packaging spike
567
- 4. Tauri shell and sidecar lifecycle proof
568
- 5. Network, temp-file, logging, crash and endpoint-security validation
569
- 6. Signed setup.exe and MSI release candidate
570
- 7. Managed Windows pilot, upgrade, rollback and uninstall validation
571
- 8. Independent security review and quality-gate closeout
572
  ```
573
 
574
- Do not collapse signing, release and security acceptance into the same autonomous worker. Do not make a production/local-only claim from successful packaging alone.
575
 
576
- ## 11. Zorgfilter v1 — evidence-driven care profile
577
 
578
- Approved: 2026-08-03 15:31 Europe/Amsterdam
579
-
580
- The first product wedge already identifies Legal and Zorg as the most relevant Dutch professional domains. Zorg now receives an explicit profile line rather than remaining an incidental subset of general and legal recognition.
581
 
582
- ### Approved policy
583
 
584
- ```text
585
- Geboortedatum: vervangen
586
- Overige exacte zorgdata: controleren en standaard geselecteerd
587
- Patiënt- en cliëntidentificatie: vervangen
588
- Zorgverleneridentificatie: controleren en standaard geselecteerd
589
- Diagnose, medicatie, dosering, labwaarden en observaties: behouden
590
- Zeldzame-casus-herleidbaarheid: auditwaarschuwing, niet blind maskeren
591
- ```
592
 
593
- The core product rule is:
594
 
595
- ```text
596
- Remove identity and patient-specific administrative references while preserving clinical meaning.
597
- ```
598
 
599
- Zorgfilter v1 is not a generic medical-word filter. It must not make care records clinically unreadable.
600
 
601
- ### Initial document scope
602
 
603
- - daily nursing/care reports;
604
- - care plans and evaluations;
605
- - nursing transfers;
606
- - medical specialist discharge letters;
607
- - GP referrals and consultation letters;
608
- - medication overviews or administration lists;
609
- - laboratory reports;
610
- - MIC/MIM/VIM care-incident reports.
611
 
612
- ### Architecture and sequencing
613
 
614
- The first packages are pure helper, policy, corpus and evidence work. They may proceed without reopening the shared Streamlit review/export flow. The future current-UI integration is permitted by explicit coordinator approval, but only after the corpus baseline, gap triage, recognizer contracts and recognizer implementation are green and no parallel worker is editing the same UI surface.
 
 
 
 
 
 
615
 
616
- Preferred helper direction:
617
 
618
  ```text
619
- care_profile_policy.py
620
- care_test_examples.py
621
- care_profile_baseline.py
622
- care_reference_taxonomy.py
623
- dutch_care_recognizers.py
624
- recognition_profiles.py
625
  ```
626
 
627
- The existing broad `NL_HEALTHCARE_REFERENCE` category must be assessed and split. Patient numbers, referral references, insurance identifiers and DBC/clinical codes do not share one safe default action.
628
-
629
- Current-engine baseline evidence is now recorded: 25 of 81 expected replace/review values were found as exact normalized spans, only 14 under the intended entity type, 11 were misclassified and 56 were missed. The bounded custom-rule baseline produced zero overlaps with the designated clinical preserve passages. Generic NER was excluded, so PERSON and e-mail results are not full-app measurements.
630
 
631
- The dedicated pure recognizer implementation is now green but remains unregistered: sixteen care entities pass 37/37 positive contracts, 16/16 negative/collision contracts and all 54 dedicated expectations in the eight-document corpus, with zero protected-clinical overlaps. The next gate is central profile composition and AGB/BSN precedence before any visible Zorg profile is added.
 
 
 
 
 
 
632
 
633
- Central profile configuration is now implemented without changing the live UI. The current three options remain exact, while future Streamlit and desktop four-profile orders, thresholds, entity groups, care policy actions and fifteen exact-span precedence winners are frozen. Care recognizer registration and visible UI integration remain the next gated package.
634
 
635
- The current Streamlit integration is now implemented and regression-green. `Zorgcontrole — streng` is added without silently becoming the default; the existing Legal profile remains initially selected. Sixteen care recognizers, central entity composition, exact-span precedence, eight synthetic examples and conservative unchecked care candidates are wired into the current flow. Review-selected care detections remain selected but show `Controle nodig`. Export, Scrub Key and reinsert semantics are unchanged. The next gates are cross-profile regression, deployment sync and live app verification.
636
 
637
- The deterministic cross-profile regression matrix is now green. Across eight care document families and twelve legal examples, Care and International retain all 108 dedicated care expectations, Care/International and Legal/International dedicated-type parity hold, no dedicated Care or Legal entities leak into the wrong profiles, and no protected clinical phrase is overlapped. The historical legal metadata remains visible as 132/148 deterministic expectations, sixteen recorded gaps and four negative observations; these are existing benchmark observations, not hidden or reclassified as Zorg success. Generic NER remains outside this matrix. GitHub-to-Hugging-Face synchronization and deployed app behavior were confirmed on 2026-08-03; the current-web Zorgfilter line is completed and app-verified. Desktop UX work remains separately gated by Phase 9 and explicit approval.
638
 
639
- GitHub-to-Hugging-Face synchronization is now independently verified for merge commit `cca4a25aaff28a7ba647c961d8e50f0e076921e2`: twelve relevant source files match byte-for-byte, all correctly scoped Zorg markers are present, the Streamlit health endpoint returns `200 / ok` and the Space root returns HTTP 200. The initial sync check produced a false negative because two marker groups were assigned to the wrong modules; the hashes were already equal and the corrected verification passed. The remaining gate is coordinator/user confirmation of the visible app behavior and generic-NER observation.
640
 
641
- ### Current and final interface direction
642
 
643
- Current prototype after test-gated integration:
644
 
645
- ```text
646
- Zorgcontrole — streng
647
- Juridische controle — streng
648
- Algemene Nederlandse controle
649
- Algemene internationale controle
650
- ```
651
 
652
- Final desktop workspace:
653
 
654
  ```text
655
- [ Algemeen NL ] [ Zorg ] [ Juridisch ] [ Internationaal ]
 
 
 
 
 
 
 
 
 
656
  ```
657
 
658
- The active profile remains visible in the document toolbar and never changes silently.
659
-
660
- ### Sequential workpackages
661
-
662
- ```text
663
- 1. SCRUB-WP_CARE_PROFILE_V1_POLICY_AND_CORPUS_FOUNDATION
664
- 2. SCRUB-WP_CARE_PROFILE_CURRENT_ENGINE_BASELINE
665
- 3. SCRUB-WP_CARE_PROFILE_GAP_TRIAGE
666
- 4. SCRUB-WP_CARE_PROFILE_RECOGNIZER_CONTRACT_TESTS
667
- 5. SCRUB-WP_CARE_PROFILE_RECOGNIZER_IMPLEMENTATION
668
- 6. SCRUB-WP_RECOGNITION_PROFILE_CONFIGURATION_REFACTOR
669
- 7. SCRUB-WP_CARE_PROFILE_CURRENT_UI_INTEGRATION
670
- 8. SCRUB-WP_CARE_PROFILE_CROSS_PROFILE_REGRESSION_MATRIX
671
- 9. SCRUB-WP_CARE_PROFILE_APP_VERIFY
672
- 10. SCRUB-WP_CARE_PROFILE_DESKTOP_UX_CONTRACT
673
- ```
674
 
675
- Safety boundaries:
676
 
677
- - synthetic data only;
678
- - no blind masking of clinical meaning;
679
- - no change to Scrub Key, export or reinsert semantics without a separate package;
680
- - human review remains required;
681
- - corpus or benchmark success does not prove production readiness;
682
- - no cloud document processing is introduced.
 
1
+ # SolidPrivacy Scrub — Product & Development Roadmap
2
 
3
+ This document is the strategic roadmap for SolidPrivacy Scrub.
4
 
5
  Use it together with:
6
 
7
+ - `WORKPACKAGES.md` — current executable queue, dependencies and verification gates;
8
+ - `RISK_REGISTER.md` — current privacy, security and product risks;
9
+ - `DECISION_LOG.md` — accepted architecture/product decisions;
10
+ - `CHANGELOG.md` — historical implementation record;
11
+ - `PROJECT_PROMPT.md` — worker/governance rules.
 
12
 
13
+ Last strategy reset: 2026-09-04 — Repository Convergence becomes the immediate execution priority before further normal feature development. After convergence, `main` becomes the active Scrub Private development line. Local/offline functionality remains recoverable and may become a separate active product only if a later business requirement justifies it.
14
 
15
  ---
16
 
17
+ ## 1. Product objective
18
 
19
+ Scrub helps professionals create reviewed pseudonymised working copies of confidential documents while preserving useful legal and clinical meaning.
20
 
21
+ The product must:
 
 
 
 
 
 
 
 
 
 
 
 
 
22
 
23
+ - detect and pseudonymise sensitive values;
24
+ - preserve professional context;
25
+ - keep meaningful human review mandatory where required;
26
+ - allow users to correct missed values;
27
+ - support controlled re-identification through a sensitive Scrub Key;
28
+ - preserve supported document structure and expose hygiene limitations;
29
+ - make residual risk visible rather than imply perfect automatic detection;
30
+ - ultimately run as a managed private service with a defensible content-handling boundary.
31
 
32
+ Core workflow:
 
 
 
 
33
 
34
  ```text
35
+ Scrub → Review → Scrub Key → AI / external use → Reinsert → Export → Audit
 
 
 
36
  ```
37
 
38
+ Scrub is primarily a pseudonymisation/privacy-processing product. A re-linkable Scrub Key means the output remains re-identifiable and must not be marketed as guaranteed irreversible anonymisation.
 
 
 
 
 
 
 
 
 
 
39
 
40
  ---
41
 
42
+ ## 2. Existing risk priorities remain binding
 
 
43
 
44
+ Repository Convergence does not demote the existing privacy/product risks.
45
 
46
+ Highest priorities remain:
 
 
47
 
48
+ 1. false negatives / missed sensitive data;
49
+ 2. Scrub Key leakage, mismatch or misuse;
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
50
  3. hidden document content and metadata leakage;
51
+ 4. placeholder corruption during external/AI roundtrip;
52
+ 5. review/state defects that can expose stale or insufficiently reviewed output;
53
+ 6. domain-specific under-detection or harmful over-masking, especially Legal and Zorg;
54
+ 7. service trust-boundary failures such as content persistence or unintended external processing.
55
 
56
+ Human review remains a required control. Synthetic benchmarks and tests do not by themselves establish production safety.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
57
 
58
  ---
59
 
60
+ ## 3. Existing architecture to reuse
61
 
62
+ The current repository already contains substantial product architecture and validation capability. The default is reuse, not reinvention.
63
 
64
+ Preserve unless current evidence proves a defect:
 
 
 
 
 
 
 
 
65
 
66
+ - layered Dutch/Legal/Zorg recognition and profile policy;
67
+ - clinical/legal context preservation;
68
+ - side-by-side human review and authoritative replacement decisions;
69
+ - direct/manual missed-value correction;
70
+ - generation-bound Standard/Expert workflow state and fail-closed export gating;
71
+ - bound Scrub Key, mapping digest and controlled reinsert;
72
+ - TXT/DOCX/text-PDF handling and DOCX hygiene/fidelity reporting;
73
+ - synthetic Legal/Zorg corpora, gold expectations, benchmark/gap-triage and E2E evidence.
74
 
75
+ No separate Evidence Framework, workflow engine, mapping service or replacement database is authorized merely by this roadmap.
76
 
77
  ---
78
 
79
+ # 4. Strategic stages
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
80
 
81
+ The strategic roadmap has five macro stages. Detailed implementation packages belong in `WORKPACKAGES.md`.
 
 
 
 
 
82
 
83
+ ## Stage 1 — Repository Convergence — CURRENT
84
 
85
+ Goal:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
86
 
87
+ > Establish one accurate current repository truth before further normal feature development.
88
 
89
+ Sequence:
90
 
91
  ```text
92
+ Preserve exact state
93
+ → reconstruct active/reachable behavior
94
+ → classify capabilities and debt
95
+ → reconcile known current issues
96
+ → remove only proven obsolete/contradictory paths
97
+ → identify canonical validation hierarchy
98
+ → align canonical documentation
99
+ → verify
100
+ → freeze clean baseline
101
  ```
102
 
103
+ Rules:
104
 
105
+ - no source-tree clone (`app_v2`, `scrub-new`, duplicate main Streamlit implementation);
106
+ - exact Git SHA is the preservation authority;
107
+ - classification is capability-level: `CANONICAL | RECONCILE | RETIRE | VARIANT-SPECIFIC`;
108
+ - no refactor for aesthetics alone;
109
+ - no new product feature work by default;
110
+ - privacy/security defects discovered during convergence may be fixed as narrow root-cause packages;
111
+ - existing `implementation_operations` / blind `governance_release_assurance` separation remains mandatory for consequential changes;
112
+ - the temporary debt ledger is execution evidence, not a new permanent source of product truth.
113
 
114
+ Exit milestone:
115
 
116
  ```text
117
+ SCRUB_REPOSITORY_CONVERGED
 
118
  ```
119
 
120
+ Exit requires one exact clean-baseline SHA, aligned current docs/issues, green relevant regression/evidence, and no known material contradiction knowingly left active.
121
 
122
+ ---
123
 
124
+ ## Stage 2 — Scrub Private Application
125
 
126
+ Starts only after `SCRUB_REPOSITORY_CONVERGED`.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
127
 
128
+ Goal:
129
 
130
+ > Reuse the existing Scrub product core while adapting only the application trust boundary required for managed private hosting.
131
 
132
+ ### Content-plane target
133
 
134
+ The following are customer document content:
135
 
136
  ```text
137
+ uploaded document
138
+ extracted text
139
+ detected values
140
+ review/replacement mappings
141
+ processed document
142
+ Scrub Key
143
+ restored document
 
 
 
144
  ```
145
 
146
+ Private application policy:
 
 
 
 
 
 
 
 
 
 
147
 
148
+ - no intentional persistent server-side storage of this content;
149
+ - no document-content logs;
150
+ - no document-content backup;
151
+ - no third-party document-processing egress;
152
+ - application-controlled session state is cleared when the processing interaction ends/reset occurs where the application controls that lifecycle.
153
 
154
+ ### Variant-specific capability
 
 
 
 
 
 
 
 
 
155
 
156
+ Current functionality such as persistent remembered replacements or external Azure/OpenAI document processing may remain recoverable through Git/possible future Local product, but is excluded from Scrub Private unless explicitly re-approved.
157
 
158
+ ### Hugging Face role
159
 
160
+ Hugging Face remains a synthetic/approved-test **application validation environment**, not the final confidential-production trust environment.
161
 
162
+ HF may prove application behavior under our control, such as:
163
 
164
+ - end-to-end Legal/Zorg flows;
165
+ - state integrity and stale-export blocking;
166
+ - no intentional application-level content persistence;
167
+ - no enabled third-party document-processing route;
168
+ - no content-bearing application logging.
169
 
170
+ HF does not prove provider-level snapshot, swap, backup, host-log or forensic-retention properties.
171
 
172
+ Exit milestone:
173
 
174
  ```text
175
+ SCRUB_HF_APPLICATION_COMPLETE
 
 
 
 
 
 
176
  ```
177
 
178
+ This is functional application evidence with synthetic data, not production-security certification.
 
 
179
 
180
+ ---
 
 
 
 
 
 
 
 
181
 
182
+ ## Stage 3 — Private Service
183
 
184
+ Starts only after `SCRUB_HF_APPLICATION_COMPLETE`.
185
 
186
+ Goal:
 
 
 
 
 
187
 
188
+ > Add the smallest production service layer needed for a real managed private deployment.
189
 
190
+ Initial architecture:
191
 
192
  ```text
193
+ Browser
194
+ ↓ HTTPS
195
+ Proven OIDC / customer identity
196
+ ↓
197
+ Scrub service
198
+ ├─ ephemeral content plane
199
+ └─ minimal persistent control plane
200
+ ↓
201
+ User download
202
  ```
203
 
204
+ Control-plane data may include only necessary non-document operational information such as customer/service identity, IdP configuration, deployment version, security/login metadata, service configuration and health status.
 
 
 
 
205
 
206
+ Principles:
207
 
208
+ - use proven OIDC/customer IdP rather than custom authentication;
209
+ - least privilege and non-root runtime;
210
+ - immutable production source/runtime behavior;
211
+ - no persistent customer-document store;
212
+ - no content backup/logging;
213
+ - deny unnecessary outbound network access;
214
+ - avoid request-time model/dependency downloads where practical;
215
+ - manual provisioning is acceptable until automation materially reduces burden;
216
+ - customer-dedicated isolation is the preferred starting hypothesis, but exact VPS/container granularity is decided from real threat/cost/assurance requirements.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
217
 
218
+ Avoid by default:
219
 
220
+ - Kubernetes;
221
+ - service mesh;
222
+ - Redis;
223
+ - queues;
224
+ - custom IAM;
225
+ - generalized multitenancy;
226
+ - billing/orchestration platforms;
227
+ - centralized document storage/processing services.
228
 
229
+ Exit milestone:
230
 
231
  ```text
232
+ SCRUB_PRIVATE_SERVICE_CANDIDATE
 
 
 
 
 
 
233
  ```
234
 
 
 
235
  ---
236
 
237
+ ## Stage 4 — External Product & Service Assurance
238
 
239
+ This is separate from the internal release-assurance role used throughout development.
240
 
241
+ ### Product-effectiveness evidence
 
 
 
242
 
243
+ Reuse existing Scrub evidence:
244
 
245
+ - synthetic Legal/Zorg corpora;
246
+ - canonical benchmark outputs;
247
+ - false-negative/false-positive and preserve/trap evidence;
248
+ - cross-profile regression;
249
+ - Scrub Key/reinsert safety;
250
+ - document hygiene/fidelity evidence;
251
+ - documented limitations and human-review controls.
252
 
253
+ ### Service privacy/security evidence
254
 
255
+ Independently assess:
 
 
 
 
 
 
 
 
256
 
257
+ - authentication/access control;
258
+ - runtime/infrastructure isolation;
259
+ - persistence/deletion behavior;
260
+ - logs and backups;
261
+ - outbound egress;
262
+ - secrets;
263
+ - update/recovery/incident controls;
264
+ - penetration-test findings;
265
+ - accuracy of customer-facing privacy/security claims.
266
 
267
+ Do not create per-document “privacy scores” or false precision.
268
 
269
+ Exit:
270
 
271
  ```text
272
+ material assurance findings closed or explicitly accepted
 
 
 
 
 
 
273
  ```
274
 
275
+ ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
276
 
277
+ ## Stage 5 — Pilot
278
 
279
+ Initial commercial bias:
280
 
281
  ```text
282
+ Legal first
283
+ → observe real workflow/trust friction
284
+ → fix evidence-backed issues
285
+ → Zorg
286
+ → scale only when justified
 
 
 
287
  ```
288
 
289
+ Pilot evidence may justify targeted recognizer/UX/identity/operations improvements.
290
 
291
+ Pilot evidence does not automatically justify a platform rewrite, broad multitenancy, batch/API architecture, dashboards or a new AI/model architecture.
292
 
293
+ ---
 
 
294
 
295
+ # 5. Current execution direction
296
 
297
+ The only active strategic line is **Stage 1 — Repository Convergence**.
 
 
 
 
 
 
 
298
 
299
+ The current executable queue is defined in `WORKPACKAGES.md`.
300
 
301
+ Do not resume historical Premium Input/Review/Export queues, installer work, pilot expansion or service infrastructure work simply because old plans/issues still mention them. Current source and evidence must justify any reopened package.
 
 
302
 
303
+ ---
304
 
305
+ # 6. Validation strategy
306
 
307
+ Scrub already has multiple evidence generations. Convergence must identify the current release-validation hierarchy rather than create another framework.
 
 
 
 
 
 
 
308
 
309
+ A validation path is a candidate for canonical release evidence when it:
310
 
311
+ 1. executes against current supported product code or is intentionally a pure contract test;
312
+ 2. uses current committed synthetic/gold material where applicable;
313
+ 3. measures a currently meaningful risk;
314
+ 4. is reproducibly automated;
315
+ 5. is part of current CI/release validation or has a clear current role;
316
+ 6. states its limitations and does not imply unsupported production safety;
317
+ 7. is not superseded by a later path solving the same responsibility more completely.
318
 
319
+ Classifications:
320
 
321
  ```text
322
+ CANONICAL RELEASE VALIDATION
323
+ SUPPLEMENTAL DIAGNOSTIC
324
+ HISTORICAL / SUPERSEDED
 
 
 
325
  ```
326
 
327
+ Likely complementary current evidence includes:
 
 
328
 
329
+ - full Python regression suite;
330
+ - recognizer-backed corpus benchmark/report workflow;
331
+ - Zorg baseline/recognizer/cross-profile evidence;
332
+ - Phase-6 synthetic E2E workflow matrix;
333
+ - Scrub Key security/roundtrip suites;
334
+ - document hygiene/fidelity suites;
335
+ - Premium Streamlit/AppTest state suites.
336
 
337
+ No new production recall/precision threshold is introduced merely as part of repository cleanup.
338
 
339
+ ---
340
 
341
+ # 7. Future Local product
342
 
343
+ Local/offline capability is preserved as a possible future product direction, not an active delivery line.
344
 
345
+ Do not pre-build a two-edition architecture.
346
 
347
+ If Local later becomes a real maintained product requirement, determine then the smallest shared-core/product-specific structure. Git history preserves currently variant-specific functionality until that requirement exists.
348
 
349
+ ---
 
 
 
 
 
350
 
351
+ # 8. Governing principle
352
 
353
  ```text
354
+ PRESERVE
355
+ → RECONSTRUCT
356
+ → REMOVE ONLY PROVEN DEBT
357
+ → RECONCILE CURRENT TRUTH
358
+ → PROVE
359
+ → FREEZE
360
+ → ADAPT ONLY WHAT PRIVATE REQUIRES
361
+ → HARDEN
362
+ → ASSURE
363
+ → PILOT
364
  ```
365
 
366
+ For every proposed change:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
367
 
368
+ > What concrete current problem disappears if we make this change, and is this the smallest complete way to remove it?
369
 
370
+ If there is no strong answer, do not add the change.
 
 
 
 
 
WORKPACKAGES.md CHANGED
The diff for this file is too large to render. See raw diff
 
handover/workpackages/20260904_2315_repository_convergence_bootstrap.md ADDED
@@ -0,0 +1,278 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Handover — SCRUB-WP_REPOSITORY_CONVERGENCE_BOOTSTRAP
2
+
3
+ Repository: `solidprivacy-nl/scrub`
4
+ Role: `implementation_operations`
5
+ Issue: #113
6
+ PR: #114
7
+ Branch: `wp/repository-convergence-bootstrap`
8
+ Starting main: `54c73e0ebf5a3a3ed7039a50596fb57694add3cd`
9
+ Status: `IMPLEMENTATION_REMEDIATED` — pre-final full regression green; final exact-head CI and fresh independent assurance still required.
10
+
11
+ ## Summary
12
+
13
+ Started the approved Repository Convergence programme without changing Scrub runtime product behavior.
14
+
15
+ The package:
16
+
17
+ - preserved the exact pre-convergence baseline by SHA;
18
+ - confirmed exact-main `Tests` and GitHub→Hugging Face sync were green at package start;
19
+ - reconstructed current reachable architecture far enough to distinguish canonical capability, genuine debt candidates and variant-specific Private conflicts;
20
+ - replaced stale roadmap/workpackage routing with one current Repository Convergence line;
21
+ - aligned worker/governance documentation while preserving the two-role assurance model;
22
+ - converted DECISION_LOG and RISK_REGISTER from historical execution narratives into current-truth documents while preserving exact pre-convergence provenance;
23
+ - archived the complete pre-convergence CHANGELOG byte-for-byte and started a clean post-convergence changelog;
24
+ - added source-level contracts protecting the new current-truth model;
25
+ - received a fresh independent FAIL on the first candidate;
26
+ - remediated only the eight test/governance inconsistencies identified by that reviewer;
27
+ - restored full-suite green without restoring obsolete current queues or changing runtime behavior.
28
+
29
+ ## Baseline preservation
30
+
31
+ Authoritative pre-convergence SHA:
32
+
33
+ ```text
34
+ 54c73e0ebf5a3a3ed7039a50596fb57694add3cd
35
+ ```
36
+
37
+ At package start:
38
+
39
+ - GitHub main: exact SHA above;
40
+ - Tests workflow: success on exact main;
41
+ - Sync to Hugging Face Space: success on exact main.
42
+
43
+ No source clone (`app_v2`, `scrub-new`, duplicate Streamlit main) was created.
44
+
45
+ ## Files added
46
+
47
+ - `REPOSITORY_CONVERGENCE_DEBT_LEDGER.md`
48
+ - `tests/test_repository_convergence_bootstrap_contracts.py`
49
+ - `workpackage_claims/SCRUB-WP_REPOSITORY_CONVERGENCE_BOOTSTRAP.md`
50
+ - `handover/workpackages/20260904_2315_repository_convergence_bootstrap.md`
51
+ - `history/CHANGELOG_PRE_CONVERGENCE_20260904.md` — exact pre-convergence CHANGELOG blob, historical/non-authoritative.
52
+
53
+ ## Files changed
54
+
55
+ Canonical documentation/governance:
56
+
57
+ - `ROADMAP.md`
58
+ - `WORKPACKAGES.md`
59
+ - `PROJECT_PROMPT.md`
60
+ - `PROJECT_PROMPT_SHORT.md`
61
+ - `AGENTS.md`
62
+ - `DECISION_LOG.md`
63
+ - `RISK_REGISTER.md`
64
+ - `CHANGELOG.md`
65
+
66
+ Governance/contract tests remediated after independent FAIL:
67
+
68
+ - `tests/test_repository_convergence_bootstrap_contracts.py`
69
+ - `tests/test_mvp_document_fidelity_pr_final_contracts.py`
70
+ - `tests/test_premium_core_flow_ui_realignment_plan.py`
71
+ - `tests/test_premium_staged_workspace_decision.py`
72
+ - `tests/test_reinsert_auto_flow_app_verify_closeout.py`
73
+
74
+ No runtime/product source file is changed by PR #114.
75
+
76
+ ## Current audit findings
77
+
78
+ ### Canonical capability to preserve
79
+
80
+ - Premium staged workspace and generation-bound source/analysis/review/export state;
81
+ - mandatory review authority, review table and direct/manual correction;
82
+ - bound Scrub Key + mapping digest + fail-closed reinsert validation;
83
+ - Legal/Dutch and Zorg recognition/profile policy;
84
+ - TXT/DOCX/text-PDF handling and DOCX hygiene/fidelity boundaries;
85
+ - existing recognizer benchmark, domain evidence, Phase-6 E2E, Scrub Key/document suites and Premium AppTests.
86
+
87
+ ### Variant-specific / Private-incompatible capability
88
+
89
+ - persistent remembered replacements (`replacement_memory.py` and Expert UI);
90
+ - Azure AI Language external document recognition;
91
+ - OpenAI/Azure synthesis;
92
+ - content-bearing synthesis prompt logging.
93
+
94
+ These are recorded but intentionally not changed in this Stage-1 bootstrap. Their Private disposition belongs after the clean baseline unless current evidence proves an immediate reference-environment safety defect.
95
+
96
+ ### Retire candidates
97
+
98
+ - Docker still invokes `fix_streamlit_nested_expanders.py` and `fix_streamlit_pdf_text_reinsert.py` even though current Premium/direct-source markers make both exit without mutating source.
99
+
100
+ This requires an isolated proof/retirement package, not deletion in the bootstrap.
101
+
102
+ ### Evidence-authority reconciliation
103
+
104
+ - current recognizer-backed recall workflow appears to be the primary automated recognition diagnostic path;
105
+ - older entity-scorecard/residual-risk helpers remain useful within explicit report-only limitations;
106
+ - Phase-6 E2E, Zorg, Scrub Key/document and Premium AppTest evidence are complementary;
107
+ - a narrow hierarchy package should classify existing paths as canonical release validation, supplemental diagnostic or historical/superseded without building a new framework or inventing thresholds.
108
+
109
+ ### GitHub issue truth
110
+
111
+ - #112 and #109 contain independent PASS evidence for PR #111/#108, both of which are merged; these are stale as active candidate gates;
112
+ - #107/#106 correspond to merged child repairs and require stale-state reconciliation;
113
+ - #105/#96 additionally required a final consolidated deployed live retest after those repairs. No evidence found in this implementation session proves that final retest occurred. Do not falsely close it as verified.
114
+
115
+ ## Independent assurance history
116
+
117
+ Fresh blind assurance reviewed exact prior head:
118
+
119
+ ```text
120
+ ea203abb04f008a7e583387242a6f4917c72e591
121
+ ```
122
+
123
+ against base:
124
+
125
+ ```text
126
+ 54c73e0ebf5a3a3ed7039a50596fb57694add3cd
127
+ ```
128
+
129
+ and returned:
130
+
131
+ ```text
132
+ FAIL
133
+ ```
134
+
135
+ Reviewer findings:
136
+
137
+ 1. exact-head `Tests` run `33920335521` was red: `8 failed, 1256 passed`;
138
+ 2. two new convergence tests encoded brittle string/prose placement instead of semantic governance contracts;
139
+ 3. six legacy tests still required historical Premium/fidelity/reinsert state to remain in current ROADMAP/WORKPACKAGES/CHANGELOG positions.
140
+
141
+ The reviewer also independently found:
142
+
143
+ - scope integrity good;
144
+ - five-stage roadmap materially coherent/minimal;
145
+ - still-binding product semantics materially preserved;
146
+ - archived pre-convergence changelog preservation appropriate;
147
+ - no runtime/HF/product activation occurred.
148
+
149
+ No prior verdict transfers to a changed head.
150
+
151
+ ## Remediation performed
152
+
153
+ Only tests/governance contracts were changed.
154
+
155
+ ### New convergence-contract repair
156
+
157
+ - stage validation now parses formal `## Stage N — ...` headings and compares the exact ordered five-stage set;
158
+ - Evidence Framework prohibition now checks a stable semantic prohibition once instead of requiring duplicate prose in multiple files.
159
+
160
+ ### Historical fidelity contract repair
161
+
162
+ - historical completion/no-duplicate evidence is read from `history/CHANGELOG_PRE_CONVERGENCE_20260904.md` and the historical handover;
163
+ - current supported DOCX scope is bound to D030 in `DECISION_LOG.md`;
164
+ - current WORKPACKAGES no longer needs to carry the historical fidelity package.
165
+
166
+ ### Premium UI contract repair
167
+
168
+ - plan/decision artifacts still protect the implemented one-document/staged-workspace semantics;
169
+ - current authority is bound to D041/D043 and the current shared-surface sequencing rule;
170
+ - obsolete Premium package names/order are explicitly not required in the current queue.
171
+
172
+ ### Reinsert contract repair
173
+
174
+ - historical app-verification evidence is read from the archived changelog/claim/handover;
175
+ - current document-first flow is bound to D031;
176
+ - current fail-closed document/key binding is bound to D037;
177
+ - old roadmap/workpackage placement is no longer release authority.
178
+
179
+ ## Validation evidence
180
+
181
+ ### Initial exact-head candidate — FAIL evidence
182
+
183
+ Head:
184
+
185
+ ```text
186
+ ea203abb04f008a7e583387242a6f4917c72e591
187
+ ```
188
+
189
+ Tests run:
190
+
191
+ ```text
192
+ 33920335521
193
+ ```
194
+
195
+ Result:
196
+
197
+ ```text
198
+ 8 failed, 1256 passed in 12.64s
199
+ ```
200
+
201
+ ### Remediated pre-final candidate — GREEN
202
+
203
+ Head:
204
+
205
+ ```text
206
+ dcdfb2c84bbaafe0beca79191bee509b9607b461
207
+ ```
208
+
209
+ PR merge candidate checked out by Actions:
210
+
211
+ ```text
212
+ ca222802fe5a5c21213df1ed71045fee0826c4ff
213
+ ```
214
+
215
+ Tests run:
216
+
217
+ ```text
218
+ 33922952965
219
+ ```
220
+
221
+ Job:
222
+
223
+ ```text
224
+ 101185112554
225
+ ```
226
+
227
+ Raw result:
228
+
229
+ ```text
230
+ 1264 passed in 12.34s
231
+ ```
232
+
233
+ This proves the reviewer-requested test/governance remediation is green.
234
+
235
+ ### Final exact-head status
236
+
237
+ Administrative claim/handover/changelog/workpackage updates follow this green run and therefore change the branch head. A final exact-head full regression is still mandatory. The final frozen SHA and run must be recorded in the PR conversation without mutating the head again.
238
+
239
+ ## Hugging Face sync status
240
+
241
+ - Baseline exact-main sync before this workpackage: success.
242
+ - PR #114 contains no runtime product change.
243
+ - No candidate deployment/app verification is required before merge for this docs/governance/test-only package.
244
+ - After authorized merge, verify exact-main Actions and HF sync/path-ignore behavior according to the runbook.
245
+
246
+ ## App verification status
247
+
248
+ Not applicable for PR #114: no product/UI/runtime behavior changes.
249
+
250
+ ## Intentionally unchanged
251
+
252
+ - recognizers/profile/threshold semantics;
253
+ - review/include/replacement authority;
254
+ - export payloads, filenames and MIME types;
255
+ - Scrub Key schema/binding/lifecycle;
256
+ - reinsert behavior;
257
+ - document-processing behavior;
258
+ - runtime dependencies;
259
+ - Docker/Streamlit runtime product behavior;
260
+ - current replacement-memory/external-provider functionality;
261
+ - mandatory human review.
262
+
263
+ ## Remaining risks / blockers
264
+
265
+ - Final administrative head must complete the full `Tests` workflow successfully.
266
+ - A completely fresh independent assurance worker must review that exact new frozen SHA and return PASS before merge.
267
+ - Prior FAIL is correctly non-transferable to the new head, and any future PASS is likewise exact-head only.
268
+ - The later technical cleanup queue must remain audit-derived rather than pre-invented.
269
+ - Parent Premium live-verification state (#105/#96) is not yet proven closed.
270
+
271
+ ## Next recommended step
272
+
273
+ 1. Complete final administration without runtime changes.
274
+ 2. Run full exact-head CI.
275
+ 3. Freeze the exact final PR head and post the raw run/job/pass count to PR #114 without changing source.
276
+ 4. Dispatch fresh blind `governance_release_assurance` on that exact SHA.
277
+ 5. Merge only on PASS; verify exact-main Actions/HF sync handling.
278
+ 6. Continue with evidence-backed convergence packages derived from the ledger.
history/CHANGELOG_PRE_CONVERGENCE_20260904.md ADDED
@@ -0,0 +1,2256 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ## 2026-08-09 23:51 Europe/Amsterdam — SCRUB-WP_REVIEW_MARKER_COMPACTION_LIVE_REGRESSION_REPAIR
2
+
3
+ Status: implementation candidate in PR #108; functional full-suite CI green; final exact-head CI and fresh blind assurance required.
4
+
5
+ Live verification after the independently PASSed/deployed Premium App Shell V2 found yellow processed-text markers shifted into neighboring ordinary text and full document-binding tokens visible instead of compact aliases. Source reconstruction confirmed one display root cause: `find_exact_highlight_spans()` normalized the entire processed document with `.strip()` before computing offsets while the renderer used the original untrimmed text. Leading DOCX/PDF whitespace therefore shifted all marker coordinates and could split a strict bound token so the compact-display segmenter could no longer recognize it.
6
+
7
+ Repair: document/processed text now retains exact outer whitespace for coordinate calculations; scalar review-cell normalization remains trimmed. Added synthetic regressions for leading whitespace, exact full-token spans, highlighted compact `[LABEL_NN]` display and marker-off compaction. No recognizer, review-authority, export, Scrub Key, reinsert or audit semantics changed.
8
+
9
+ Functional evidence: head `a9e977b2ad21ff4f5c86d405d56ebce60cafe3ec`; merge candidate `5b9af3a09519128d40b842b50fe1421b289ea2bd`; Tests #2279 / run `31337824222`, job `93306469803`; Streamlit 1.61.1; `1247 passed in 12.80s`.
10
+
11
+ Separate live finding: Dutch address spans around `Polderweg 8` absorb adjacent ordinary words and proliferate near-duplicate review rows. This is isolated in issue #107 and is intentionally not repaired in PR #108.
12
+
13
+ Governance transparency: two accidental empty `operator_triggers/.keep` adds on main were each immediately followed by clean-tree fast-forward commits restoring the exact prior repository tree; final tree delta is zero and the path is excluded from HF sync. Details are in the #106 handover.
14
+
15
+ ---
16
+
17
+ ## 2026-08-08 21:50 Europe/Amsterdam — SCRUB-WP_PREMIUM_APP_SHELL_POST_MERGE_STATE_REPAIR_V2 — executable fail-closed Expert export repair
18
+
19
+ Status: `IMPLEMENTATION_IN_PROGRESS` until the post-administration exact-head full-suite run is green; fresh independent assurance remains mandatory.
20
+
21
+ Reason:
22
+ - independent assurance issue #101 correctly returned `FAIL` on PR #99 because a real Expert review edit cleared review/export lineage but the same render still exposed document, Scrub Key and audit downloads;
23
+ - the same review also found that prior evidence lacked an executable Streamlit transition test.
24
+
25
+ Repair:
26
+ - all existing export/download controls now sit behind current `CoreFlowState.export_is_current` lineage;
27
+ - a real Expert review edit immediately blocks the entire downstream export surface;
28
+ - Expert exposes one explicit `Controle opnieuw afronden` action only when current processing is still reviewable;
29
+ - review is never silently auto-completed after edits;
30
+ - real source/processing-setting changes remain blocked until reprocessing and cannot use the review-recompletion shortcut;
31
+ - authoritative edited review rows remain generation-bound and survive Standard/Expert presentation switches;
32
+ - product export payload bytes, filenames and MIME semantics are unchanged once eligibility is current.
33
+
34
+ Executable regression:
35
+ - added `streamlit.testing.v1.AppTest` coverage for Standard Download → Expert → review edit → export blocked → explicit review completion → Standard with source/generation/analysis/review state preserved;
36
+ - added executable processing-setting invalidation coverage;
37
+ - added executable Expert-only `highlight` preservation coverage;
38
+ - CI now installs Streamlit for the test environment.
39
+
40
+ Pre-administration machine evidence:
41
+ - PR #104 head `d688cc3b980da63c3c3f0bfb98e990a00b746c4b`;
42
+ - base main `2623524c858216318d238213e37445193510fa73`;
43
+ - tested merge candidate `abeb56ee683fb4c97e0c2fef7786ed65c825b582`;
44
+ - Tests #2261 / run `31275276471`, job `93147717998`;
45
+ - raw checkout confirms the exact merge candidate;
46
+ - Streamlit `1.61.1` installed in CI;
47
+ - `python -m pytest -q tests` → `1244 passed in 13.89s`.
48
+
49
+ Protected semantics intentionally unchanged:
50
+ - recognizers/profile rules and threshold meaning;
51
+ - authoritative include/replacement decisions and direct masking;
52
+ - eligible export bytes, filenames and MIME types;
53
+ - Scrub Key schema/binding/lifecycle;
54
+ - reinsert and audit semantics;
55
+ - local/cloud processing boundary;
56
+ - mandatory human review.
57
+
58
+ Final exact-head CI is mandatory after this administration. Implementation does not self-certify or self-merge. Parent issue #96 and Premium Input Stage remain blocked pending fresh assurance, merge, deployment evidence and live app verification.
59
+
60
+ ---
61
+
62
+ ## 2026-08-08 20:35 Europe/Amsterdam — SCRUB-WP_PREMIUM_APP_SHELL_POST_MERGE_STATE_REPAIR
63
+
64
+ Status: `IMPLEMENTATION_IN_PROGRESS`; issue #98 / PR #99 repairs the reproduced issue #92 cross-mode source, analysis-cache and review-state defects. Standard and Expert now share authoritative source context, current-generation analysis and generation-bound review rows. Real source/processing/review changes continue to invalidate downstream state fail-closed.
65
+
66
+ Pre-administration evidence: Tests #2247 / run `31272188513`, job `93139919879`, head `69fde971783aa20d5cbcb2c13cd8a8538fa1ebe4`, merge candidate `65ca1ab73f03a2513a2004d49f190c2a233d261e`, `1240 passed in 9.65s`.
67
+
68
+ Fresh exact-head CI and independent assurance remain mandatory. Issue #96 and Premium Input Stage remain blocked.
69
+
70
+ ---
71
+
72
+ ## 2026-08-08 16:55 Europe/Amsterdam — SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION — repair after blind-assurance FAIL
73
+
74
+ Status: `RELEASE_CANDIDATE_READY` only after a fresh exact-head full-suite run; fresh independent assurance remains mandatory.
75
+
76
+ Independent assurance on prior head `2b04ca6260bddee07fbcf901239cee2955bd6dc7` returned `FAIL` because switching `Standaard → Expert` could silently reset an explicitly selected Zorg profile to the hard-coded Expert default Juridisch. The same repair cycle reviewed operator, threshold, entity selection, allow/deny lists and analyzer configuration as processing-affecting state.
77
+
78
+ Repair:
79
+ - Standard and Expert now hydrate the same persisted processing settings;
80
+ - entering Expert no longer uses a hard-coded recognition-profile or operator default when a valid current choice exists;
81
+ - returning to Standard rehydrates the Standard profile widget from the shared processing state;
82
+ - threshold, entity selection, allow/deny lists and analyzer/model settings are preserved across presentation-only switches;
83
+ - deterministic processing-generation synchronization now runs in both presentation modes;
84
+ - presentation-only switching with unchanged settings preserves valid downstream lineage;
85
+ - a real processing-affecting Expert change invalidates processed/review/export lineage fail-closed;
86
+ - Standard still refuses unsupported Expert-only operators without silently rewriting them.
87
+
88
+ Regression evidence before administrative finalization:
89
+ - GitHub Actions Tests run #2229 / ID `31263099583`, job `93116829430`;
90
+ - PR merge candidate `bd3c404f7075d86620272bf28c9a5192006e8209`;
91
+ - `python -m pytest -q tests` → `1235 passed in 14.48s`;
92
+ - new coverage explicitly includes Standard Zorg → Expert → Standard state preservation and real-change invalidation.
93
+
94
+ Protected semantics unchanged:
95
+ - recognizers/profile rules and threshold meaning;
96
+ - review-table/include authority and direct masking;
97
+ - export bytes, filenames and MIME types;
98
+ - Scrub Key, reinsert and audit semantics;
99
+ - dependencies and local/cloud processing boundary;
100
+ - mandatory human review.
101
+
102
+ A fresh exact-head regression is required after this administrative closeout, followed by a new blind `governance_release_assurance` review. The later Premium Input Stage package remains blocked.
103
+
104
+ ---
105
+
106
+ ## 2026-08-08 15:28 Europe/Amsterdam — SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION — release candidate prepared
107
+
108
+ Status: `RELEASE_CANDIDATE_READY`; final exact-head CI and fresh independent assurance required before merge.
109
+
110
+ Purpose:
111
+ - move the merged Premium staged-workspace architecture from helper/state primitives into the production Streamlit shell;
112
+ - make Standard behave as one persistent document workspace with `Toevoegen → Controleren → Downloaden` and exactly one dominant stage;
113
+ - preserve Expert and all existing processing/review/export/Scrub Key/reinsert/audit semantics.
114
+
115
+ Implemented:
116
+ - global `Anonimiseren | Terugzetten` and `Standaard | Expert` controls;
117
+ - persistent application-panel stage headers with active/completed/future states;
118
+ - compact completed summaries and passive future stages;
119
+ - primary `Document verwerken` and `Controle afronden` progression actions;
120
+ - explicit earlier-stage return;
121
+ - deterministic processing lineage and fail-closed downstream invalidation;
122
+ - current-generation analysis cache to prevent silent recognition reruns during stage navigation;
123
+ - cached review rows; reopening a completed Review invalidates Download until Review is explicitly completed again;
124
+ - Standard hides the permanent settings sidebar while Expert retains advanced controls;
125
+ - Standard does not silently modify Expert-only `highlight`/`synthesize` operator choices;
126
+ - legacy runtime source patch exits for the direct Premium shell, preventing the retired form/two-mode UI from being re-injected at startup.
127
+
128
+ Files added:
129
+ - `premium_streamlit_state.py`;
130
+ - `premium_streamlit_shell_ui.py`;
131
+ - `tests/test_premium_streamlit_state.py`;
132
+ - `tests/test_premium_app_shell_streamlit_integration.py`;
133
+ - implementation claim and handover.
134
+
135
+ Files changed:
136
+ - `premium_app_shell.py`;
137
+ - `presidio_streamlit.py`;
138
+ - `fix_streamlit_nested_expanders.py`;
139
+ - source-level UI contract tests updated where literal expectations described the retired shell;
140
+ - `WORKPACKAGES.md`, `CHANGELOG.md`, `RELEASE_NOTES.md` for closeout administration.
141
+
142
+ Validation:
143
+ - clean runtime/product head `0e1a5fbb3d6c3b8f8293779e598ececd6ea4aa1d`;
144
+ - GitHub Actions Tests #2200 / run `31259576962`, job `93108182555`;
145
+ - `python -m pytest -q tests` → `1225 passed in 12.55s`;
146
+ - final post-administration exact-head full regression still required before assurance;
147
+ - Hugging Face sync and live app verification are pending after PASS/merge because runtime UI changed.
148
+
149
+ Intentionally unchanged:
150
+ - recognizers/profile rules and threshold meaning;
151
+ - replacement/include authority and direct masking semantics;
152
+ - export bytes, filenames and MIME types;
153
+ - Scrub Key schema/binding/lifecycle;
154
+ - reinsert and audit semantics;
155
+ - dependencies and cloud/local-processing boundary;
156
+ - mandatory human review.
157
+
158
+ Next gate:
159
+ - freeze final exact head after this administrative closeout;
160
+ - full GitHub Actions regression;
161
+ - fresh blind `SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION_VERIFY`;
162
+ - no merge and no Input Stage work before independent PASS.
163
+
164
+ ---
165
+
166
+ ## 2026-08-08 Europe/Amsterdam — SCRUB-WP_PREMIUM_STAGED_WORKSPACE_DECISION_FREEZE — staged workspace architecture candidate
167
+
168
+ Status: `IMPLEMENTATION_IN_PROGRESS`; exact-head CI and independent assurance required before merge.
169
+
170
+ Purpose:
171
+ - convert the coordinator-approved first-principles comparison of three routed screens versus a one-page staged workspace into a binding Premium UI architecture decision;
172
+ - intervene before PR #85 enters production `presidio_streamlit.py` integration;
173
+ - make the Premium staged-workspace sequence the authoritative active execution queue.
174
+
175
+ Decision:
176
+ ```text
177
+ One document. One workspace. Three stages. One active task.
178
+
179
+ Toevoegen → Controleren → Downloaden
180
+ ```
181
+
182
+ Binding Standard behavior:
183
+ - all three stage headers remain in one persistent page/workspace;
184
+ - exactly one stage is expanded/dominant at a time;
185
+ - completed stages collapse to compact status summaries;
186
+ - future stages remain visible but passive;
187
+ - successful completion auto-advances to the next stage;
188
+ - deliberate return to an earlier stage remains possible;
189
+ - processing-affecting earlier changes invalidate processed/review/export lineage fail-closed;
190
+ - three separate routed pages are rejected for these core stages;
191
+ - a classic nested-expander form is also rejected as the default Standard pattern.
192
+
193
+ First-principles rationale:
194
+ - Scrub is an iterative document-review workflow, not a strictly linear checkout;
195
+ - the interface should represent document state rather than page navigation;
196
+ - the staged workspace preserves one-task-at-a-time focus while retaining document identity, progress, correction context and visible trust state;
197
+ - it aligns directly with the merged `premium_core_flow_state.py` generation/invalidation model and reduces routing/state-restoration complexity in Streamlit.
198
+
199
+ Current PR #85 consequence:
200
+ - PR #85 is amended, not discarded;
201
+ - current pure `premium_app_shell.py` helper work remains reusable;
202
+ - production Streamlit integration is gated until this decision is independently assured and incorporated;
203
+ - PR #85 must add/test completed/future/active panels, compact summaries, auto-progression hooks, prior-stage return/edit and no-three-page semantics before production integration.
204
+
205
+ Files added/changed in the candidate:
206
+ - added `PREMIUM_STAGED_WORKSPACE_DECISION.md`;
207
+ - updated `ROADMAP.md`;
208
+ - updated `WORKPACKAGES.md` with an authoritative current Premium queue while retaining older content as historical records;
209
+ - updated `DECISION_LOG.md` with D043;
210
+ - updated `CHANGELOG.md`;
211
+ - added `tests/test_premium_staged_workspace_decision.py`;
212
+ - added implementation claim and handover administration.
213
+
214
+ Validation boundary:
215
+ - documentation/architecture/contract-only package;
216
+ - no production Streamlit or runtime product behavior changes;
217
+ - no recognizer, threshold, replacement, review authority, export bytes/names/MIME, Scrub Key, reinsert, audit, dependency or Hugging Face behavior changes;
218
+ - Hugging Face sync: not applicable;
219
+ - app verification: not applicable;
220
+ - exact-head GitHub Actions and independent `governance_release_assurance` remain required before merge.
221
+
222
+ ---
223
+
224
+ ## 2026-08-07 Europe/Amsterdam — Issue #70 Actions evidence recovery repair
225
+
226
+ Status: `RELEASE_CANDIDATE_READY`; independent exact-head assurance pending.
227
+
228
+ - Replaced PR #73's expired 2026-06-17 carrier dependency with a purpose-built read-only no-op carrier workflow.
229
+ - Demonstrated current connector executability: carrier run `31216068355` was rerun successfully as attempt 2; latest rerun job `92989859101` concluded `success`.
230
+ - Full PR regression run `31216068325` / #2115 passed all `1170` tests in `11.00s` using the unchanged `python -m pytest -q tests` command.
231
+ - Strengthened the workflow contract tests to freeze carrier safety, rerun gating, default-branch checkout semantics and unchanged full-suite behavior.
232
+ - No recognizer, review, replacement, export, Scrub Key, reinsert, UI, document-processing, dependency or Hugging Face product behavior changed.
233
+ - Fresh blind `governance_release_assurance` remains mandatory for the final exact PR head before merge; issue #70 stays open until independent exact-main post-merge evidence is confirmed.
234
+
235
+ ## 2026-08-06 21:30 Europe/Amsterdam — SCRUB-WP_TWO_ROLE_GOVERNANCE_ADOPTION_VERIFY / SCRUB-WP_PROCESSED_TEXT_SELECTION_CROSS_FLOW_REGRESSION_VERIFY
236
+
237
+ Status: both initial assurance decisions `PASS`; PR #69 merged unchanged; post-action closeout remains `ACTION_EXECUTED_UNVERIFIED` because no distinct Actions push run on the actual merged SHA was observable.
238
+
239
+ Purpose:
240
+ - independently reconstruct and decide the two verification packages under issue #70;
241
+ - preserve the blind-review boundary before reading implementation handovers, claims or conclusions;
242
+ - merge the passing candidate without assurance-side repair;
243
+ - record separate verification claims and handovers.
244
+
245
+ Assurance decisions:
246
+ - `SCRUB-WP_TWO_ROLE_GOVERNANCE_ADOPTION_VERIFY`: `PASS`;
247
+ - `SCRUB-WP_PROCESSED_TEXT_SELECTION_CROSS_FLOW_REGRESSION_VERIFY`: `PASS`.
248
+
249
+ Action and evidence:
250
+ - candidate head: `41bf09abe3966ae40a51c526d162c57a824557e8`;
251
+ - tested merge candidate: `13d55b6d74ad6f31446e16bcad0794abea32f9e7`;
252
+ - raw run #2105 / ID `31091265208`: `1165 passed in 12.41s`;
253
+ - actual merge commit: `07cf12d0cfa57dd81bd2c964cb081e5f8f8c4f71`;
254
+ - actual and tested merge candidates have identical parents and identical tree `4c993cbed86eade252cec6799f7dae5919b84085`;
255
+ - candidate repairs: none.
256
+
257
+ Files added:
258
+ - `workpackage_claims/scrub_wp_two_role_governance_adoption_verify.md`;
259
+ - `workpackage_claims/scrub_wp_processed_text_selection_cross_flow_regression_verify.md`;
260
+ - `handover/workpackages/20260806_2130_two_role_governance_adoption_verify.md`;
261
+ - `handover/workpackages/20260806_2130_processed_text_selection_cross_flow_regression_verify.md`.
262
+
263
+ Files changed:
264
+ - `WORKPACKAGES.md`;
265
+ - `CHANGELOG.md`.
266
+
267
+ Validation:
268
+ - implementation handovers and claims were opened only after the two initial decisions and were administratively complete;
269
+ - no distinct GitHub Actions push run on merged SHA `07cf12d0cfa57dd81bd2c964cb081e5f8f8c4f71` was observable at closeout-writing time, so `OUTCOME_CONFIRMED` is not claimed;
270
+ - Hugging Face sync: not applicable because no runtime file changed and all changed paths are ignored by the sync workflow;
271
+ - app verification: not applicable because no UI behavior changed.
272
+
273
+ Intentionally not changed:
274
+ - candidate source, tests or candidate head;
275
+ - production Python, Streamlit, frontend, runtime, dependency or deployment behavior;
276
+ - recognizers, review semantics, exports, Scrub Key, reinsert or audit semantics;
277
+ - human-review and local-processing boundaries.
278
+
279
+ ## 2026-08-06 11:37 Europe/Amsterdam — SCRUB-WP_TWO_ROLE_GOVERNANCE_ADOPTION / SCRUB-WP_PROCESSED_TEXT_SELECTION_CROSS_FLOW_REGRESSION
280
+
281
+ Status: implementation `RELEASE_CANDIDATE_READY`; GitHub Actions PR #69 run #2097 green (`1165 passed in 9.62s`); independent governance assurance pending.
282
+
283
+ Purpose:
284
+ - adopt the Weekly ETF donor's canonical implementation-versus-release-assurance model for Scrub;
285
+ - strengthen it with a blind-review boundary so the assurance worker does not read implementation conclusions before its initial decision;
286
+ - add end-to-end synthetic regression evidence from processed-text selection commit through exports, bound Scrub Key, TXT/DOCX reinsert and audit outputs.
287
+
288
+ Files added:
289
+ - `control/PROJECT_GOVERNANCE_BOOTSTRAP.md`;
290
+ - `control/SCRUB_RELEASE_ASSURANCE_CONTRACT_V1.md`;
291
+ - `PROCESSED_TEXT_SELECTION_CROSS_FLOW_REGRESSION.md`;
292
+ - `tests/test_processed_text_selection_cross_flow_regression.py`;
293
+ - implementation claim and handover files.
294
+
295
+ Files changed:
296
+ - `PROJECT_PROMPT.md`;
297
+ - `PROJECT_PROMPT_SHORT.md`;
298
+ - `ROADMAP.md`;
299
+ - `WORKPACKAGES.md`;
300
+ - `DECISION_LOG.md`;
301
+ - `CHANGELOG.md`.
302
+
303
+ Regression coverage:
304
+ - selection-created row provenance, binding, all-exact occurrence count and authoritative include state;
305
+ - processed text/TXT export;
306
+ - original-DOCX replacement path;
307
+ - schema-1.1 bound Scrub Key generation and fail-closed custom-text behavior;
308
+ - verified TXT and DOCX reinsert;
309
+ - replacement CSV and scrub-report audit evidence;
310
+ - local-only/no-AI/no-cloud metadata.
311
+
312
+ Validation:
313
+ - local execution unavailable in this connector-only session;
314
+ - GitHub Actions PR #69 run #2097: `1165 passed in 9.62s`;
315
+ - independent governance decisions deliberately not issued by implementation.
316
+
317
+ Intentionally not changed:
318
+ - production Python/Streamlit behavior;
319
+ - recognizers, profiles, thresholds or review decisions;
320
+ - export bytes, filenames or MIME types;
321
+ - Scrub Key schema/binding or reinsert semantics;
322
+ - audit semantics, dependencies, runtime or Hugging Face app behavior.
323
+
324
+ ## 2026-08-05 10:49 Europe/Amsterdam — SCRUB-WP_PREMIUM_CORE_FLOW_UI_REALIGNMENT_PLAN
325
+
326
+ Status: completed planning/design-only.
327
+
328
+ Purpose:
329
+ - Convert new live-app UX evidence into a structural interface strategy rather than another isolated decluttering patch.
330
+
331
+ Result:
332
+ - added `PREMIUM_CORE_FLOW_UI_REALIGNMENT_PLAN.md`;
333
+ - documented the gap between the implemented review-local Basic/Expert split and an application-wide presentation model;
334
+ - defined top-level Anonymize/Reinsert workflows;
335
+ - defined global Standard/Expert visibility and grouping;
336
+ - defined one-active-stage `Toevoegen → Controleren → Downloaden` behavior;
337
+ - defined no permanent settings sidebar in Standard;
338
+ - defined one recommended document download with other formats, Scrub Key and audit evidence secondary;
339
+ - sequenced contract, pure state model, app shell, input, review, export, Expert parity and app verification packages;
340
+ - added planning contract tests;
341
+ - changed no runtime product behavior.
342
+
343
+ Intentionally not changed:
344
+ - Streamlit product code;
345
+ - recognizers, profiles, replacement logic or review decisions;
346
+ - export payloads, filenames or MIME types;
347
+ - Scrub Key or reinsert semantics;
348
+ - runtime, dependencies or cloud-processing boundaries.
349
+
350
+ ## 2026-08-05 10:49 Europe/Amsterdam — SCRUB-WP_BOUND_PLACEHOLDER_DISPLAY_COMPACTION_APP_VERIFY_CLOSEOUT
351
+
352
+ Status: completed; deployment and live app verification green.
353
+
354
+ Validation:
355
+ - PR #66 merged as `74b7a15ee74f6330f7fc37892b65246c1a61afaf`;
356
+ - final run #2080: 1155 tests passed in 12.44s;
357
+ - deployment run #2082: 4/4 files exact, Space health `ok`, root HTTP 200, frontend tests passed and 1155 tests passed in 11.49s;
358
+ - coordinator/user confirmed shorter replacement codes are visible and working.
359
+
360
+ Intentionally not changed:
361
+ - full bound tokens in exports;
362
+ - 80-bit binding entropy;
363
+ - Scrub Key, reinsert or export semantics;
364
+ - human-review and production-readiness boundaries.
365
+
366
+ ## 2026-08-04 22:22 Europe/Amsterdam — SCRUB-WP_PROCESSED_TEXT_SELECTION_APP_VERIFY_CLOSEOUT
367
+
368
+ Status: completed; deployment and live app verification green.
369
+
370
+ Purpose:
371
+ - Close the processed-text selection integration after independent deployment checks and coordinator/user browser verification.
372
+
373
+ Validation:
374
+ - PR #63 merged as `53fad202ae88a97b1ea476a9c3ba787932cd62ae`;
375
+ - final merge-candidate run #2051: 1146 tests passed in 11.59s;
376
+ - independent deployment run #2064: 11/11 runtime/component files matched Hugging Face, Space health was `ok`, root returned HTTP 200, frontend tests passed, and 1146 Python tests passed in 11.47s;
377
+ - coordinator/user confirmed `Het werkt.` with a deployed-app screenshot.
378
+
379
+ Observed follow-up:
380
+ - bound placeholders remain safe but visually noisy because the same 80-bit binding ID is repeated;
381
+ - four-character binding IDs are rejected as an unsafe weakening;
382
+ - display-only compaction is routed to `SCRUB-WP_BOUND_PLACEHOLDER_DISPLAY_COMPACTION`.
383
+
384
+ Intentionally not changed:
385
+ - product code, placeholder grammar and binding entropy;
386
+ - export, Scrub Key, reinsert, recognizers, profiles or dependencies;
387
+ - production-readiness and human-review boundaries.
388
+
389
+ ## 2026-08-04 01:34 Europe/Amsterdam — SCRUB-WP_PROCESSED_TEXT_SELECTION_TABLE_INTEGRATION
390
+
391
+ Status: completed in GitHub; deployment and app verification pending.
392
+
393
+ Purpose:
394
+ - Make direct selection in `Verwerkte tekst` a safe input route into the existing manual replacement-table path.
395
+
396
+ Files added:
397
+ - `processed_text_selection_integration.py`
398
+ - `tests/test_processed_text_selection_integration.py`
399
+ - `tests/test_processed_text_selection_table_integration_contract.py`
400
+ - `PROCESSED_TEXT_SELECTION_TABLE_INTEGRATION.md`
401
+ - `workpackage_claims/scrub_wp_processed_text_selection_table_integration.md`
402
+ - `handover/workpackages/20260804_0134_processed_text_selection_table_integration.md`
403
+
404
+ Files changed:
405
+ - `processed_text_selection_component.py`
406
+ - `side_by_side_review_panel_ui.py`
407
+ - `presidio_streamlit.py`
408
+ - `PROCESSED_TEXT_SELECTION_MASKING_CONTRACT.md`
409
+ - `ROADMAP.md`
410
+ - `WORKPACKAGES.md`
411
+ - `RELEASE_NOTES.md`
412
+ - `RISK_REGISTER.md`
413
+
414
+ Implementation:
415
+ - promoted the local component wrapper while retaining the isolated spike alias;
416
+ - added interactive review with environment-controlled and exception-safe static fallback;
417
+ - added server-protected spans independent of marker visibility;
418
+ - processed inspect/commit only after the editable table and before serial review/export;
419
+ - appended one bound normal manual row through document-scoped `manual_mask_rows`;
420
+ - added immediate reruns, feedback and safe one-step undo;
421
+ - blocked undo when the visible table row was subsequently edited;
422
+ - retained the full review table and existing manual form.
423
+
424
+ Validation:
425
+ - cleanup run #2044: frontend component tests passed; two obsolete phase-status assertions failed in Python only;
426
+ - final clean PR run #2047: 1146 Python tests passed in 11.97s;
427
+ - Hugging Face sync and live app verification pending.
428
+
429
+ Intentionally not changed:
430
+ - all-exact version-one scope;
431
+ - export formats, filenames, MIME or download behavior;
432
+ - Scrub Key schema, binding, digest, warning or lifecycle;
433
+ - reinsert semantics;
434
+ - recognizers, profiles, thresholds or dependencies;
435
+ - external network, telemetry, browser persistence or cloud processing.
436
+
437
+ ## 2026-08-04 01:00 Europe/Amsterdam — SCRUB-WP_PROCESSED_TEXT_SELECTION_COMPONENT_SPIKE
438
+
439
+ Status: completed; technical validation green.
440
+
441
+ Purpose:
442
+ - Prove the browser/component layer before connecting selection events to the production replacement table.
443
+
444
+ Files added:
445
+ - `processed_text_selection_component.py`
446
+ - `processed_text_selection_component_spike_demo.py`
447
+ - `frontend/processed_text_selection_component/index.html`
448
+ - `frontend/processed_text_selection_component/styles.css`
449
+ - `frontend/processed_text_selection_component/streamlit_bridge.js`
450
+ - `frontend/processed_text_selection_component/component_core.js`
451
+ - `frontend/processed_text_selection_component/component.js`
452
+ - `frontend/processed_text_selection_component/NOTICE.md`
453
+ - `tests/test_processed_text_selection_component_spike.py`
454
+ - `tests/frontend/processed_text_selection_component_core.test.js`
455
+ - `PROCESSED_TEXT_SELECTION_COMPONENT_SPIKE.md`
456
+ - `workpackage_claims/scrub_wp_processed_text_selection_component_spike.md`
457
+ - `handover/workpackages/20260804_0100_processed_text_selection_component_spike.md`
458
+
459
+ Files changed:
460
+ - `PROCESSED_TEXT_SELECTION_MASKING_CONTRACT.md`
461
+ - `ROADMAP.md`
462
+ - `WORKPACKAGES.md`
463
+ - `RISK_REGISTER.md`
464
+
465
+ Implementation:
466
+ - local lazy Streamlit Components v1 wrapper on pinned Streamlit 1.39;
467
+ - Python code-point highlight validation and conversion to UTF-16 browser offsets;
468
+ - safe text-node rendering and marked segments;
469
+ - DOM selection offsets across plain and marked text nodes;
470
+ - synchronized scrolling and scroll restoration;
471
+ - right-click, keyboard and visible selection entry points;
472
+ - accessible type/confirmation menu;
473
+ - inspect event, server result and commit-intent transport;
474
+ - standalone synthetic inspection-only demo;
475
+ - no runtime build step, external assets, network, storage or telemetry.
476
+
477
+ Validation:
478
+ - run #1977: 1126 tests passed in 13.83s;
479
+ - Streamlit 1.39 smoke run #1979: 1126 tests passed in 13.79s;
480
+ - AppTest completed without script exceptions;
481
+ - local server health `ok`, root HTML and startup log checks passed;
482
+ - clean post-governance standard run #1989: 1126 tests passed in 10.87s.
483
+
484
+ Intentionally not changed:
485
+ - `presidio_streamlit.py` and production side-by-side renderer;
486
+ - replacement table, exports, Scrub Key or reinsert;
487
+ - recognizers, profiles, dependencies or cloud processing;
488
+ - occurrence-specific masking.
489
+
490
+ ## 2026-08-04 00:30 Europe/Amsterdam — SCRUB-WP_PROCESSED_TEXT_SELECTION_MASKING_ACTION_MODEL
491
+
492
+ Status: completed; validation green.
493
+
494
+ Purpose:
495
+ - Implement the server-authoritative selection action model before any browser or Streamlit integration.
496
+
497
+ Files added:
498
+ - `selection_mask_action.py`
499
+ - `tests/test_selection_mask_action.py`
500
+ - `PROCESSED_TEXT_SELECTION_MASKING_ACTION_MODEL.md`
501
+ - `workpackage_claims/scrub_wp_processed_text_selection_masking_action_model.md`
502
+ - `handover/workpackages/20260804_0030_processed_text_selection_masking_action_model.md`
503
+
504
+ Files changed:
505
+ - `manual_mask_entry.py`
506
+ - `PROCESSED_TEXT_SELECTION_MASKING_CONTRACT.md`
507
+ - `ROADMAP.md`
508
+ - `WORKPACKAGES.md`
509
+ - `RISK_REGISTER.md`
510
+
511
+ Implementation:
512
+ - strict schema/action/event/scope/hash/payload parsing;
513
+ - exact UTF-16-to-Python index conversion and surrogate-pair rejection;
514
+ - quick-selection length, line, control, placeholder and marked-range checks;
515
+ - exact occurrence and Unicode-aware embedded-token analysis;
516
+ - duplicate and nested included-replacement collision guards;
517
+ - ready, confirmation-required and blocked inspection results;
518
+ - bounded replay state and opaque single-use inspections;
519
+ - commit-time revalidation of document, binding, source, processed text, replacement table and impact;
520
+ - existing document-bound manual-row construction for eight internal quick types;
521
+ - stable manual action records and fail-closed undo.
522
+
523
+ Validation:
524
+ - initial run #1957 failed from one local event-ID assignment defect and one overly literal test; no safety boundary was weakened;
525
+ - corrected run #1961: 1106 tests passed in 10.66s;
526
+ - clean standard run #1970: 1106 tests passed in 10.71s.
527
+
528
+ Intentionally not changed:
529
+ - visible manual form options;
530
+ - Streamlit, browser component or session state;
531
+ - review-table and export/download flows;
532
+ - Scrub Key, reinsert, recognizers, profiles or cloud processing;
533
+ - occurrence-specific masking or dependencies.
534
+
535
+ ## 2026-08-04 00:09 Europe/Amsterdam — SCRUB-WP_PROCESSED_TEXT_SELECTION_MASKING_CONTRACT
536
+
537
+ Status: completed; validation green.
538
+
539
+ Purpose:
540
+ - Convert the approved processed-text selection direction into an exact implementation contract before product code changes.
541
+
542
+ Files added:
543
+ - `PROCESSED_TEXT_SELECTION_MASKING_CONTRACT.md`
544
+ - `test_cases/processed_text_selection_masking/contract.json`
545
+ - `tests/test_processed_text_selection_masking_contract.py`
546
+ - `workpackage_claims/scrub_wp_processed_text_selection_masking_contract.md`
547
+ - `handover/workpackages/20260804_0009_processed_text_selection_masking_contract.md`
548
+
549
+ Files changed:
550
+ - `PROCESSED_TEXT_SELECTION_MASKING_PLAN.md`
551
+ - `ROADMAP.md`
552
+ - `WORKPACKAGES.md`
553
+ - `RISK_REGISTER.md`
554
+ - `DECISION_LOG.md`
555
+
556
+ Contract result:
557
+ - froze a two-stage `inspect_selection` / `commit_manual_mask` protocol so occurrence impact is server-derived before commitment;
558
+ - froze all-exact scope, UTF-16 offset semantics and strict stale/replay handling;
559
+ - froze 1–5 ready, 6–20 confirmation-required and >20 blocked impact bands;
560
+ - froze 160-code-point single-line selections and an 8192-byte payload cap;
561
+ - froze eight broad quick types with server-owned entity and placeholder mappings;
562
+ - froze embedded-substring, nested replacement and marked-range collision blocking;
563
+ - froze visible, right-click and keyboard access plus one-step undo behavior;
564
+ - froze no-network, no-browser-persistence, escaped-rendering and fail-closed boundaries;
565
+ - authorized only the pure action-model package next.
566
+
567
+ Validation:
568
+ - machine-readable contract fixture and contract tests added;
569
+ - clean contract run #1954: 1027 tests passed in 11.48s;
570
+ - Hugging Face sync not functionally relevant;
571
+ - app verification not applicable because no runtime behavior changed.
572
+
573
+ Intentionally not changed:
574
+ - `manual_mask_entry.py`, `presidio_streamlit.py` or side-by-side renderer;
575
+ - Streamlit/dependency versions or Docker runtime;
576
+ - review table, export, Scrub Key or reinsert semantics;
577
+ - recognizers, profiles, cloud processing or product claims.
578
+
579
+ ## 2026-08-03 23:42 Europe/Amsterdam — SCRUB-WP_PROCESSED_TEXT_SELECTION_MASKING_PLAN
580
+
581
+ Status: completed as planning; validation green.
582
+
583
+ Purpose:
584
+ - Assess a document-centric path for correcting missed sensitive values directly from the processed-text pane.
585
+
586
+ Files added:
587
+ - `PROCESSED_TEXT_SELECTION_MASKING_PLAN.md`
588
+ - `tests/test_processed_text_selection_masking_plan.py`
589
+ - `workpackage_claims/scrub_wp_processed_text_selection_masking_plan.md`
590
+ - `handover/workpackages/20260803_2342_processed_text_selection_masking_plan.md`
591
+
592
+ Files changed:
593
+ - `ROADMAP.md`
594
+ - `WORKPACKAGES.md`
595
+ - `RISK_REGISTER.md`
596
+ - `SIDE_BY_SIDE_REVIEW_UX_DIRECTION.md`
597
+
598
+ Planning result:
599
+ - confirmed that the current static `components.html` pane cannot return a supported mutation event;
600
+ - recommended a bounded bidirectional Streamlit v1 custom component on the current Streamlit 1.39 stack;
601
+ - routed accepted actions through server-side validation and the existing `build_manual_mask_row` path;
602
+ - kept the review table authoritative and the manual form available as fallback;
603
+ - limited version one to all exact occurrences and deferred occurrence-specific masking to a separate span-aware architecture line;
604
+ - defined quick type choices, occurrence-impact warnings, collision blocking, event replay/stale-view protection, scroll restoration, undo, accessibility, XSS and no-external-network requirements;
605
+ - defined six sequential test-first implementation workpackages.
606
+
607
+ Validation:
608
+ - documentation/architecture contract tests added;
609
+ - PR #59 run #1937: 1015 tests passed in 11.64s;
610
+ - one final clean regression follows the status-only update;
611
+ - Hugging Face sync not functionally relevant;
612
+ - app verification not applicable because no product behavior changed.
613
+
614
+ Intentionally not changed:
615
+ - `presidio_streamlit.py`, side-by-side renderer or manual mask helper;
616
+ - Streamlit/dependency versions or Docker runtime;
617
+ - replacement table, export, Scrub Key or reinsert semantics;
618
+ - recognizers, profiles, cloud processing or product claims.
619
+
620
+ ## 2026-08-03 23:26 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_LONG_FORM_SYNTHETIC_CORPUS_APP_VERIFY_CLOSEOUT
621
+
622
+ Status: completed; final documentation-only regression pending.
623
+
624
+ Purpose:
625
+ - Close the long-form synthetic Zorgfilter corpus after technical deployment verification and coordinator/user app confirmation.
626
+
627
+ Files added:
628
+ - `workpackage_claims/scrub_wp_care_profile_long_form_synthetic_corpus_app_verify_closeout.md`
629
+ - `handover/workpackages/20260803_2326_care_profile_long_form_synthetic_corpus_app_verify_closeout.md`
630
+
631
+ Files changed:
632
+ - `workpackage_claims/scrub_wp_care_profile_long_form_synthetic_corpus.md`
633
+ - `handover/workpackages/20260803_2217_care_profile_long_form_synthetic_corpus.md`
634
+ - `WORKPACKAGES.md`
635
+ - `CHANGELOG.md`
636
+
637
+ Validation evidence:
638
+ - final clean PR #56 run #1926: 1003 tests passed in 11.51s;
639
+ - deployment verification run #1931: exact GitHub/Hugging Face matches for both runtime files, health HTTP 200 / `ok`, and 1003 tests passed in 11.35s;
640
+ - coordinator/user confirmation at 2026-08-03 23:26 Europe/Amsterdam: `Alles werkt.`
641
+
642
+ Intentionally not changed:
643
+ - product code, synthetic corpus content, UI, recognizers or profile behavior;
644
+ - replacement table, export, Scrub Key or reinsert semantics;
645
+ - dependencies, cloud processing or product claims.
646
+
647
+ ## 2026-08-03 22:17 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_LONG_FORM_SYNTHETIC_CORPUS
648
+
649
+ Status: completed, synchronized and app-verified.
650
+
651
+ Purpose:
652
+ - Give Zorgfilter testers realistic long-form care documents instead of examples that stop after one short clinical paragraph.
653
+
654
+ Files added:
655
+ - `care_test_example_expansions.py`
656
+ - `tests/test_care_profile_long_form_corpus.py`
657
+ - `CARE_PROFILE_LONG_FORM_SYNTHETIC_CORPUS.md`
658
+ - `workpackage_claims/scrub_wp_care_profile_long_form_synthetic_corpus.md`
659
+ - `handover/workpackages/20260803_2217_care_profile_long_form_synthetic_corpus.md`
660
+
661
+ Files changed:
662
+ - `care_test_examples.py`
663
+ - `WORKPACKAGES.md`
664
+ - `RELEASE_NOTES.md`
665
+ - `RISK_REGISTER.md`
666
+
667
+ Implementation result:
668
+ - retained all eight stable care-document IDs, names, sectors and document types;
669
+ - retained every replace, review-selected, preserve, audit-only and ambiguity-trap contract;
670
+ - appended five document-specific sections and at least two hundred words of synthetic care context to every example;
671
+ - kept the additions free of digits and new names, identifiers, dates, addresses, contact details, organizations and locations;
672
+ - exposed the expanded texts through the existing Zorgfilter example selector without changing the selector or Streamlit flow;
673
+ - added tests for length, structure, non-mutation, identity-marker absence, exact expected-value occurrence and UI-adapter parity.
674
+
675
+ Validation:
676
+ - final clean PR run #1926: 1003 tests passed in 11.51s;
677
+ - deployment verification run #1931: both runtime files matched Hugging Face byte-for-byte, Space health returned HTTP 200 / `ok`, and 1003 tests passed in 11.35s;
678
+ - coordinator/user app verification confirmed at 2026-08-03 23:26 Europe/Amsterdam: `Alles werkt.`
679
+
680
+ Intentionally not changed:
681
+ - recognizers, thresholds, profile composition or collision precedence;
682
+ - review selection or replacement-table behavior;
683
+ - export filenames, MIME types or formats;
684
+ - Scrub Key schema, binding, warnings or lifecycle;
685
+ - reinsert behavior, dependencies, cloud processing or production claims.
686
+
687
+ ## 2026-08-03 19:12 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_APP_VERIFY
688
+
689
+ Status: completed and app-verified after technical deployment verification.
690
+
691
+ Purpose:
692
+ - Independently verify that the merged Zorgfilter integration reached Hugging Face before requesting human app verification.
693
+
694
+ Files added:
695
+ - `CARE_PROFILE_APP_VERIFICATION.md`
696
+ - `output/validation/care_profile_hf_sync_verification.json`
697
+ - `tests/test_care_profile_hf_sync_verification.py`
698
+ - `workpackage_claims/scrub_wp_care_profile_app_verify.md`
699
+ - `handover/workpackages/20260803_1912_care_profile_app_verify.md`
700
+
701
+ Verification result:
702
+ - compared twelve relevant GitHub and Hugging Face files byte-for-byte;
703
+ - verified equal SHA-256 values for all twelve files;
704
+ - verified correctly scoped Zorgfilter markers;
705
+ - verified Streamlit health `HTTP 200 / ok` and root HTTP 200;
706
+ - corrected an initial verification false negative caused by two marker groups being checked in the wrong modules;
707
+ - recorded coordinator/user confirmation `alles groen` at 2026-08-03 20:35 Europe/Amsterdam;
708
+ - confirmed all nine visible verification checks and retained the non-production/human-review boundary.
709
+
710
+ Validation context:
711
+ - UI integration final run #1885 passed: 986 tests;
712
+ - cross-profile matrix final run #1906 passed: 995 tests;
713
+ - verification-only run #1908 passed 998 tests; final closeout run #1909 passed 998 tests in 10.45s.
714
+
715
+ Intentionally not changed:
716
+ - product code, recognizers, profile behavior or UI;
717
+ - review, export, Scrub Key or reinsert semantics;
718
+ - runtime dependencies, cloud processing or production claims.
719
+
720
+ ## 2026-08-03 18:58 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_CROSS_PROFILE_REGRESSION_MATRIX
721
+
722
+ Status: completed and regression-green.
723
+
724
+ Purpose:
725
+ - Add deterministic evidence that the new Care profile remains isolated from Legal and General profiles and preserves clinical meaning.
726
+
727
+ Files added:
728
+ - `care_profile_cross_profile_matrix.py`
729
+ - `CARE_PROFILE_CROSS_PROFILE_REGRESSION_MATRIX.md`
730
+ - `tests/test_care_profile_cross_profile_matrix.py`
731
+ - `output/validation/care_profile_cross_profile_matrix.json`
732
+ - `workpackage_claims/scrub_wp_care_profile_cross_profile_regression_matrix.md`
733
+ - `handover/workpackages/20260803_1858_care_profile_cross_profile_regression_matrix.md`
734
+
735
+ Implementation result:
736
+ - executed real deterministic Dutch custom recognizers across four configured profiles;
737
+ - covered eight care-document families and twelve legal examples;
738
+ - verified 108/108 dedicated Care expectations across Care and International;
739
+ - verified Care/International and Legal/International dedicated-type parity;
740
+ - verified no dedicated Care/Legal leakage into the wrong profiles;
741
+ - verified zero overlap with protected clinical phrases;
742
+ - separated hard profile gates from historical legal metadata observations;
743
+ - preserved sixteen historical legal metadata gaps and four negative observations in the evidence snapshot;
744
+ - added snapshot reproducibility checks for all twenty observations.
745
+
746
+ Validation:
747
+ - run #1887 exposed an invalid hard-contract assumption for historical legal metadata;
748
+ - run #1888 made all twenty observations explicit;
749
+ - run #1890 passed after correcting the methodology: 994 tests;
750
+ - run #1897 exposed only an incorrect legal-example count in the snapshot;
751
+ - run #1899 passed: 995 tests in 9.56s;
752
+ - final clean run #1906 passed: 995 tests in 9.96s.
753
+
754
+ Intentionally not changed:
755
+ - current Streamlit UI or profile selector;
756
+ - review table, export, Scrub Key or reinsert semantics;
757
+ - runtime dependencies, cloud processing or production claims;
758
+ - generic NER behavior.
759
+
760
+ ## 2026-08-03 18:28 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_CURRENT_UI_INTEGRATION
761
+
762
+ Status: completed, deployed and app-verified.
763
+
764
+ Purpose:
765
+ - Promote the approved Zorgfilter profile into the current Streamlit/analyzer flow after policy, corpus, recognizer and profile-configuration gates passed.
766
+
767
+ Files added:
768
+ - `care_candidate_scanner.py`
769
+ - `profile_ui_support.py`
770
+ - `CARE_PROFILE_CURRENT_UI_INTEGRATION.md`
771
+ - `output/validation/care_profile_current_ui_integration.json`
772
+ - `tests/test_care_candidate_scanner.py`
773
+ - `tests/test_profile_ui_support.py`
774
+ - `tests/test_presidio_helpers_care_registration.py`
775
+ - `tests/test_care_profile_current_ui_integration_snapshot.py`
776
+ - `workpackage_claims/scrub_wp_care_profile_current_ui_integration.md`
777
+ - `handover/workpackages/20260803_1828_care_profile_current_ui_integration.md`
778
+
779
+ Files changed:
780
+ - `presidio_helpers.py`
781
+ - `presidio_streamlit.py`
782
+ - `document_tools.py`
783
+ - `display_labels_nl.py`
784
+ - `ui_texts_nl.py`
785
+ - `ROADMAP.md`
786
+ - `WORKPACKAGES.md`
787
+ - `CHANGELOG.md`
788
+ - `RELEASE_NOTES.md`
789
+ - `RISK_REGISTER.md`
790
+
791
+ Implementation result:
792
+ - registered sixteen dedicated care recognizers;
793
+ - added `Zorgcontrole — streng` while preserving the existing three labels and Legal default;
794
+ - centralized thresholds, profile descriptions and default entity composition;
795
+ - applied profile-level exact-span collision resolution;
796
+ - added eight synthetic care examples and a conservative unchecked candidate layer;
797
+ - aligned review-selected care detections to `Controle nodig` while keeping them selected;
798
+ - added care display labels and stable placeholders;
799
+ - generalized app copy from Legal-only to professional Legal/Zorg use;
800
+ - preserved review table, export, Scrub Key and reinsert semantics.
801
+
802
+ Validation:
803
+ - run #1876 failed only because the new registration test imported optional Streamlit dependencies absent from lean CI;
804
+ - test isolation was corrected without changing runtime code or dependencies;
805
+ - run #1877 passed: 983 tests in 9.69s;
806
+ - final clean integration run #1885 passed: 986 tests;
807
+ - Hugging Face synchronization verified byte-for-byte for 12/12 relevant files;
808
+ - deployed app verification confirmed `alles groen` by the coordinator/user.
809
+
810
+ Intentionally not changed:
811
+ - export filenames, MIME types and document formats;
812
+ - Scrub Key schema, binding, warnings or lifecycle;
813
+ - TXT/DOCX/PDF-to-TXT reinsert semantics;
814
+ - cloud processing, runtime dependencies or production claims;
815
+ - broad free-text medical scanning.
816
+
817
+ ## 2026-08-03 17:12 Europe/Amsterdam — SCRUB-WP_RECOGNITION_PROFILE_CONFIGURATION_REFACTOR
818
+
819
+ Status: completed; pure recognition-profile configuration implemented.
820
+
821
+ Purpose:
822
+ - Replace scattered future profile branching with one tested source of truth before exposing Zorg in the current UI.
823
+
824
+ Files added:
825
+ - `recognition_profiles.py`
826
+ - `recognition_profile_validation.py`
827
+ - `RECOGNITION_PROFILE_CONFIGURATION.md`
828
+ - `tests/test_recognition_profiles.py`
829
+ - `tests/test_recognition_profile_validation.py`
830
+ - `output/validation/recognition_profile_configuration.json`
831
+ - `workpackage_claims/scrub_wp_recognition_profile_configuration_refactor.md`
832
+ - `handover/workpackages/20260803_1712_recognition_profile_configuration_refactor.md`
833
+
834
+ Implementation result:
835
+ - preserved the exact current three-profile order and labels;
836
+ - defined the future Streamlit and desktop four-profile orders;
837
+ - centralized thresholds, entity groups, candidate/example direction and Care policy actions;
838
+ - added deterministic exact-span precedence for fifteen care-specific winners;
839
+ - kept partial overlaps and all non-Care profile results unchanged;
840
+ - kept live UI and care recognizer registration explicitly false.
841
+
842
+ Validation:
843
+ - initial GitHub Actions run #1865 passed: 965 tests;
844
+ - final clean validation pending after governance finalization;
845
+ - Hugging Face sync not functionally relevant;
846
+ - app verification not applicable.
847
+
848
+ Intentionally not changed:
849
+ - `presidio_streamlit.py`, `presidio_helpers.py` or current analyzer registration;
850
+ - visible profile selector, thresholds or entity defaults;
851
+ - review, export, Scrub Key or reinsert semantics;
852
+ - runtime, dependencies or cloud processing.
853
+
854
+ ## 2026-08-03 16:52 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_RECOGNIZER_IMPLEMENTATION
855
+
856
+ Status: completed; pure recognizer implementation validated.
857
+
858
+ Purpose:
859
+ - Implement the frozen Zorgfilter recognizer contract without registering it in the current app.
860
+
861
+ Files added:
862
+ - `dutch_care_recognizers.py`
863
+ - `care_recognizer_validation.py`
864
+ - `CARE_RECOGNIZER_IMPLEMENTATION_V1.md`
865
+ - `scripts/generate_care_recognizer_validation.py`
866
+ - `tests/test_dutch_care_recognizers.py`
867
+ - `tests/test_care_recognizer_validation.py`
868
+ - `output/validation/care_recognizer_implementation_validation.json`
869
+ - `workpackage_claims/scrub_wp_care_profile_recognizer_implementation.md`
870
+ - `handover/workpackages/20260803_1652_care_profile_recognizer_implementation.md`
871
+
872
+ Implementation result:
873
+ - implemented sixteen dedicated, context-bound care entities;
874
+ - returned exact value-only Presidio spans with explanations and metadata;
875
+ - corrected bounded variants for Dutch client labels, prescription numbers, generic incident labels and lowercase residential-care organization labels;
876
+ - passed all frozen positive and negative contracts;
877
+ - covered all 54 dedicated corpus expectations with zero protected-clinical overlaps;
878
+ - kept app registration explicitly false.
879
+
880
+ Validation:
881
+ - initial run #1850 exposed five bounded missing context variants and no clinical-overmasking failure;
882
+ - corrected GitHub Actions run #1854 passed: 953 tests;
883
+ - final clean validation pending after governance finalization;
884
+ - Hugging Face sync not functionally relevant;
885
+ - app verification not applicable.
886
+
887
+ Intentionally not changed:
888
+ - `presidio_helpers.py`, `presidio_streamlit.py` or current profile behavior;
889
+ - thresholds, entity defaults or generic NER;
890
+ - review, export, Scrub Key or reinsert semantics;
891
+ - runtime, dependencies or cloud processing.
892
+
893
+ ## 2026-08-03 16:34 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_RECOGNIZER_CONTRACT_TESTS
894
+
895
+ Status: completed; test/specification contract frozen.
896
+
897
+ Purpose:
898
+ - Define dedicated Zorgfilter recognizer behavior before writing implementation code.
899
+
900
+ Files added:
901
+ - `CARE_RECOGNIZER_CONTRACT_V1.md`
902
+ - `care_recognizer_contracts.py`
903
+ - `care_recognizer_contract_summary.py`
904
+ - `tests/test_care_recognizer_contracts.py`
905
+ - `scripts/generate_care_recognizer_contract_summary.py`
906
+ - `output/validation/care_recognizer_contract_v1_summary.json`
907
+ - `workpackage_claims/scrub_wp_care_profile_recognizer_contract_tests.md`
908
+ - `handover/workpackages/20260803_1634_care_profile_recognizer_contract_tests.md`
909
+
910
+ Implementation result:
911
+ - froze sixteen dedicated care entities and the future pure helper API;
912
+ - added 37 positive exact-span fixtures;
913
+ - added 16 negative/collision/clinical-preservation fixtures;
914
+ - froze replace/review-selected policy alignment;
915
+ - froze care-event/date-of-birth and AGB/BSN precedence;
916
+ - kept generic PERSON/e-mail outside the dedicated care module.
917
+
918
+ Validation:
919
+ - GitHub Actions pending final PR validation;
920
+ - Hugging Face sync not functionally relevant;
921
+ - app verification not applicable.
922
+
923
+ Intentionally not changed:
924
+ - recognizer implementation or registration;
925
+ - current profile selector, thresholds or entity defaults;
926
+ - review, export, Scrub Key and reinsert semantics;
927
+ - runtime, dependencies or cloud processing.
928
+
929
+ ## 2026-08-03 16:25 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_GAP_TRIAGE
930
+
931
+ Status: completed; evidence-driven gap routing implemented.
932
+
933
+ Purpose:
934
+ - Convert the current-engine care baseline into explicit recognizer, profile and collision-prevention contract routes.
935
+
936
+ Files added:
937
+ - `CARE_PROFILE_GAP_TRIAGE.md`
938
+ - `care_profile_gap_triage.py`
939
+ - `care_profile_gap_triage_summary.py`
940
+ - `tests/test_care_profile_gap_triage.py`
941
+ - `scripts/generate_care_profile_gap_triage.py`
942
+ - `output/validation/care_profile_v1_gap_triage.json`
943
+ - `workpackage_claims/scrub_wp_care_profile_gap_triage.md`
944
+ - `handover/workpackages/20260803_1625_care_profile_gap_triage.md`
945
+
946
+ Implementation result:
947
+ - classified all 81 expectations with zero unclassified values;
948
+ - froze six implementation routes and five contract families;
949
+ - separated generic NER dependencies from care-specific rule work;
950
+ - routed ten broad-entity matches to care-specific reclassification;
951
+ - routed 36 context-dependent values to review-selected recognition;
952
+ - made AGB/BSN and medical-number collision guards mandatory;
953
+ - preserved clinical-content negative contracts as a first-class requirement.
954
+
955
+ Validation:
956
+ - GitHub Actions pending final PR validation;
957
+ - Hugging Face sync not functionally relevant;
958
+ - app verification not applicable.
959
+
960
+ Intentionally not changed:
961
+ - recognizer implementation or registration;
962
+ - profile selector, thresholds or entity defaults;
963
+ - review, export, Scrub Key or reinsert semantics;
964
+ - runtime, dependencies or cloud processing.
965
+
966
+ ## 2026-08-03 16:10 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_CURRENT_ENGINE_BASELINE
967
+
968
+ Status: completed; evidence baseline implemented and validated.
969
+
970
+ Purpose:
971
+ - Establish a corrected, reproducible pre-Zorgfilter measurement for the current deterministic Dutch custom recognizers.
972
+
973
+ Files added:
974
+ - `CARE_PROFILE_CURRENT_ENGINE_BASELINE.md`
975
+ - `care_profile_baseline_summary.py`
976
+ - `tests/test_care_profile_baseline_summary.py`
977
+ - `output/validation/care_profile_v1_current_engine_baseline.json`
978
+ - `workpackage_claims/scrub_wp_care_profile_current_engine_baseline.md`
979
+ - `handover/workpackages/20260803_1610_care_profile_current_engine_baseline.md`
980
+
981
+ Files changed:
982
+ - `care_profile_baseline.py`
983
+ - `scripts/generate_care_profile_baseline.py`
984
+ - `tests/test_care_profile_current_engine_baseline.py`
985
+ - `ROADMAP.md`
986
+ - `WORKPACKAGES.md`
987
+ - `RISK_REGISTER.md`
988
+
989
+ Implementation result:
990
+ - changed baseline matching from substring containment to exact normalized span equality;
991
+ - prevented false AGB coverage from an eight-digit prefix inside a longer BIG number;
992
+ - added compact policy/entity/case summaries;
993
+ - committed a reproducible JSON evidence artifact;
994
+ - documented 25/81 span recall, 14/81 correct-entity recall, 11 misclassifications, 56 misses and zero protected-clinical overlaps.
995
+
996
+ Validation:
997
+ - diagnostic run #1825 produced the evidence while 918 non-diagnostic tests passed;
998
+ - final clean GitHub Actions validation required after temporary diagnostics were removed;
999
+ - Hugging Face sync not functionally relevant;
1000
+ - app verification not applicable.
1001
+
1002
+ Intentionally not changed:
1003
+ - recognizer registration or behavior;
1004
+ - current profile selector or UI;
1005
+ - thresholds or entity defaults;
1006
+ - review, export, Scrub Key and reinsert semantics;
1007
+ - dependencies, runtime or cloud processing.
1008
+
1009
+ ## 2026-08-03 15:31 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_V1_POLICY_AND_CORPUS_FOUNDATION
1010
+
1011
+ Status: completed; policy/corpus foundation implemented and validated.
1012
+
1013
+ Purpose:
1014
+ - Create the evidence base for an explicit Dutch Zorg profile without prematurely changing recognizers or the UI.
1015
+
1016
+ Files added:
1017
+ - `CARE_PROFILE_V1_PLAN.md`
1018
+ - `care_profile_policy.py`
1019
+ - `care_test_examples.py`
1020
+ - `care_profile_baseline.py`
1021
+ - `scripts/generate_care_profile_baseline.py`
1022
+ - `tests/test_care_profile_policy_contract.py`
1023
+ - `tests/test_care_profile_corpus_contracts.py`
1024
+ - `tests/test_care_profile_current_engine_baseline.py`
1025
+ - `workpackage_claims/scrub_wp_care_profile_v1_policy_and_corpus_foundation.md`
1026
+
1027
+ Main changes:
1028
+ - froze replace/review/preserve/audit-only care policy actions;
1029
+ - added eight fully synthetic document families;
1030
+ - separated patient identifiers from provider/location review and clinical content preservation;
1031
+ - added a deterministic baseline helper for current Dutch custom recognizers;
1032
+ - made no recognizer or product-UI behavior change.
1033
+
1034
+ Validation:
1035
+ - full GitHub Actions run #1818 passed: 918 tests;
1036
+ - Hugging Face sync not functionally relevant because no runtime/UI file changed;
1037
+ - app verification not applicable.
1038
+
1039
+ Intentionally not changed:
1040
+ - current three profile choices;
1041
+ - recognizer registration or thresholds;
1042
+ - review table, export, Scrub Key and reinsert semantics;
1043
+ - runtime, dependencies or cloud processing.
1044
+
1045
+ ## 2026-08-03 14:47 Europe/Amsterdam — SCRUB-WP_AI_FIRST_DESKTOP_PACKAGING_ROADMAP_ALIGNMENT
1046
+
1047
+ Status: completed; roadmap/decision documentation only.
1048
+
1049
+ Purpose:
1050
+ - Incorporate an AI-first cost and authority model into the final local EXE/MSI roadmap while preserving security and release accountability.
1051
+
1052
+ Files added:
1053
+ - `AI_FIRST_DESKTOP_PACKAGING_EXECUTION_MODEL.md`
1054
+ - `handover/workpackages/20260803_1447_ai_first_desktop_packaging_roadmap_alignment.md`
1055
+ - `workpackage_claims/scrub_wp_ai_first_desktop_packaging_roadmap_alignment.md`
1056
+
1057
+ Files changed:
1058
+ - `ROADMAP.md`
1059
+ - `DESKTOP_PACKAGING_DECISION.md`
1060
+ - `DECISION_LOG.md`
1061
+ - `WORKPACKAGES.md`
1062
+ - `CHANGELOG.md`
1063
+
1064
+ Main changes:
1065
+ - Phase 9 target refined to signed setup.exe/MSI distribution around a Tauri shell and bundled PyInstaller onedir local engine.
1066
+ - AI-agent planning assumption recorded: 60–70% first-cycle labor substitution and 75–90% later repetitive release automation.
1067
+ - Indicative post-agent development/integration budget recorded as EUR 8,000–24,000, excluding retained independent security review and authority costs.
1068
+ - Human gates retained for signing, release, security claims, UX acceptance and safety-critical semantics.
1069
+
1070
+ Validation:
1071
+ - Documentation markers and phase gates checked by the branch governance operator.
1072
+ - No product tests required because no product code or runtime changed.
1073
+ - GitHub Actions: pending PR validation.
1074
+ - Hugging Face sync: not functionally relevant.
1075
+ - App verification: not applicable.
1076
+
1077
+ Intentionally not changed:
1078
+ - active Phase 6 execution order;
1079
+ - installer implementation authorization;
1080
+ - runtime, UI, dependencies, recognizers, export, Scrub Key or reinsert behavior;
1081
+ - cloud processing or telemetry behavior.
1082
+
1083
+ ## 2026-07-28 00:52 Europe/Amsterdam — SCRUB-WP_MVP_SCRUB_KEY_BINDING_REINSERT_INTEGRATION
1084
+
1085
+ Status: implemented; full GitHub Actions passed; final PR validation pending.
1086
+
1087
+ Purpose:
1088
+ - Prevent a wrong, mixed or accidentally corrupted Scrub Key from restoring values before document/key binding is verified.
1089
+
1090
+ Files added:
1091
+ - `scrub_key_binding_reinsert_status.py`
1092
+ - `tests/test_scrub_key_binding_reinsert_integration.py`
1093
+ - `tests/test_scrub_key_binding_reinsert_status.py`
1094
+ - `tests/test_scrub_key_binding_reinsert_ui.py`
1095
+ - `output/validation/mvp_scrub_key_binding_reinsert_validation.json`
1096
+ - `handover/workpackages/20260728_0052_mvp_scrub_key_binding_reinsert_integration.md`
1097
+ - `workpackage_claims/scrub_wp_mvp_scrub_key_binding_reinsert_integration.md`
1098
+
1099
+ Files changed:
1100
+ - `scrub_key_import.py`
1101
+ - `scrub_key_reinsert.py`
1102
+ - `scrub_key_document_reinsert.py`
1103
+ - `reinsert_mode_ui.py`
1104
+ - `tests/test_scrub_key_binding_model.py`
1105
+ - `WORKPACKAGES.md`
1106
+ - `RELEASE_NOTES.md`
1107
+ - `ROADMAP.md`
1108
+ - `MVP_PHASE6_EXECUTION_PLAN.md`
1109
+ - `RISK_REGISTER.md`
1110
+ - `DECISION_LOG.md`
1111
+
1112
+ Implementation result:
1113
+ - Dual-read import supports structurally valid legacy v1.0 and bound v1.1 keys.
1114
+ - Binding validation runs before any local deterministic replacement.
1115
+ - `bound_match` is verified; `legacy_unbound` remains compatible but explicitly unverified.
1116
+ - Six frozen mismatch/corruption states fail closed with zero replacements.
1117
+ - DOCX mismatch returns exact original bytes; no partial package is produced.
1118
+ - Binding status, IDs, digest state and warnings are shown in existing feedback/report surfaces.
1119
+ - No new source/key execution button or acknowledgement checkbox was added.
1120
+
1121
+ Validation:
1122
+ - Normal full GitHub Actions run #1789 passed.
1123
+ - Synthetic adversarial coverage spans text, TXT, DOCX body/header/footer and PDF-to-TXT.
1124
+ - Human review remains required; production readiness remains false.
1125
+
1126
+ Intentionally not changed:
1127
+ - Output filenames or MIME types.
1128
+ - Supported TXT, DOCX and PDF-to-TXT boundaries.
1129
+ - Legacy key migration.
1130
+ - Signing/HMAC, secret storage, cloud, AI, OCR or restored-PDF behavior.
1131
+
1132
+ Next recommended step:
1133
+ - Complete final PR validation and merge, verify sync, then run `SCRUB-WP_MVP_SCRUB_KEY_BINDING_APP_VERIFY`.
1134
+
1135
+ ---
1136
+
1137
+ ## 2026-07-27 20:05 Europe/Amsterdam — SCRUB-WP_MVP_SCRUB_KEY_BINDING_MODEL_IMPLEMENTATION
1138
+
1139
+ Status: implemented; targeted validation passed; PR verification pending.
1140
+
1141
+ Purpose:
1142
+ - Implement the frozen document/Scrub-Key binding contract as pure helpers before changing shared placeholder generation, export or reinsert behavior.
1143
+
1144
+ Files added:
1145
+ - `scrub_key_binding.py`
1146
+ - `tests/test_scrub_key_binding_model.py`
1147
+ - `tests/test_scrub_key_binding_model_validation.py`
1148
+ - `output/validation/mvp_scrub_key_binding_model_validation.json`
1149
+ - `handover/workpackages/20260727_2005_mvp_scrub_key_binding_model_implementation.md`
1150
+ - `workpackage_claims/scrub_wp_mvp_scrub_key_binding_model_implementation.md`
1151
+
1152
+ Files changed:
1153
+ - `WORKPACKAGES.md`
1154
+ - `CHANGELOG.md`
1155
+ - `ROADMAP.md`
1156
+ - `MVP_PHASE6_EXECUTION_PLAN.md`
1157
+ - `RISK_REGISTER.md`
1158
+ - `DECISION_LOG.md`
1159
+
1160
+ Implementation result:
1161
+ - Local document binding ID generation supports injected deterministic bytes and normal `secrets` randomness.
1162
+ - Bound automatic/manual placeholder creation and parsing match the frozen grammar.
1163
+ - Canonical digest output matches the fixed synthetic SHA-256 fixture.
1164
+ - Bound key validation checks schema, policies, items, item bindings, duplicates and digest.
1165
+ - Document/key validation implements `bound_match`, `legacy_unbound`, `binding_mismatch`, `mixed_document_bindings`, `missing_document_binding`, `invalid_mapping_digest`, `invalid_bound_key` and `legacy_key_for_bound_document`.
1166
+ - Bound failures prohibit replacement; legacy compatibility remains explicitly unverified.
1167
+ - Inputs remain unmodified.
1168
+
1169
+ Intentionally not changed:
1170
+ - current generic automatic/manual placeholder generation;
1171
+ - current Scrub Key builder, serializer, export or import integration;
1172
+ - current reinsert execution helpers or UI;
1173
+ - filenames or MIME types;
1174
+ - legacy migration;
1175
+ - signing/HMAC or secret storage;
1176
+ - cloud, AI, OCR or restored-PDF behavior.
1177
+
1178
+ Next recommended step:
1179
+ - Start `SCRUB-WP_MVP_SCRUB_KEY_BINDING_EXPORT_INTEGRATION` after PR validation and merge.
1180
+
1181
+ ---
1182
+
1183
+ ## 2026-07-27 19:38 Europe/Amsterdam — SCRUB-WP_MVP_SCRUB_KEY_BINDING_CONTRACT_TESTS
1184
+
1185
+ Status: completed; contract frozen; PR validation pending.
1186
+
1187
+ Purpose:
1188
+ - Freeze the bound-placeholder, Scrub Key metadata, mapping-digest, legacy compatibility and fail-closed validation contracts before implementing the model.
1189
+
1190
+ Files added:
1191
+ - `SCRUB_KEY_BINDING_CONTRACT.md`
1192
+ - `test_cases/mvp_phase6/scrub_key_binding_contract.json`
1193
+ - `tests/test_mvp_scrub_key_binding_contracts.py`
1194
+ - `tests/test_mvp_scrub_key_binding_contract_validation.py`
1195
+ - `output/validation/mvp_scrub_key_binding_contract_validation.json`
1196
+ - `handover/workpackages/20260727_1938_mvp_scrub_key_binding_contract_tests.md`
1197
+ - `workpackage_claims/scrub_wp_mvp_scrub_key_binding_contract_tests.md`
1198
+
1199
+ Files changed:
1200
+ - `WORKPACKAGES.md`
1201
+ - `CHANGELOG.md`
1202
+ - `ROADMAP.md`
1203
+ - `MVP_PHASE6_EXECUTION_PLAN.md`
1204
+ - `RISK_REGISTER.md`
1205
+ - `DECISION_LOG.md`
1206
+
1207
+ Contract result:
1208
+ - Binding ID grammar: `B[A-Z2-7]{16}`.
1209
+ - Automatic placeholder grammar: `[LABEL_BINDINGID_INDEX]`.
1210
+ - Manual placeholder grammar: `[LABEL_BINDINGID_HANDMATIG_INDEX]`.
1211
+ - Proposed bound key schema: `1.1`; binding version: `1`.
1212
+ - Canonical digest algorithm: SHA-256.
1213
+ - Eight binding statuses; six are fail-closed with zero replacements.
1214
+ - Legacy v1.0 remains explicit unbound compatibility and cannot be used for bound documents.
1215
+ - Current three-step reinsert UX remains unchanged; no new source/key buttons or checkboxes.
1216
+ - Product code changed: false.
1217
+ - Production ready: false; human review required: true.
1218
+
1219
+ Intentionally not changed:
1220
+ - existing product helpers;
1221
+ - automatic/manual placeholder generation;
1222
+ - Scrub Key build, validation, serialization or import implementation;
1223
+ - export or reinsert semantics;
1224
+ - UI or download behavior;
1225
+ - signing/HMAC or secret storage;
1226
+ - cloud, AI, OCR or restored-PDF behavior.
1227
+
1228
+ Next recommended step:
1229
+ - Start `SCRUB-WP_MVP_SCRUB_KEY_BINDING_MODEL_IMPLEMENTATION` after PR validation and merge.
1230
+
1231
+ ---
1232
+
1233
+ ## 2026-07-27 19:18 Europe/Amsterdam — SCRUB-WP_MVP_SCRUB_KEY_DOCUMENT_BINDING_GAP_TRIAGE
1234
+
1235
+ Status: completed; targeted validation passed; PR verification pending.
1236
+
1237
+ Purpose:
1238
+ - Determine the smallest safe cross-format mitigation for the critical document/Scrub-Key binding gap before changing schema, placeholders, export or reinsert behavior.
1239
+
1240
+ Files added:
1241
+ - `MVP_SCRUB_KEY_DOCUMENT_BINDING_GAP_TRIAGE.md`
1242
+ - `output/validation/mvp_scrub_key_document_binding_gap_triage.json`
1243
+ - `output/validation/mvp_scrub_key_document_binding_gap_triage_validation.json`
1244
+ - `tests/test_mvp_scrub_key_document_binding_gap_triage.py`
1245
+ - `tests/test_mvp_scrub_key_document_binding_gap_triage_validation.py`
1246
+ - `handover/workpackages/20260727_1918_mvp_scrub_key_document_binding_gap_triage.md`
1247
+ - `workpackage_claims/scrub_wp_mvp_scrub_key_document_binding_gap_triage.md`
1248
+
1249
+ Files changed:
1250
+ - `WORKPACKAGES.md`
1251
+ - `CHANGELOG.md`
1252
+ - `ROADMAP.md`
1253
+ - `MVP_PHASE6_EXECUTION_PLAN.md`
1254
+ - `RISK_REGISTER.md`
1255
+ - `DECISION_LOG.md`
1256
+
1257
+ Triage result:
1258
+ - Primary MVP threat: accidental wrong-document/key pairing.
1259
+ - Secondary MVP threat: accidental key corruption.
1260
+ - Deferred threat: malicious tampering requiring protected signing-key infrastructure.
1261
+ - Recommended primary control: document-specific non-sensitive binding ID in all placeholders and the key.
1262
+ - Recommended complementary control: canonical SHA-256 mapping digest.
1263
+ - Explicitly not sufficient: document labels, filenames, content hashes, placeholder-list hashes or metadata-only binding.
1264
+ - Legacy v1.0 keys require explicit unbound status and warning; they must not be silently treated as bound.
1265
+ - Bound-key mismatch, mixed IDs and digest mismatch must fail closed with zero replacements.
1266
+ - Human review remains required; production readiness remains false.
1267
+
1268
+ Intentionally not changed:
1269
+ - product code or UI;
1270
+ - Scrub Key schema/version or serialization;
1271
+ - placeholder generation or grammar;
1272
+ - export/download or reinsert semantics;
1273
+ - document processing;
1274
+ - cloud, AI, OCR or secret storage.
1275
+
1276
+ Next recommended step:
1277
+ - Start `SCRUB-WP_MVP_SCRUB_KEY_BINDING_CONTRACT_TESTS` before implementation.
1278
+
1279
+ ---
1280
+
1281
+ ## 2026-07-27 18:55 Europe/Amsterdam — SCRUB-WP_MVP_SCRUB_KEY_ROUNDTRIP_VALIDATION
1282
+
1283
+ Status: completed; deterministic validation passed; PR verification pending.
1284
+
1285
+ Purpose:
1286
+ - Validate Scrub Key import/reinsert and placeholder roundtrip behavior against adversarial synthetic mutations.
1287
+ - Record evidence before authorizing any schema, export or reinsert changes.
1288
+
1289
+ Files added:
1290
+ - `test_cases/mvp_phase6/scrub_key_roundtrip_manifest.json`
1291
+ - `mvp_scrub_key_roundtrip_validation.py`
1292
+ - `scripts/run_mvp_scrub_key_roundtrip_validation.py`
1293
+ - `tests/test_mvp_scrub_key_roundtrip_validation.py`
1294
+ - `tests/test_mvp_scrub_key_roundtrip_report_contract.py`
1295
+ - `output/validation/mvp_scrub_key_roundtrip_validation_report.json`
1296
+ - `handover/workpackages/20260727_1855_mvp_scrub_key_roundtrip_validation.md`
1297
+
1298
+ Files changed:
1299
+ - `WORKPACKAGES.md`
1300
+ - `CHANGELOG.md`
1301
+ - `ROADMAP.md`
1302
+ - `MVP_PHASE6_EXECUTION_PLAN.md`
1303
+ - `RISK_REGISTER.md`
1304
+ - `DECISION_LOG.md`
1305
+ - `workpackage_claims/scrub_wp_mvp_scrub_key_roundtrip_validation.md`
1306
+
1307
+ Validation result:
1308
+ - 15 synthetic cases; 0 failed cases.
1309
+ - 1 critical finding: no reliable document/key binding when a wrong valid key reuses the same placeholder namespace.
1310
+ - 1 medium finding: malformed placeholder mutations outside the grammar are signalled indirectly.
1311
+ - Existing duplicate, incomplete, invalid, unknown and translated cases fail closed or remain visibly auditable as expected.
1312
+ - Local-only: true; external AI: false; cloud processing: false.
1313
+ - Production ready: false; human review required: true.
1314
+
1315
+ Intentionally not changed:
1316
+ - product code or UI;
1317
+ - Scrub Key schema, mappings, export, storage or lifecycle;
1318
+ - placeholder grammar or automatic repair;
1319
+ - reinsert helper semantics;
1320
+ - filenames, MIME types or audit fields;
1321
+ - cloud, AI, OCR or restored-PDF behavior.
1322
+
1323
+ Next recommended step:
1324
+ - Start `SCRUB-WP_MVP_SCRUB_KEY_DOCUMENT_BINDING_GAP_TRIAGE` before implementing a fix.
1325
+
1326
+ ---
1327
+
1328
+ ## 2026-07-27 18:28 Europe/Amsterdam — SCRUB-WP_MVP_REINSERT_AUTO_FLOW_SIMPLIFICATION_APP_VERIFY_CLOSEOUT
1329
+
1330
+ Status: completed and app-verified.
1331
+
1332
+ Purpose:
1333
+ - Record successful live verification of the document-first automatic reinsert workflow.
1334
+ - Close the evidence-driven UI blocker before continuing Scrub Key roundtrip validation.
1335
+
1336
+ Validation result:
1337
+ - PR #38 merge commit: `390f381c1464883f220716655c5067dadd0bb4c9`.
1338
+ - Final clean PR GitHub Actions run #1678: passed.
1339
+ - Full repository suite before merge: 797 passed.
1340
+ - Hugging Face deployment: confirmed by live testing of the merged three-step workflow.
1341
+ - App verification: passed; coordinator reported `getest en werkend`.
1342
+
1343
+ Verified product boundaries:
1344
+ - Document/text remains step 1, Scrub Key step 2 and restored download step 3.
1345
+ - Automatic source recognition, key validation and local deterministic reinsert work as intended.
1346
+ - One final confidential-output acknowledgement remains at download.
1347
+ - No Scrub Key schema, helper semantics, export filenames/MIME types, cloud, AI, OCR or restored-PDF behavior changed.
1348
+ - Human review remains required; no production-readiness claim is made.
1349
+
1350
+ Files added:
1351
+ - `workpackage_claims/scrub_wp_mvp_reinsert_auto_flow_simplification_app_verify_closeout.md`
1352
+ - `handover/workpackages/20260727_1828_mvp_reinsert_auto_flow_simplification_app_verify_closeout.md`
1353
+
1354
+ Files changed:
1355
+ - `WORKPACKAGES.md`
1356
+ - `CHANGELOG.md`
1357
+ - `ROADMAP.md`
1358
+ - `workpackage_claims/scrub_wp_mvp_reinsert_auto_flow_simplification_implementation.md`
1359
+ - `handover/workpackages/20260727_1706_mvp_reinsert_auto_flow_simplification_implementation.md`
1360
+
1361
+ Next recommended step:
1362
+ - Start `SCRUB-WP_MVP_SCRUB_KEY_ROUNDTRIP_VALIDATION`.
1363
+
1364
+ ---
1365
+
1366
+ ## 2026-07-27 — SCRUB-WP_MVP_REINSERT_AUTO_FLOW_SIMPLIFICATION_IMPLEMENTATION
1367
+
1368
+ Status: implemented; full suite passed; final PR validation pending.
1369
+
1370
+ Purpose:
1371
+
1372
+ - Remove redundant source- and Scrub-Key confirmation steps from the local reinsert workflow after live Phase 6 evidence showed that uploaded inputs looked complete while hidden action gates remained.
1373
+ - Present the workflow in the user’s natural order: source document/text, corresponding Scrub Key, restored result.
1374
+ - Preserve a clear confidentiality decision at the final restored-output download boundary.
1375
+
1376
+ Files added:
1377
+
1378
+ - `reinsert_auto_flow.py`
1379
+ - `tests/test_reinsert_auto_flow.py`
1380
+ - `tests/test_reinsert_auto_flow_ui.py`
1381
+ - `handover/workpackages/20260727_1706_mvp_reinsert_auto_flow_simplification_implementation.md`
1382
+ - `output/validation/mvp_reinsert_auto_flow_validation.json`
1383
+
1384
+ Files changed:
1385
+
1386
+ - `reinsert_mode_ui.py`
1387
+ - `tests/test_reinsert_interface_simplification_ui.py`
1388
+ - `tests/test_mvp_document_fidelity_ui_copy.py`
1389
+ - `tests/test_mvp_document_fidelity_pr_final_contracts.py`
1390
+ - `CHANGELOG.md`
1391
+ - `WORKPACKAGES.md`
1392
+ - `ROADMAP.md`
1393
+ - `RELEASE_NOTES.md`
1394
+ - `DECISION_LOG.md`
1395
+ - `RISK_REGISTER.md`
1396
+ - `workpackage_claims/scrub_wp_mvp_document_hygiene_fidelity_hardening.md`
1397
+ - `handover/workpackages/20260717_2230_mvp_document_hygiene_fidelity_hardening.md`
1398
+ - `workpackage_claims/scrub_wp_mvp_reinsert_auto_flow_simplification_implementation.md`
1399
+
1400
+ Implementation result:
1401
+
1402
+ - Step 1 is now the source document or pasted text.
1403
+ - One uploader recognises TXT, DOCX and text-based PDF by extension.
1404
+ - Step 2 automatically parses and validates the uploaded or pasted Scrub Key.
1405
+ - Local deterministic reinsert runs automatically once one valid source and one valid key are present.
1406
+ - Separate source/key acknowledgement checkboxes and execution buttons were removed.
1407
+ - One final confidentiality acknowledgement remains directly before restored-output download.
1408
+ - Existing output filenames, MIME types, reinsert helpers, audit fields and explicit DOCX/PDF boundaries are preserved.
1409
+
1410
+ Validation:
1411
+
1412
+ - Full repository suite: 797 passed.
1413
+ - Helper dispatch, deterministic request signatures and input precedence are covered.
1414
+ - Source-level UI contracts verify document-first order, automatic key validation, automatic local reinsert and removal of redundant gates.
1415
+ - Prior DOCX live verification passed for body, table, header and footer restoration.
1416
+ - Final GitHub Actions, merge, Hugging Face sync and live app verification remain pending.
1417
+
1418
+ Intentionally not changed:
1419
+
1420
+ - Scrub Key schema, mappings, lifecycle or storage;
1421
+ - document replacement or reinsert helper semantics;
1422
+ - recognizers or thresholds;
1423
+ - export filenames, MIME types or audit semantics;
1424
+ - cloud, AI or OCR processing;
1425
+ - restored-PDF support;
1426
+ - unsupported DOCX-part boundaries;
1427
+ - the requirement for human review and a final confidential-output warning.
1428
+
1429
+ Next recommended step:
1430
+
1431
+ - Complete final PR validation, merge and sync, then live-verify the three-step automatic flow.
1432
+ - Continue with `SCRUB-WP_MVP_SCRUB_KEY_ROUNDTRIP_VALIDATION` after app verification.
1433
+
1434
+ ---
1435
+
1436
+ ## 2026-07-17 — SCRUB-WP_MVP_DOCUMENT_HYGIENE_FIDELITY_HARDENING
1437
+
1438
+ Status: completed / ready for PR verification.
1439
+
1440
+ Purpose:
1441
+
1442
+ - Resolve the reproducible DOCX header/footer reinsert fidelity gap from the Phase 6 matrix.
1443
+ - Preserve DOCX hygiene visibility and explicit unsupported-part boundaries.
1444
+ - Keep the PDF restored-TXT-only/no-OCR boundary unchanged.
1445
+
1446
+ Files added:
1447
+
1448
+ - `mvp_document_fidelity_report.py`
1449
+ - `scripts/run_mvp_document_hygiene_fidelity_report.py`
1450
+ - `tests/test_mvp_document_hygiene_fidelity_hardening.py`
1451
+ - `tests/test_mvp_document_fidelity_report.py`
1452
+ - `tests/test_mvp_document_fidelity_ui_copy.py`
1453
+ - `tests/test_mvp_document_fidelity_pr_final_contracts.py`
1454
+ - `output/validation/mvp_phase6_document_hygiene_fidelity_hardening_report.json`
1455
+ - `output/validation/mvp_document_fidelity_pr_validation.json`
1456
+ - `output/validation/mvp_document_fidelity_pr_validation.log`
1457
+ - `handover/workpackages/20260717_2230_mvp_document_hygiene_fidelity_hardening.md`
1458
+
1459
+ Files changed:
1460
+
1461
+ - `scrub_key_document_reinsert.py`
1462
+ - `reinsert_mode_ui.py`
1463
+ - `mvp_phase6_document_cases.py`
1464
+ - `tests/test_mvp_phase6_e2e_synthetic_validation_matrix.py`
1465
+ - `CHANGELOG.md`
1466
+ - `WORKPACKAGES.md`
1467
+ - `RELEASE_NOTES.md`
1468
+ - `DECISION_LOG.md`
1469
+ - `RISK_REGISTER.md`
1470
+ - `workpackage_claims/scrub_wp_mvp_document_hygiene_fidelity_hardening.md`
1471
+
1472
+ Implementation result:
1473
+
1474
+ - DOCX body paragraphs and tables remain supported.
1475
+ - `word/header*.xml` and `word/footer*.xml` text nodes are restored deterministically.
1476
+ - The DOCX reinsert capability copy matches the supported body/table/header/footer scope.
1477
+ - The synthetic header/footer residual-placeholder finding is resolved: `true`.
1478
+ - Resolved findings: 1.
1479
+ - Remaining findings: 1.
1480
+ - The remaining finding is the explicit PDF restored-TXT-only/no-OCR product boundary.
1481
+
1482
+ Intentionally not changed:
1483
+
1484
+ - recognizers, thresholds or replacement semantics;
1485
+ - Scrub Key schema or lifecycle;
1486
+ - DOCX comments, tracked-change-only parts, footnotes/endnotes, text boxes or metadata;
1487
+ - split-placeholder support across Word text nodes;
1488
+ - export filenames or MIME types;
1489
+ - restored PDF or OCR support;
1490
+ - Streamlit controls/flow, runtime or dependencies; only capability copy was aligned.
1491
+
1492
+ Next recommended step:
1493
+
1494
+ - After Actions, sync and app verification, start `SCRUB-WP_MVP_SCRUB_KEY_ROUNDTRIP_VALIDATION`.
1495
+
1496
+ ---
1497
+
1498
+ ## 2026-07-17 — Hugging Face Space runtime incident recovery and sync-churn guard
1499
+
1500
+ Status: completed and app-verified.
1501
+
1502
+ Purpose:
1503
+
1504
+ - Restore the Hugging Face Space after it entered an error/rebuild state.
1505
+ - Diagnose the incident without exposing secrets or changing product behavior.
1506
+ - Prevent clearly non-runtime-only commits from repeatedly rebuilding the live Space.
1507
+
1508
+ Result:
1509
+
1510
+ - Sanitized runtime evidence observed the Space first at `BUILDING` and subsequently at `RUNNING`.
1511
+ - Streamlit started on port 7860 and the Flair model loaded.
1512
+ - The coordinator confirmed that the application opens again.
1513
+ - PR #35 added a conservative deployment `paths-ignore` guard while preserving runtime-relevant deployments and manual dispatch.
1514
+ - Temporary incident recovery/probe workflows and triggers were removed after verification.
1515
+
1516
+ Intentionally not changed:
1517
+
1518
+ - product code or UI;
1519
+ - recognizers, thresholds or replacement semantics;
1520
+ - export, Scrub Key or reinsert semantics;
1521
+ - dependencies, Dockerfile, hardware or Hugging Face configuration;
1522
+ - privacy and human-review controls.
1523
+
1524
+ Next recommended step:
1525
+
1526
+ - Resume PR #33 and the Phase 6 document-fidelity sequence.
1527
+
1528
+ ---
1529
+
1530
+ ## 2026-07-17 — SCRUB-WP_MVP_FALSE_NEGATIVE_GAP_TRIAGE
1531
+
1532
+ Status: completed / ready for PR verification.
1533
+
1534
+ Purpose:
1535
+
1536
+ - Classify every evidence gap from the first Phase 6 synthetic validation report.
1537
+ - Decide whether the evidence justifies recognizer or threshold changes.
1538
+ - Route document-fidelity and product-boundary findings to the correct next package.
1539
+
1540
+ Files added:
1541
+
1542
+ - `MVP_PHASE6_FALSE_NEGATIVE_GAP_TRIAGE.md`
1543
+ - `output/validation/mvp_phase6_false_negative_gap_triage.json`
1544
+ - `tests/test_mvp_phase6_false_negative_gap_triage.py`
1545
+ - `handover/workpackages/20260717_2208_mvp_false_negative_gap_triage.md`
1546
+
1547
+ Files changed:
1548
+
1549
+ - `CHANGELOG.md`
1550
+ - `WORKPACKAGES.md`
1551
+ - `DECISION_LOG.md`
1552
+ - `RISK_REGISTER.md`
1553
+ - `workpackage_claims/scrub_wp_mvp_false_negative_gap_triage.md`
1554
+
1555
+ Triage result:
1556
+
1557
+ - Input evidence gaps: 2.
1558
+ - Reproducible detection false negatives: 0.
1559
+ - Misclassifications: 0.
1560
+ - Legal-role over-masking findings: 0.
1561
+ - Recognizer fix required: `false`.
1562
+ - Next package: `SCRUB-WP_MVP_DOCUMENT_HYGIENE_FIDELITY_HARDENING`.
1563
+
1564
+ Decision:
1565
+
1566
+ - The DOCX finding is a header/footer reinsert and fidelity-scope issue, not a detection failure.
1567
+ - The PDF finding is the approved restored-TXT-only/no-OCR product boundary, not a detection failure.
1568
+ - No recognizer implementation package is opened from this evidence.
1569
+
1570
+ Intentionally not changed:
1571
+
1572
+ - product recognizers or thresholds;
1573
+ - replacement semantics;
1574
+ - document processing or reinsert behavior;
1575
+ - export, Scrub Key or audit semantics;
1576
+ - UI, runtime or dependencies.
1577
+
1578
+ Next recommended step:
1579
+
1580
+ - Start `SCRUB-WP_MVP_DOCUMENT_HYGIENE_FIDELITY_HARDENING`.
1581
+
1582
+ ---
1583
+
1584
+ ## 2026-07-17 — SCRUB-WP_MVP_E2E_SYNTHETIC_VALIDATION_MATRIX
1585
+
1586
+ Status: completed / ready for PR verification.
1587
+
1588
+ Purpose:
1589
+
1590
+ - Establish a repeatable synthetic evidence baseline for the supported MVP workflow.
1591
+ - Exercise TXT, DOCX and text-based PDF paths across import, review-row replacement, manual addition, Scrub Key creation/validation, deterministic reinsert, export representations and audit evidence.
1592
+ - Record known limitations and reproducible gaps without weakening tests or making production-readiness claims.
1593
+
1594
+ Files added:
1595
+
1596
+ - `test_cases/mvp_phase6/validation_manifest.json`
1597
+ - `mvp_phase6_validation_manifest.py`
1598
+ - `mvp_phase6_detection_matrix.py`
1599
+ - `mvp_phase6_workflow_core.py`
1600
+ - `mvp_phase6_document_cases.py`
1601
+ - `mvp_phase6_validation_report.py`
1602
+ - `scripts/run_mvp_phase6_validation_matrix.py`
1603
+ - `tests/test_mvp_phase6_e2e_synthetic_validation_matrix.py`
1604
+ - `output/validation/mvp_phase6_synthetic_validation_report.json`
1605
+ - `handover/workpackages/20260717_2020_mvp_e2e_synthetic_validation_matrix.md`
1606
+
1607
+ Files changed:
1608
+
1609
+ - `CHANGELOG.md`
1610
+ - `WORKPACKAGES.md`
1611
+ - `RISK_REGISTER.md`
1612
+ - `workpackage_claims/scrub_wp_mvp_e2e_synthetic_validation_matrix.md`
1613
+
1614
+ Validation result:
1615
+
1616
+ - Synthetic cases: 3.
1617
+ - Failing cases: 0.
1618
+ - Evidence gaps/known limitations: 2.
1619
+ - Gap categories: known_docx_reinsert_limitation, known_pdf_reinsert_limitation.
1620
+ - Human review required: `true`.
1621
+ - Production ready: `false`.
1622
+ - Local-only validation: `true`.
1623
+ - External AI/cloud/OCR processing: none.
1624
+
1625
+ Methodology correction:
1626
+
1627
+ - Standard deterministic Presidio email recognition is included alongside the Dutch recognizer pack, preventing a standard e-mail value from being misclassified as a Dutch-pack false negative.
1628
+
1629
+ Intentionally not changed:
1630
+
1631
+ - Streamlit UI or review controls;
1632
+ - recognizers or detection thresholds in product code;
1633
+ - replacement semantics;
1634
+ - export payload, filename or MIME semantics;
1635
+ - Scrub Key schema or lifecycle behavior;
1636
+ - reinsert semantics;
1637
+ - document-processing implementation;
1638
+ - runtime/startup or dependencies.
1639
+
1640
+ Next recommended step:
1641
+
1642
+ - Start `SCRUB-WP_MVP_FALSE_NEGATIVE_GAP_TRIAGE` and classify the report evidence before implementing any fix.
1643
+
1644
+ ---
1645
+
1646
+ ## 2026-07-17 — SCRUB-WP_MVP_PHASE6_ROADMAP_REALIGNMENT
1647
+
1648
+ Status: completed / ready for PR verification.
1649
+
1650
+ Purpose:
1651
+
1652
+ - Realign the central roadmap after completion and live verification of the MVP UI simplification line.
1653
+ - Make Phase 6 end-to-end workflow validation and trust hardening the active development line.
1654
+ - Define an ordered evidence-driven workpackage queue before pilot or packaging work resumes.
1655
+
1656
+ Files changed:
1657
+
1658
+ - `ROADMAP.md`
1659
+ - `WORKPACKAGES.md`
1660
+ - `CHANGELOG.md`
1661
+ - `DECISION_LOG.md`
1662
+ - `RISK_REGISTER.md`
1663
+ - `workpackage_claims/scrub_wp_mvp_phase6_roadmap_realignment.md`
1664
+
1665
+ Files added:
1666
+
1667
+ - `MVP_PHASE6_EXECUTION_PLAN.md`
1668
+ - `handover/workpackages/20260717_2012_mvp_phase6_roadmap_realignment.md`
1669
+
1670
+ Main changes:
1671
+
1672
+ - The verified UI baseline is no longer the active development focus.
1673
+ - Phase 6 starts with a synthetic end-to-end validation matrix.
1674
+ - False-negative, document-hygiene, Scrub Key/roundtrip and audit work must be driven by reproducible evidence.
1675
+ - Phase 7 pilots and local packaging remain gated.
1676
+
1677
+ Validation status:
1678
+
1679
+ - Documentation-only package.
1680
+ - GitHub Actions pending after PR.
1681
+ - Hugging Face functional sync not applicable.
1682
+ - App verification not applicable.
1683
+
1684
+ Intentionally not changed:
1685
+
1686
+ - product code or tests;
1687
+ - UI behavior;
1688
+ - recognizers or replacement semantics;
1689
+ - export payloads, filenames or MIME types;
1690
+ - Scrub Key JSON or reinsert behavior;
1691
+ - document processing, runtime/startup or dependencies.
1692
+
1693
+ Next recommended step:
1694
+
1695
+ - Start `SCRUB-WP_MVP_E2E_SYNTHETIC_VALIDATION_MATRIX` after this realignment is merged.
1696
+
1697
+ ---
1698
+
1699
+ ## 2026-07-16 — SCRUB-WP_MANUAL_CORRECTION_PANEL_DENSITY_SIMPLIFICATION_APP_VERIFY_CLOSEOUT
1700
+
1701
+ Status: completed and app-verified.
1702
+
1703
+ Purpose:
1704
+
1705
+ - Record live Hugging Face app verification after PR #28 merged.
1706
+ - Close the manual correction panel density simplification line.
1707
+ - Confirm the compact layout preserves the existing manual correction workflow.
1708
+
1709
+ Verification evidence:
1710
+
1711
+ - Coordinator live app screenshot reviewed at 2026-07-16 23:43 Europe/Amsterdam.
1712
+ - `Gemiste waarde toevoegen` remains collapsed by default and opens without a duplicate internal heading.
1713
+ - The value, type and replacement controls appear in one compact row.
1714
+ - The full-width `Toevoegen aan vervangtabel` action remains visible.
1715
+ - Synthetic value `lantaarnbloem` was added successfully.
1716
+ - The replacement table shows `lantaarnbloem` with `[WAARDE_HANDMATIG_01]` and status `Handmatig toegevoegd`.
1717
+ - No Script execution error is visible.
1718
+ - The live screenshot confirms GitHub-to-Hugging-Face deployment of the merged UI.
1719
+
1720
+ Files changed:
1721
+
1722
+ - `CHANGELOG.md`
1723
+ - `WORKPACKAGES.md`
1724
+ - `workpackage_claims/scrub_wp_manual_correction_panel_density_simplification_implementation.md`
1725
+ - `handover/workpackages/20260716_2040_manual_correction_panel_density_implementation.md`
1726
+
1727
+ Files added:
1728
+
1729
+ - `workpackage_claims/scrub_wp_manual_correction_panel_density_simplification_app_verify_closeout.md`
1730
+ - `handover/workpackages/20260716_2343_manual_correction_panel_density_app_verify_closeout.md`
1731
+
1732
+ Intentionally not changed:
1733
+
1734
+ - product code or tests;
1735
+ - recognizer or replacement semantics;
1736
+ - validation or session-state behavior;
1737
+ - export payloads, filenames or MIME types;
1738
+ - Scrub Key JSON or reinsert behavior;
1739
+ - document processing, startup/runtime or dependencies.
1740
+
1741
+ Next recommended step:
1742
+
1743
+ - Use the simplified MVP UI with representative synthetic legal documents before approving another UI package.
1744
+
1745
+ ---
1746
+
1747
+ ## 2026-07-16 — SCRUB-WP_MANUAL_CORRECTION_PANEL_DENSITY_SIMPLIFICATION_IMPLEMENTATION
1748
+
1749
+ Status: implemented; local validation passed.
1750
+
1751
+ Purpose:
1752
+
1753
+ - Make the existing `Gemiste waarde toevoegen` panel materially shorter and less form-like.
1754
+ - Remove the duplicate internal heading and group value, type and replacement controls in one compact row.
1755
+ - Preserve the existing validation, session-state and replacement-table workflow.
1756
+
1757
+ Files changed:
1758
+
1759
+ - `presidio_streamlit.py`
1760
+ - `CHANGELOG.md`
1761
+ - `WORKPACKAGES.md`
1762
+ - `RELEASE_NOTES.md`
1763
+ - `workpackage_claims/scrub_wp_manual_correction_panel_density_simplification_implementation.md`
1764
+
1765
+ Files added:
1766
+
1767
+ - `tests/test_manual_correction_panel_density_implementation.py`
1768
+ - `handover/workpackages/20260716_2040_manual_correction_panel_density_implementation.md`
1769
+
1770
+ Validation status:
1771
+
1772
+ - Required worker validation passed.
1773
+ - GitHub Actions pending after PR update.
1774
+ - Hugging Face sync pending after merge.
1775
+ - Live app verification required because visible UI behavior changed.
1776
+
1777
+ Intentionally not changed:
1778
+
1779
+ - validation rules or duplicate detection;
1780
+ - placeholder generation or entity types;
1781
+ - replacement-row structure or replacement semantics;
1782
+ - export payloads, filenames or MIME types;
1783
+ - Scrub Key JSON or warning behavior;
1784
+ - reinsert behavior;
1785
+ - recognizers, thresholds, document processing, runtime/startup or dependencies.
1786
+
1787
+ Next recommended step:
1788
+
1789
+ - Verify PR Actions, merge when green, verify Hugging Face sync and request live app verification.
1790
+
1791
+ ---
1792
+
1793
+ ## 2026-07-05 — SCRUB-WP_REVIEW_EXPORT_VERTICAL_DENSITY_SIMPLIFICATION_APP_VERIFY_CLOSEOUT
1794
+
1795
+ Status: completed and app-verified.
1796
+
1797
+ Purpose:
1798
+
1799
+ - Record live app verification for `SCRUB-WP_REVIEW_EXPORT_VERTICAL_DENSITY_SIMPLIFICATION_IMPLEMENTATION`.
1800
+ - Close the Review/Export vertical-density implementation line after PR #26 merge and live Hugging Face verification.
1801
+ - Add concise closeout notes for the related plan and contract-test packages that were previously missing from `CHANGELOG.md`.
1802
+
1803
+ Verification evidence:
1804
+
1805
+ - Coordinator live Hugging Face screenshot reviewed.
1806
+ - The app starts without Script execution error.
1807
+ - The input section remains coherent.
1808
+ - `2. Controleer resultaat` remains visible.
1809
+ - Basiscontrole / Expertcontrole remain visible.
1810
+ - `Markeringen tonen` and side-by-side review remain visible.
1811
+ - `Gemiste waarde toevoegen` and the vervangtabel remain accessible.
1812
+ - `3. Exporteer resultaat` remains visible.
1813
+ - TXT/DOCX/PDF downloads are visible in a compact row.
1814
+ - Scrub Key, audit/technical files and DOCX hygiene audit remain separate and accessible.
1815
+
1816
+ Files changed:
1817
+
1818
+ - `CHANGELOG.md`
1819
+ - `WORKPACKAGES.md`
1820
+ - `workpackage_claims/scrub_wp_review_export_vertical_density_simplification_implementation.md`
1821
+ - `workpackage_claims/scrub_wp_review_export_vertical_density_simplification_app_verify_closeout.md`
1822
+ - `handover/workpackages/20260705_2213_review_export_vertical_density_implementation.md`
1823
+ - `handover/workpackages/20260705_2242_review_export_vertical_density_app_verify_closeout.md`
1824
+
1825
+ Intentionally not changed:
1826
+
1827
+ - product code;
1828
+ - tests;
1829
+ - recognizer logic;
1830
+ - replacement logic;
1831
+ - export payloads;
1832
+ - download filenames;
1833
+ - MIME types;
1834
+ - Scrub Key JSON;
1835
+ - reinsert behavior;
1836
+ - startup/runtime behavior.
1837
+
1838
+ Related package closeout:
1839
+
1840
+ - `SCRUB-WP_REVIEW_EXPORT_VERTICAL_DENSITY_SIMPLIFICATION_PLAN` — completed and merged; planning-only.
1841
+ - `SCRUB-WP_REVIEW_EXPORT_VERTICAL_DENSITY_SIMPLIFICATION_CONTRACT_TESTS` — completed and merged; source-level guardrails only.
1842
+
1843
+ Next recommended step:
1844
+
1845
+ - Decide whether the current MVP UI is good enough for this pass, or start a new separately approved small UI package.
1846
+
1847
+ ---
1848
+
1849
+ ## 2026-07-05 — SCRUB-WP_REVIEW_EXPORT_VERTICAL_DENSITY_SIMPLIFICATION_IMPLEMENTATION
1850
+
1851
+ Status: implemented / local validation passed.
1852
+
1853
+ Purpose:
1854
+
1855
+ - Reduce vertical density in the Review and Export areas after plan and contract-test packages were merged.
1856
+ - Compress repeated Review helper copy while keeping side-by-side review, marker toggle, manual missed-value entry and replacement table accessible.
1857
+ - Put the three primary document downloads in a compact three-column layout.
1858
+ - Keep Scrub Key, audit/technical files and DOCX hygiene audit separate and accessible.
1859
+
1860
+ Files changed:
1861
+
1862
+ - `presidio_streamlit.py`
1863
+ - `side_by_side_review_panel_ui.py`
1864
+ - `CHANGELOG.md`
1865
+ - `WORKPACKAGES.md`
1866
+ - `workpackage_claims/scrub_wp_review_export_vertical_density_simplification_implementation.md`
1867
+
1868
+ Files added:
1869
+
1870
+ - `handover/workpackages/20260705_2213_review_export_vertical_density_implementation.md`
1871
+
1872
+ Validation status:
1873
+
1874
+ - Local validation passed.
1875
+ - GitHub Actions pending after PR.
1876
+ - Hugging Face sync pending after merge.
1877
+ - App verification required after sync because visible UI behavior changed.
1878
+
1879
+ Intentionally not changed:
1880
+
1881
+ - recognizer logic;
1882
+ - replacement logic;
1883
+ - review table semantics;
1884
+ - export payloads;
1885
+ - download filenames;
1886
+ - MIME types;
1887
+ - Scrub Key JSON semantics;
1888
+ - Scrub Key warning meaning;
1889
+ - reinsert behavior;
1890
+ - DOCX/PDF parsing;
1891
+ - startup/runtime behavior;
1892
+ - dependencies;
1893
+ - benchmark or recall logic.
1894
+
1895
+ Next recommended step:
1896
+
1897
+ - Open PR, verify GitHub Actions, merge when green, verify Hugging Face sync, then request live app verification.
1898
+
1899
+ ---
1900
+
1901
+ ## 2026-07-05 — SCRUB-WP_DUPLICATE_INPUT_SURFACE_SIMPLIFICATION_IMPLEMENTATION
1902
+
1903
+ Status: implemented / local validation passed.
1904
+
1905
+ Purpose:
1906
+
1907
+ - Resume the duplicate input surface implementation after the connector-only worker was blocked.
1908
+ - Keep one visible `1. Voeg document of tekst toe` step and group upload, synthetic example selection and pasted/extracted text into one input surface.
1909
+ - Preserve TXT/DOCX/PDF upload support, synthetic legal examples, pasted/extracted text handling and input precedence.
1910
+ - Preserve review, export/download, Scrub Key, reinsert, audit and DOCX hygiene behavior.
1911
+
1912
+ Files changed:
1913
+
1914
+ - `presidio_streamlit.py`
1915
+ - `tests/test_duplicate_input_surface_simplification_contracts.py`
1916
+ - `WORKPACKAGES.md`
1917
+ - `CHANGELOG.md`
1918
+ - `workpackage_claims/scrub_wp_duplicate_input_surface_simplification_implementation.md`
1919
+
1920
+ Files added:
1921
+
1922
+ - `handover/workpackages/20260705_0113_duplicate_input_surface_implementation.md`
1923
+
1924
+ Validation status:
1925
+
1926
+ - Local validation passed:
1927
+ - `python -m pytest -q tests/test_duplicate_input_surface_simplification_contracts.py`
1928
+ - related UI/export guardrail tests
1929
+ - `git diff --check`
1930
+ - GitHub Actions pending after PR.
1931
+ - Hugging Face sync pending after merge.
1932
+ - App verification required after merge/sync because visible UI grouping changed.
1933
+
1934
+ Intentionally not changed:
1935
+
1936
+ - document parsing behavior;
1937
+ - upload backend;
1938
+ - recognizer logic;
1939
+ - replacement logic;
1940
+ - export content;
1941
+ - download filenames;
1942
+ - download MIME types;
1943
+ - Scrub Key JSON semantics;
1944
+ - reinsert behavior;
1945
+ - runtime/startup behavior;
1946
+ - dependencies.
1947
+
1948
+ Next recommended step:
1949
+
1950
+ - Open PR, verify GitHub Actions, merge when green, verify Hugging Face sync, and request live app verification.
1951
+
1952
+ ---
1953
+
1954
+ ## 2026-07-04 — SCRUB-WP_DUPLICATE_INPUT_SURFACE_SIMPLIFICATION_CONTRACT_TESTS
1955
+
1956
+ Status: completed / PR validation pending.
1957
+
1958
+ Purpose:
1959
+
1960
+ - Lock the single-input-surface contract before the duplicate input implementation touches `presidio_streamlit.py`.
1961
+ - Preserve existing TXT/DOCX/PDF upload support, synthetic legal example support, pasted/extracted text handling and input precedence.
1962
+ - Preserve review, replacement table, Scrub Key, export/download, audit and DOCX hygiene surfaces.
1963
+ - Block duplicate-input runtime/startup patching and prohibited scope expansion.
1964
+
1965
+ Files added:
1966
+
1967
+ - `tests/test_duplicate_input_surface_simplification_contracts.py`
1968
+ - `workpackage_claims/scrub_wp_duplicate_input_surface_simplification_contract_tests.md`
1969
+ - `handover/workpackages/20260704_2318_duplicate_input_surface_contract_tests.md`
1970
+
1971
+ Files changed:
1972
+
1973
+ - `WORKPACKAGES.md`
1974
+ - `CHANGELOG.md`
1975
+
1976
+ Validation status:
1977
+
1978
+ - Source-level contract tests added.
1979
+ - GitHub Actions pending after PR.
1980
+ - Hugging Face sync not applicable until merge.
1981
+ - App verification not applicable because this package changes no UI behavior.
1982
+
1983
+ Intentionally not changed:
1984
+
1985
+ - product implementation code;
1986
+ - Streamlit UI behavior;
1987
+ - document ingestion behavior;
1988
+ - export content;
1989
+ - download filenames;
1990
+ - download MIME types;
1991
+ - Scrub Key JSON semantics;
1992
+ - reinsert behavior;
1993
+ - recognizer logic;
1994
+ - benchmark logic;
1995
+ - DOCX/PDF parsing behavior;
1996
+ - runtime/startup behavior;
1997
+ - dependencies.
1998
+
1999
+ Next recommended step:
2000
+
2001
+ - Review PR validation.
2002
+ - If green, merge this contract-test package.
2003
+ - Then start `SCRUB-WP_DUPLICATE_INPUT_SURFACE_SIMPLIFICATION_IMPLEMENTATION` as the next narrow implementation package.
2004
+
2005
+ ---
2006
+
2007
+ ## 2026-07-03 — SCRUB-WP_SECONDARY_CONTROL_GROUPING_POLISH_IMPLEMENTATION
2008
+
2009
+ Status: implemented / PR validation pending.
2010
+
2011
+ Purpose:
2012
+
2013
+ - Make the secondary review controls under `2. Controleer resultaat` calmer and easier to understand.
2014
+ - Add a clear `Meer controleopties` grouping cue below the side-by-side review without introducing nested Streamlit expanders.
2015
+ - Preserve side-by-side review, manual missed-value entry, replacement table, serial review, Scrub Key, export/download, audit and DOCX hygiene controls.
2016
+
2017
+ Files changed:
2018
+
2019
+ - `side_by_side_review_panel_ui.py`
2020
+ - `RELEASE_NOTES.md`
2021
+ - `CHANGELOG.md`
2022
+ - `workpackage_claims/scrub_wp_secondary_control_grouping_polish_implementation.md`
2023
+
2024
+ Files added:
2025
+
2026
+ - `tests/test_secondary_control_grouping_polish_implementation.py`
2027
+ - `handover/workpackages/20260703_0000_secondary_control_grouping_polish_implementation.md`
2028
+
2029
+ Validation status:
2030
+
2031
+ - Source-level implementation tests added.
2032
+ - GitHub Actions pending after PR.
2033
+ - Hugging Face sync pending after merge.
2034
+ - App verification required after merge/sync because visible UI behavior changed.
2035
+
2036
+ Intentionally not changed:
2037
+
2038
+ - replacement logic;
2039
+ - review table data semantics;
2040
+ - export content;
2041
+ - download filenames;
2042
+ - download MIME types;
2043
+ - Scrub Key JSON semantics;
2044
+ - Scrub Key warning meaning;
2045
+ - reinsert behavior;
2046
+ - recognizer logic;
2047
+ - benchmark logic;
2048
+ - DOCX/PDF parsing behavior;
2049
+ - runtime/startup behavior;
2050
+ - dependencies.
2051
+
2052
+ Next recommended step:
2053
+
2054
+ - Review PR validation.
2055
+ - If green, merge and verify Hugging Face sync.
2056
+ - Then request coordinator live app verification for the new `Meer controleopties` grouping cue.
2057
+
2058
+ ---
2059
+
2060
+ ## 2026-07-03 — SCRUB-WP_SECONDARY_CONTROL_GROUPING_POLISH
2061
+
2062
+ Status: implemented planning/contract-tests-only; PR validation pending.
2063
+
2064
+ Purpose:
2065
+
2066
+ - Prepare the final small UI-polish step for calmer grouping of secondary review controls.
2067
+ - Avoid nested Streamlit expanders before touching `presidio_streamlit.py`.
2068
+ - Preserve side-by-side review, manual missed-value entry, replacement table, serial review, Scrub Key, export/download, audit and DOCX hygiene controls.
2069
+
2070
+ Files added:
2071
+
2072
+ - `SECONDARY_CONTROL_GROUPING_POLISH_PLAN.md`
2073
+ - `tests/test_secondary_control_grouping_polish_contracts.py`
2074
+ - `workpackage_claims/scrub_wp_secondary_control_grouping_polish.md`
2075
+ - `handover/workpackages/20260703_0000_secondary_control_grouping_polish.md`
2076
+
2077
+ Validation status:
2078
+
2079
+ - Source-level contract tests added.
2080
+ - GitHub Actions pending after PR.
2081
+ - Hugging Face sync not applicable until merge.
2082
+ - App verification not applicable because no UI behavior changed in this planning/contract step.
2083
+
2084
+ Intentionally not changed:
2085
+
2086
+ - product code;
2087
+ - Streamlit UI;
2088
+ - replacement logic;
2089
+ - review table data semantics;
2090
+ - export content;
2091
+ - download filenames;
2092
+ - download MIME types;
2093
+ - Scrub Key JSON semantics;
2094
+ - Scrub Key warning meaning;
2095
+ - reinsert behavior;
2096
+ - recognizer logic;
2097
+ - benchmark logic;
2098
+ - DOCX/PDF parsing behavior;
2099
+ - runtime/startup behavior;
2100
+ - dependencies.
2101
+
2102
+ Next recommended step:
2103
+
2104
+ - Run PR validation for the contract tests.
2105
+ - If green, merge this plan/contract package.
2106
+ - Then start `SCRUB-WP_SECONDARY_CONTROL_GROUPING_POLISH_IMPLEMENTATION` as the actual narrow UI implementation.
2107
+
2108
+ ---
2109
+
2110
+ ## 2026-07-03 — SCRUB-WP_REVIEW_SURFACE_SIMPLIFICATION_IMPLEMENTATION
2111
+
2112
+ Status: completed and app-verified.
2113
+
2114
+ Purpose:
2115
+
2116
+ - Make the side-by-side review surface calmer and less form-like.
2117
+ - Keep the source-vs-processed comparison central while pointing users toward the safe download step.
2118
+ - Preserve review table, manual missed-value entry, serial review, Scrub Key, export/download and audit controls.
2119
+
2120
+ Files changed:
2121
+
2122
+ - `side_by_side_review_panel_ui.py`
2123
+ - `tests/test_review_copy_polish_ui.py`
2124
+ - `tests/test_side_by_side_review_consolidation_dutch_sample.py`
2125
+ - `tests/test_side_by_side_review_ui_patch.py`
2126
+ - `WORKPACKAGES.md`
2127
+ - `RELEASE_NOTES.md`
2128
+ - `workpackage_claims/scrub_wp_review_surface_simplification_implementation.md`
2129
+
2130
+ Files added:
2131
+
2132
+ - `tests/test_review_surface_simplification_implementation.py`
2133
+ - `handover/workpackages/20260703_0000_review_surface_simplification_implementation.md`
2134
+
2135
+ Validation status:
2136
+
2137
+ - Source-level implementation tests added.
2138
+ - Related copy-polish and side-by-side tests updated.
2139
+ - PR #12 initially failed on stale copy expectations; a narrow test-expectation fix was applied.
2140
+ - PR #12 Tests passed after the narrow fix.
2141
+ - PR #12 merged to `main`.
2142
+ - Main Tests for commit `41cf304` passed.
2143
+ - GitHub to Hugging Face sync for commit `41cf304` passed.
2144
+ - App verification passed by coordinator screenshot.
2145
+
2146
+ Intentionally not changed:
2147
+
2148
+ - replacement logic;
2149
+ - review table data semantics;
2150
+ - export content;
2151
+ - download filenames;
2152
+ - download MIME types;
2153
+ - Scrub Key JSON semantics;
2154
+ - Scrub Key warning meaning;
2155
+ - reinsert behavior;
2156
+ - recognizer logic;
2157
+ - benchmark logic;
2158
+ - DOCX/PDF parsing behavior;
2159
+ - runtime/startup behavior;
2160
+ - dependencies.
2161
+
2162
+ Next recommended step:
2163
+
2164
+ - Do not start broader UI/reinsert/export work without a dedicated workpackage.
2165
+ - Decide whether further secondary-control grouping is desired as a separate small package, or return to recall/benchmark follow-up if product UI is good enough for the current MVP pass.
2166
+
2167
+ ---
2168
+
2169
+ ## 2026-06-23 20:52 Europe/Amsterdam — Full-suite validation update — SCRUB-WP_REINSERT_INTERFACE_SIMPLIFICATION_IMPLEMENTATION
2170
+
2171
+ ## 2026-06-23 — SCRUB-WP_DOCX_SIDE_BY_SIDE_TEXT_ORDER_TRIAGE
2172
+
2173
+ - Fixed DOCX plain-text extraction order for side-by-side preview.
2174
+ - DOCX body paragraphs and tables are now read in document XML order instead of all paragraphs first and all tables afterwards.
2175
+ - Added synthetic regression coverage for interleaved paragraph/table order.
2176
+ - Preserved DOCX export, Scrub Key and reinsert semantics.
2177
+ - Validation: `python -m pytest tests -x -vv` → 649 passed in 102.51s.
2178
+
2179
+ - Full suite passed: `python -m pytest tests -x -vv` → 647 passed in 108.30s.
2180
+ - `git diff --check` passed.
2181
+ - Local implementation validation complete.
2182
+ - GitHub Actions, GitHub to Hugging Face sync and live app verification remain pending until PR/merge/sync.
2183
+
2184
+ ## 2026-06-23 20:43 Europe/Amsterdam — SCRUB-WP_REINSERT_INTERFACE_SIMPLIFICATION_IMPLEMENTATION
2185
+
2186
+ - Implemented direct-source reinsert interface simplification.
2187
+ - Added `reinsert_mode_ui.py` with the visible four-step reinsert flow:
2188
+ 1. Voeg Scrub Key toe
2189
+ 2. Voeg tekst of document toe
2190
+ 3. Controleer herstelrapport
2191
+ 4. Download herstelde output
2192
+ - Added a minimal direct hook in `presidio_streamlit.py`.
2193
+ - Added a no-op guard to `fix_streamlit_pdf_text_reinsert.py` so startup does not mutate the direct-source reinsert UI.
2194
+ - Preserved Scrub Key warnings, acknowledgement gates, restored download filenames, MIME types and local-only/no-AI/no-cloud/no-OCR/no-restored-PDF boundaries.
2195
+ - Added `tests/test_reinsert_interface_simplification_ui.py`.
2196
+
2197
+ Validation so far:
2198
+ - `tests/test_reinsert_interface_simplification_ui.py`: 8 passed
2199
+ - reinsert patch tests: 39 passed
2200
+ - warning/two-mode UI tests: 23 passed
2201
+
2202
+
2203
+ # Changelog — SolidPrivacy Scrub
2204
+
2205
+ ## SCRUB-WP_EXECUTION_INTERFACE_SIMPLIFICATION_IMPLEMENTATION_RESTART — Execution interface simplification
2206
+
2207
+ Status: completed and verified on `main` via PR #6 (`a34700c`).
2208
+
2209
+ Summary:
2210
+
2211
+ - Simplified the default Scrub flow toward `1. Voeg document of tekst toe`, `2. Controleer resultaat`, `3. Exporteer resultaat`.
2212
+ - Edited `presidio_streamlit.py` directly; no startup patch, runtime hook, sitecustomize hook or Dockerfile startup change was added.
2213
+ - Collapsed secondary controls by default while keeping them available:
2214
+ - control-mode explanation;
2215
+ - recognition details;
2216
+ - review guidance;
2217
+ - manual missed-value entry;
2218
+ - focus filter / extra control helpers;
2219
+ - candidate audit values;
2220
+ - replacement table;
2221
+ - technical replacement details;
2222
+ - step-by-step review;
2223
+ - reusable replacements;
2224
+ - Scrub Key download;
2225
+ - audit/technical downloads.
2226
+ - Kept side-by-side review visible as the main review surface.
2227
+ - Kept the replacement table as source of truth and export input.
2228
+ - Kept primary document downloads visible.
2229
+
2230
+ Tests:
2231
+
2232
+ - `tests/test_execution_interface_simplification_ui.py` — 6 passed.
2233
+ - Side-by-side, serial review and replace logic UI tests — 37 passed.
2234
+ - Export/download contract and implementation tests — 19 passed.
2235
+ - `git diff --check` — passed.
2236
+ - Full local test suite — 639 passed.
2237
+ - PR #6 checks — green.
2238
+ - Main Tests — green.
2239
+ - GitHub to Hugging Face sync — green.
2240
+ - Live app verification — passed by coordinator screenshot on 2026-06-23.
2241
+
2242
+ Intentionally not changed:
2243
+
2244
+ - export semantics;
2245
+ - download file contents;
2246
+ - download filenames;
2247
+ - Scrub Key JSON semantics;
2248
+ - Scrub Key warning meaning;
2249
+ - reinsert behavior;
2250
+ - recognizer logic;
2251
+ - benchmark thresholds;
2252
+ - document processing behavior;
2253
+ - cloud processing;
2254
+ - local packaging;
2255
+ - Dockerfile startup behavior;
2256
+ - runtime mutation behavior;
tests/test_mvp_document_fidelity_pr_final_contracts.py CHANGED
@@ -3,10 +3,7 @@ from pathlib import Path
3
 
4
  ROOT = Path(__file__).resolve().parents[1]
5
  FIDELITY_HEADING = "## 2026-07-17 — SCRUB-WP_MVP_DOCUMENT_HYGIENE_FIDELITY_HARDENING"
6
- FIDELITY_WORKPACKAGE_HEADING = (
7
- "## 2026-07-17 22:30 Europe/Amsterdam — "
8
- "SCRUB-WP_MVP_DOCUMENT_HYGIENE_FIDELITY_HARDENING"
9
- )
10
 
11
 
12
  def _heading_section(text: str, heading: str) -> str:
@@ -20,9 +17,11 @@ def test_temporary_document_fidelity_patch_scripts_are_absent() -> None:
20
  assert not (ROOT / "scripts" / "finalize_mvp_document_fidelity_pr_cleanup.py").exists()
21
 
22
 
23
- def test_document_fidelity_governance_evidence_has_no_duplicate_lines() -> None:
24
- changelog = (ROOT / "CHANGELOG.md").read_text(encoding="utf-8")
25
- workpackages = (ROOT / "WORKPACKAGES.md").read_text(encoding="utf-8")
 
 
26
  handover = (
27
  ROOT
28
  / "handover"
@@ -30,22 +29,24 @@ def test_document_fidelity_governance_evidence_has_no_duplicate_lines() -> None:
30
  / "20260717_2230_mvp_document_hygiene_fidelity_hardening.md"
31
  ).read_text(encoding="utf-8")
32
 
33
- changelog_section = _heading_section(changelog, FIDELITY_HEADING)
34
- workpackage_section = _heading_section(
35
- workpackages,
36
- FIDELITY_WORKPACKAGE_HEADING,
37
- )
38
 
39
- assert changelog_section.count("- `tests/test_mvp_document_fidelity_ui_copy.py`") == 1
40
- assert changelog_section.count(
41
  "The DOCX reinsert capability copy matches the supported body/table/header/footer scope."
42
  ) == 1
43
- assert workpackage_section.count(
44
- "Aligned existing DOCX reinsert copy with the supported body/table/header/footer scope without adding controls."
45
- ) == 1
46
  assert handover.count("- `tests/test_mvp_document_fidelity_ui_copy.py`") == 1
47
  assert handover.count("- Capability-copy contract tests.") == 1
48
 
 
 
 
 
 
 
 
 
49
 
50
  def test_final_claim_preserves_verification_and_product_boundaries() -> None:
51
  claim = (
 
3
 
4
  ROOT = Path(__file__).resolve().parents[1]
5
  FIDELITY_HEADING = "## 2026-07-17 — SCRUB-WP_MVP_DOCUMENT_HYGIENE_FIDELITY_HARDENING"
6
+ D030_HEADING = "## D030 — 2026-07-17 — Supported DOCX reinsert includes body, tables, headers and footers"
 
 
 
7
 
8
 
9
  def _heading_section(text: str, heading: str) -> str:
 
17
  assert not (ROOT / "scripts" / "finalize_mvp_document_fidelity_pr_cleanup.py").exists()
18
 
19
 
20
+ def test_document_fidelity_history_is_preserved_and_current_scope_remains_bound() -> None:
21
+ archived_changelog = (
22
+ ROOT / "history" / "CHANGELOG_PRE_CONVERGENCE_20260904.md"
23
+ ).read_text(encoding="utf-8")
24
+ decisions = (ROOT / "DECISION_LOG.md").read_text(encoding="utf-8")
25
  handover = (
26
  ROOT
27
  / "handover"
 
29
  / "20260717_2230_mvp_document_hygiene_fidelity_hardening.md"
30
  ).read_text(encoding="utf-8")
31
 
32
+ archived_section = _heading_section(archived_changelog, FIDELITY_HEADING)
33
+ d030 = _heading_section(decisions, D030_HEADING)
 
 
 
34
 
35
+ assert archived_section.count("- `tests/test_mvp_document_fidelity_ui_copy.py`") == 1
36
+ assert archived_section.count(
37
  "The DOCX reinsert capability copy matches the supported body/table/header/footer scope."
38
  ) == 1
 
 
 
39
  assert handover.count("- `tests/test_mvp_document_fidelity_ui_copy.py`") == 1
40
  assert handover.count("- Capability-copy contract tests.") == 1
41
 
42
+ # Current product authority lives in D030, not in a historical WORKPACKAGES slot.
43
+ assert "body, tables, headers and footers" in d030
44
+ assert "word/document.xml" in d030
45
+ assert "word/header*.xml" in d030
46
+ assert "word/footer*.xml" in d030
47
+ assert "comments" in d030
48
+ assert "footnotes/endnotes" in d030
49
+
50
 
51
  def test_final_claim_preserves_verification_and_product_boundaries() -> None:
52
  claim = (
tests/test_premium_core_flow_ui_realignment_plan.py CHANGED
@@ -45,27 +45,37 @@ def test_plan_preserves_safety_and_roundtrip_semantics() -> None:
45
  assert required in text
46
 
47
 
48
- def test_roadmap_and_workpackages_route_the_new_ui_line_sequentially() -> None:
49
  roadmap = ROADMAP.read_text(encoding="utf-8")
50
  workpackages = WORKPACKAGES.read_text(encoding="utf-8")
51
-
52
- assert "Premium core-flow UI realignment" in roadmap
53
- assert "SCRUB-WP_PREMIUM_CORE_FLOW_UI_CONTRACT" in roadmap
54
- assert "SCRUB-WP_PREMIUM_CORE_FLOW_STATE_MODEL" in roadmap
55
- assert "SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION" in roadmap
56
- assert "SCRUB-WP_PREMIUM_CORE_FLOW_APP_VERIFY_CLOSEOUT" in roadmap
57
- assert "SCRUB-WP_PREMIUM_CORE_FLOW_UI_REALIGNMENT_PLAN" in workpackages
58
- assert "SCRUB-WP_PROCESSED_TEXT_SELECTION_CROSS_FLOW_REGRESSION" in workpackages
59
- assert "do not run the shared Streamlit UI packages in parallel" in workpackages
 
 
 
 
 
 
 
 
60
 
61
 
62
  def test_decision_log_records_global_view_and_stage_model() -> None:
63
  text = DECISIONS.read_text(encoding="utf-8")
64
 
65
- assert "D041" in text
66
- assert "application-wide Standard/Expert presentation model" in text
67
- assert "one active stage at a time" in text
68
- assert "visibility and grouping only" in text
 
 
69
 
70
 
71
  def test_claim_is_planning_only() -> None:
 
45
  assert required in text
46
 
47
 
48
+ def test_current_strategy_preserves_implemented_ui_contract_without_reactivating_old_queue() -> None:
49
  roadmap = ROADMAP.read_text(encoding="utf-8")
50
  workpackages = WORKPACKAGES.read_text(encoding="utf-8")
51
+ decisions = DECISIONS.read_text(encoding="utf-8")
52
+
53
+ assert "generation-bound Standard/Expert workflow state" in roadmap
54
+ assert "## D041" in decisions
55
+ assert "## D043" in decisions
56
+ assert "global `Standaard | Expert` presentation" in decisions
57
+ assert "exactly one Standard stage is dominant at a time" in decisions
58
+ assert "Shared Streamlit/review/export/runtime surfaces remain sequential" in workpackages
59
+
60
+ # Historical implementation packages are evidence, not the current queue.
61
+ for obsolete_current_package in [
62
+ "SCRUB-WP_PREMIUM_CORE_FLOW_UI_CONTRACT",
63
+ "SCRUB-WP_PREMIUM_CORE_FLOW_STATE_MODEL",
64
+ "SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION",
65
+ "SCRUB-WP_PREMIUM_CORE_FLOW_APP_VERIFY_CLOSEOUT",
66
+ ]:
67
+ assert obsolete_current_package not in workpackages
68
 
69
 
70
  def test_decision_log_records_global_view_and_stage_model() -> None:
71
  text = DECISIONS.read_text(encoding="utf-8")
72
 
73
+ assert "## D041" in text
74
+ assert "global `Standaard | Expert` presentation" in text
75
+ assert "presentation-only switching" in text
76
+ assert "## D043" in text
77
+ assert "exactly one Standard stage is dominant at a time" in text
78
+ assert "Standard is lower cognitive load, not lower safety" in text
79
 
80
 
81
  def test_claim_is_planning_only() -> None:
tests/test_premium_staged_workspace_decision.py CHANGED
@@ -59,30 +59,39 @@ def test_decision_preserves_state_safety_and_product_semantics() -> None:
59
  assert required in text
60
 
61
 
62
- def test_roadmap_makes_staged_workspace_the_urgent_premium_gate() -> None:
63
- text = ROADMAP.read_text(encoding="utf-8")
 
 
 
 
 
 
 
64
 
65
- assert "PREMIUM_STAGED_WORKSPACE_DECISION.md" in text
66
- assert "SCRUB-WP_PREMIUM_STAGED_WORKSPACE_DECISION_FREEZE — URGENT CURRENT PACKAGE" in text
67
- assert "production integration paused until package 3 is incorporated" in text
68
- assert "PR #85 is **amended, not discarded**" in text
69
- assert "one persistent document workspace" in text
70
- assert "no three routed pages" in text
71
 
72
 
73
- def test_workpackages_and_decision_log_bind_the_execution_order() -> None:
74
  workpackages = WORKPACKAGES.read_text(encoding="utf-8")
75
  decisions = DECISION_LOG.read_text(encoding="utf-8")
76
 
77
- assert "SCRUB-WP_PREMIUM_STAGED_WORKSPACE_DECISION_FREEZE" in workpackages
78
- assert "SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION" in workpackages
79
- assert "SCRUB-WP_PREMIUM_INPUT_STAGE_SIMPLIFICATION" in workpackages
80
- assert "SCRUB-WP_PREMIUM_REVIEW_STAGE_SIMPLIFICATION" in workpackages
81
- assert "SCRUB-WP_PREMIUM_EXPORT_STAGE_SIMPLIFICATION" in workpackages
82
- assert "SCRUB-WP_PREMIUM_EXPERT_PARITY_REGRESSION" in workpackages
83
- assert "SCRUB-WP_PREMIUM_CORE_FLOW_APP_VERIFY_CLOSEOUT" in workpackages
84
- assert "do not run the shared Streamlit UI packages in parallel" in workpackages
85
-
86
- assert "D043" in decisions
87
  assert "single-page staged document workspace" in decisions
88
- assert "three separate routed screens" in decisions
 
 
 
 
 
 
 
 
 
 
 
 
 
59
  assert required in text
60
 
61
 
62
+ def test_current_roadmap_preserves_staged_workspace_without_reactivating_old_gate() -> None:
63
+ roadmap = ROADMAP.read_text(encoding="utf-8")
64
+ decisions = DECISION_LOG.read_text(encoding="utf-8")
65
+
66
+ assert "generation-bound Standard/Expert workflow state" in roadmap
67
+ assert "Stage 1 — Repository Convergence — CURRENT" in roadmap
68
+ assert "## D043" in decisions
69
+ assert "single-page staged document workspace" in decisions
70
+ assert "three routed pages" in decisions
71
 
72
+ # Old implementation-gate prose is historical, not current strategy.
73
+ assert "SCRUB-WP_PREMIUM_STAGED_WORKSPACE_DECISION_FREEZE — URGENT CURRENT PACKAGE" not in roadmap
74
+ assert "production integration paused until package 3 is incorporated" not in roadmap
75
+ assert "PR #85 is **amended, not discarded**" not in roadmap
 
 
76
 
77
 
78
+ def test_current_workpackages_preserve_shared_surface_sequencing_not_historical_package_order() -> None:
79
  workpackages = WORKPACKAGES.read_text(encoding="utf-8")
80
  decisions = DECISION_LOG.read_text(encoding="utf-8")
81
 
82
+ assert "Shared Streamlit/review/export/runtime surfaces remain sequential" in workpackages
83
+ assert "## D043" in decisions
 
 
 
 
 
 
 
 
84
  assert "single-page staged document workspace" in decisions
85
+ assert "three separate routed screens" not in decisions
86
+ assert "three routed pages" in decisions
87
+
88
+ for obsolete_current_package in [
89
+ "SCRUB-WP_PREMIUM_STAGED_WORKSPACE_DECISION_FREEZE",
90
+ "SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION",
91
+ "SCRUB-WP_PREMIUM_INPUT_STAGE_SIMPLIFICATION",
92
+ "SCRUB-WP_PREMIUM_REVIEW_STAGE_SIMPLIFICATION",
93
+ "SCRUB-WP_PREMIUM_EXPORT_STAGE_SIMPLIFICATION",
94
+ "SCRUB-WP_PREMIUM_EXPERT_PARITY_REGRESSION",
95
+ "SCRUB-WP_PREMIUM_CORE_FLOW_APP_VERIFY_CLOSEOUT",
96
+ ]:
97
+ assert obsolete_current_package not in workpackages
tests/test_reinsert_auto_flow_app_verify_closeout.py CHANGED
@@ -4,10 +4,11 @@ from pathlib import Path
4
  ROOT = Path(__file__).resolve().parents[1]
5
 
6
 
7
- def test_reinsert_auto_flow_closeout_is_recorded() -> None:
8
- roadmap = (ROOT / "ROADMAP.md").read_text(encoding="utf-8")
9
- workpackages = (ROOT / "WORKPACKAGES.md").read_text(encoding="utf-8")
10
- changelog = (ROOT / "CHANGELOG.md").read_text(encoding="utf-8")
 
11
  claim = (
12
  ROOT
13
  / "workpackage_claims"
@@ -20,13 +21,19 @@ def test_reinsert_auto_flow_closeout_is_recorded() -> None:
20
  / "20260727_1706_mvp_reinsert_auto_flow_simplification_implementation.md"
21
  ).read_text(encoding="utf-8")
22
 
23
- assert "completed and app-verified" in roadmap
24
- assert "SCRUB-WP_MVP_SCRUB_KEY_ROUNDTRIP_VALIDATION" in roadmap
25
- assert "SCRUB-WP_MVP_REINSERT_AUTO_FLOW_SIMPLIFICATION_APP_VERIFY_CLOSEOUT" in workpackages
26
- assert "coordinator reported `getest en werkend`" in changelog
27
  assert "Status: completed and app-verified" in claim
28
  assert "Completed and app-verified." in handover
29
 
 
 
 
 
 
 
 
 
 
30
 
31
  def test_closeout_preserves_product_boundaries() -> None:
32
  closeout = (
 
4
  ROOT = Path(__file__).resolve().parents[1]
5
 
6
 
7
+ def test_reinsert_auto_flow_closeout_history_and_current_contract_are_preserved() -> None:
8
+ archived_changelog = (
9
+ ROOT / "history" / "CHANGELOG_PRE_CONVERGENCE_20260904.md"
10
+ ).read_text(encoding="utf-8")
11
+ decisions = (ROOT / "DECISION_LOG.md").read_text(encoding="utf-8")
12
  claim = (
13
  ROOT
14
  / "workpackage_claims"
 
21
  / "20260727_1706_mvp_reinsert_auto_flow_simplification_implementation.md"
22
  ).read_text(encoding="utf-8")
23
 
24
+ assert "coordinator reported `getest en werkend`" in archived_changelog
 
 
 
25
  assert "Status: completed and app-verified" in claim
26
  assert "Completed and app-verified." in handover
27
 
28
+ # Current authority lives in accepted decisions, not old roadmap/workpackage slots.
29
+ assert "## D031" in decisions
30
+ assert "source document/text" in decisions
31
+ assert "corresponding Scrub Key" in decisions
32
+ assert "deterministic validation/reinsert" in decisions
33
+ assert "meaningful confidentiality acknowledgement" in decisions
34
+ assert "## D037" in decisions
35
+ assert "fails closed with zero replacements" in decisions
36
+
37
 
38
  def test_closeout_preserves_product_boundaries() -> None:
39
  closeout = (
tests/test_repository_convergence_bootstrap_contracts.py ADDED
@@ -0,0 +1,228 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import re
2
+ from pathlib import Path
3
+
4
+
5
+ ROOT = Path(__file__).resolve().parents[1]
6
+
7
+
8
+ def read(name: str) -> str:
9
+ return (ROOT / name).read_text(encoding="utf-8")
10
+
11
+
12
+ def test_roadmap_has_only_the_five_current_macro_stages() -> None:
13
+ roadmap = read("ROADMAP.md")
14
+
15
+ stage_headings = re.findall(
16
+ r"^## Stage (\d+) — (.+?)(?: — CURRENT)?$",
17
+ roadmap,
18
+ flags=re.MULTILINE,
19
+ )
20
+ assert stage_headings == [
21
+ ("1", "Repository Convergence"),
22
+ ("2", "Scrub Private Application"),
23
+ ("3", "Private Service"),
24
+ ("4", "External Product & Service Assurance"),
25
+ ("5", "Pilot"),
26
+ ]
27
+
28
+ for milestone in [
29
+ "SCRUB_REPOSITORY_CONVERGED",
30
+ "SCRUB_HF_APPLICATION_COMPLETE",
31
+ "SCRUB_PRIVATE_SERVICE_CANDIDATE",
32
+ ]:
33
+ assert milestone in roadmap
34
+
35
+ assert "Phase 9 — Final local desktop/offline installer path" not in roadmap
36
+ assert "active draft PR #85" not in roadmap
37
+ assert "The broader direction is a local-first" not in roadmap
38
+
39
+
40
+ def test_workpackages_contains_one_current_queue_not_historical_overrides() -> None:
41
+ workpackages = read("WORKPACKAGES.md")
42
+
43
+ assert "# SolidPrivacy Scrub — Current Workpackages" in workpackages
44
+ assert "Repository Convergence" in workpackages
45
+ assert "SCRUB-WP_REPOSITORY_CONVERGENCE_BOOTSTRAP" in workpackages
46
+ assert "Current execution status override" not in workpackages
47
+ assert "WP-CONVERGENCE-02..N" in workpackages
48
+ assert "DERIVED, NOT PRE-INVENTED" in workpackages
49
+ assert "SCRUB_REPOSITORY_CONVERGED" in workpackages
50
+
51
+
52
+ def test_project_prompt_preserves_start_governance_and_safety_contracts() -> None:
53
+ prompt = read("PROJECT_PROMPT.md")
54
+
55
+ ordered = [
56
+ "1. `PROJECT_PROMPT.md`",
57
+ "2. `ROADMAP.md`",
58
+ "3. `WORKPACKAGES.md`",
59
+ "4. `CHANGELOG.md`",
60
+ ]
61
+ positions = [prompt.index(marker) for marker in ordered]
62
+ assert positions == sorted(positions)
63
+
64
+ for marker in [
65
+ "implementation_operations",
66
+ "governance_release_assurance",
67
+ "PASS`, `FAIL` or `INDETERMINATE",
68
+ "Scrub Private",
69
+ "Repository Convergence",
70
+ "human review",
71
+ "false negatives",
72
+ "GitHub→Hugging Face",
73
+ "handover/workpackages/YYYYMMDD_HHMM_<workpackage_slug>.md",
74
+ ]:
75
+ assert marker in prompt
76
+
77
+ assert "Immediate line: v12 Review UX" not in prompt
78
+ assert "broader direction is a local-first" not in prompt
79
+
80
+
81
+ def test_short_prompt_routes_workers_to_the_same_current_direction() -> None:
82
+ short = read("PROJECT_PROMPT_SHORT.md")
83
+
84
+ for marker in [
85
+ "Repository Convergence",
86
+ "Scrub Private Application",
87
+ "Private Service",
88
+ "External Product & Service Assurance",
89
+ "Pilot",
90
+ "governance_release_assurance",
91
+ "synthetic/approved-test application-validation",
92
+ ]:
93
+ assert marker in short
94
+
95
+ assert "Immediate line: v12 Review UX" not in short
96
+
97
+
98
+ def test_agents_routes_claims_and_execution_to_convergence() -> None:
99
+ agents = read("AGENTS.md")
100
+
101
+ for marker in [
102
+ "Repository Convergence",
103
+ "SCRUB_REPOSITORY_CONVERGED",
104
+ "implementation_operations",
105
+ "governance_release_assurance",
106
+ "workpackage_claims/",
107
+ "Do not refactor for aesthetics",
108
+ ]:
109
+ assert marker in agents
110
+
111
+ assert "local-first" not in agents.lower()
112
+
113
+
114
+ def test_temporary_ledger_is_explicitly_non_authoritative_and_capability_level() -> None:
115
+ ledger = read("REPOSITORY_CONVERGENCE_DEBT_LEDGER.md")
116
+
117
+ assert "TEMPORARY EXECUTION ARTIFACT — NON-AUTHORITATIVE AFTER CONVERGENCE" in ledger
118
+ assert "54c73e0ebf5a3a3ed7039a50596fb57694add3cd" in ledger
119
+
120
+ for classification in [
121
+ "CANONICAL",
122
+ "RECONCILE",
123
+ "RETIRE",
124
+ "VARIANT-SPECIFIC",
125
+ ]:
126
+ assert classification in ledger
127
+
128
+ for capability in [
129
+ "Persistent replacement memory",
130
+ "Azure AI Language document recognition",
131
+ "OpenAI/Azure OpenAI synthesis operator",
132
+ "Legacy startup patch script invocation in Dockerfile",
133
+ "Recognizer-backed recall benchmark",
134
+ ]:
135
+ assert capability in ledger
136
+
137
+
138
+ def test_decision_log_keeps_current_binding_decisions_and_adds_d044() -> None:
139
+ decisions = read("DECISION_LOG.md")
140
+
141
+ for marker in [
142
+ "D044 — 2026-09-04 — Repository Convergence",
143
+ "D043",
144
+ "D042",
145
+ "D041",
146
+ "D040",
147
+ "D039",
148
+ "D037",
149
+ "D036",
150
+ "D034",
151
+ "D033",
152
+ "D031",
153
+ "D030",
154
+ "D021",
155
+ "54c73e0ebf5a3a3ed7039a50596fb57694add3cd",
156
+ ]:
157
+ assert marker in decisions
158
+
159
+ assert "main` becomes the active Scrub Private development line" in decisions
160
+ assert "human review" in decisions.lower()
161
+ assert "fail" in decisions.lower() and "closed" in decisions.lower()
162
+
163
+
164
+ def test_risk_register_keeps_critical_product_risks_and_adds_current_truth_risks() -> None:
165
+ risks = read("RISK_REGISTER.md")
166
+
167
+ assert "R1 — False negatives / missed sensitive data" in risks
168
+ assert "R2 — Scrub Key leakage, mismatch or misuse" in risks
169
+ assert "R10 — Zorg under-detection and clinical over-masking" in risks
170
+ assert "R11 — Repository/source-of-truth drift" in risks
171
+ assert "R12 — Legacy runtime mutation / hidden startup authority" in risks
172
+
173
+ for heading in [
174
+ "R1 — False negatives / missed sensitive data",
175
+ "R2 — Scrub Key leakage, mismatch or misuse",
176
+ "R10 — Zorg under-detection and clinical over-masking",
177
+ ]:
178
+ start = risks.index(heading)
179
+ section = risks[start : start + 700]
180
+ assert "Impact: `critical`" in section, heading
181
+
182
+ assert "R5 — Scrub Private content-retention / external-egress trust boundary" in risks
183
+ r5 = risks[risks.index("R5 — Scrub Private") : risks.index("R6 — Review UX")]
184
+ assert "Impact: `critical`" in r5
185
+ assert "replacement_memory.py" in r5
186
+ assert "Azure AI Language" in r5
187
+
188
+
189
+ def test_changelog_preserves_preconvergence_history_without_using_it_as_current_queue() -> None:
190
+ changelog = read("CHANGELOG.md")
191
+ archived_path = ROOT / "history" / "CHANGELOG_PRE_CONVERGENCE_20260904.md"
192
+
193
+ assert archived_path.exists()
194
+ archived = archived_path.read_text(encoding="utf-8")
195
+
196
+ assert "SCRUB-WP_REPOSITORY_CONVERGENCE_BOOTSTRAP" in changelog
197
+ assert "history/CHANGELOG_PRE_CONVERGENCE_20260904.md" in changelog
198
+ assert "54c73e0ebf5a3a3ed7039a50596fb57694add3cd" in changelog
199
+ assert "SCRUB-WP_REVIEW_MARKER_COMPACTION_LIVE_REGRESSION_REPAIR" in archived
200
+ assert "historical provenance only" in changelog
201
+
202
+
203
+ def test_bootstrap_does_not_redefine_hugging_face_as_production_assurance() -> None:
204
+ combined = "\n".join(
205
+ [
206
+ read("ROADMAP.md"),
207
+ read("PROJECT_PROMPT.md"),
208
+ read("WORKPACKAGES.md"),
209
+ ]
210
+ ).lower()
211
+
212
+ assert "hugging face" in combined
213
+ assert "application-validation" in combined or "application validation" in combined
214
+ assert "not the final confidential-production trust environment" in combined
215
+ assert "zero retention" not in combined
216
+
217
+
218
+ def test_bootstrap_does_not_authorize_source_cloning_or_a_new_evidence_framework() -> None:
219
+ roadmap = read("ROADMAP.md")
220
+ ledger = read("REPOSITORY_CONVERGENCE_DEBT_LEDGER.md")
221
+
222
+ assert "no source-tree clone" in roadmap
223
+ assert "No separate Evidence Framework" in roadmap
224
+ assert "/app_v2" not in roadmap
225
+ assert "/scrub-new" not in roadmap
226
+ # One stable semantic prohibition is enough; do not make duplicate prose
227
+ # across documents a release requirement.
228
+ assert "no new framework" in ledger.lower()
workpackage_claims/SCRUB-WP_REPOSITORY_CONVERGENCE_BOOTSTRAP.md ADDED
@@ -0,0 +1,83 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Workpackage claim — SCRUB-WP_REPOSITORY_CONVERGENCE_BOOTSTRAP
2
+
3
+ Status: implementation_remediated; final exact-head CI and fresh independent assurance required
4
+ Role: `implementation_operations`
5
+ Issue: #113
6
+ PR: #114
7
+ Branch: `wp/repository-convergence-bootstrap`
8
+ Starting main: `54c73e0ebf5a3a3ed7039a50596fb57694add3cd`
9
+ Started: 2026-09-04 Europe/Amsterdam
10
+
11
+ ## Objective
12
+
13
+ Preserve the exact pre-convergence baseline, reconstruct reachable current truth, and reset canonical strategy/execution documentation to the approved Repository Convergence → Scrub Private direction without changing runtime product semantics.
14
+
15
+ ## Scope
16
+
17
+ Documentation, governance, issue/current-truth inventory, audit-ledger and governance/contract-test convergence only.
18
+
19
+ No recognizer, review authority, export, Scrub Key, reinsert, document-processing, dependency, runtime or Hugging Face product behavior change is authorized in this package.
20
+
21
+ ## Independent assurance history
22
+
23
+ Fresh blind assurance on prior exact head:
24
+
25
+ ```text
26
+ ea203abb04f008a7e583387242a6f4917c72e591
27
+ ```
28
+
29
+ returned:
30
+
31
+ ```text
32
+ FAIL
33
+ ```
34
+
35
+ Release-blocking findings:
36
+
37
+ 1. exact-head full regression was red: `8 failed, 1256 passed`;
38
+ 2. two new convergence tests encoded brittle global-string/prose placement rather than stable semantic contracts;
39
+ 3. six legacy tests still required historical Premium/fidelity/reinsert status to remain in current ROADMAP/WORKPACKAGES/CHANGELOG locations.
40
+
41
+ The reviewer found the underlying roadmap/scope/product semantics materially sound and explicitly required a narrow test/governance remediation rather than runtime changes.
42
+
43
+ ## Remediation
44
+
45
+ Implementation changed only governance/contract tests:
46
+
47
+ - roadmap macro-stage validation now parses formal `## Stage N — ...` headings and compares the ordered five-stage set;
48
+ - Evidence Framework prohibition is bound to one stable semantic prohibition rather than duplicated prose;
49
+ - historical fidelity completion evidence now reads from the exact archived pre-convergence changelog/handovers, while current DOCX scope is bound to D030;
50
+ - Premium staged-workspace/core-flow tests now bind current semantics to D041/D043 and current shared-surface sequencing instead of requiring superseded package queues to stay current;
51
+ - reinsert completion evidence now reads from archived history/handovers, while current document-first/fail-closed behavior is bound to D031/D037.
52
+
53
+ No runtime/product file was changed by this remediation.
54
+
55
+ ## Pre-final validation evidence
56
+
57
+ After the test/governance remediation, branch head:
58
+
59
+ ```text
60
+ dcdfb2c84bbaafe0beca79191bee509b9607b461
61
+ ```
62
+
63
+ was tested by GitHub Actions `Tests` run:
64
+
65
+ ```text
66
+ 33922952965
67
+ ```
68
+
69
+ Raw job output:
70
+
71
+ ```text
72
+ 1264 passed in 12.34s
73
+ ```
74
+
75
+ That run proves the remediation itself is functionally green. Administrative claim/handover/changelog updates follow, so a **new final exact-head full regression is still mandatory** before the release candidate is frozen.
76
+
77
+ ## Required final validation
78
+
79
+ - compare final branch against exact starting main and confirm documentation/governance/tests/administration-only scope;
80
+ - full GitHub Actions regression on the final exact PR head;
81
+ - freeze that exact head;
82
+ - fresh independent `governance_release_assurance` on the new SHA before merge;
83
+ - no prior FAIL/PASS verdict transfers to a changed head.