Spaces:
Sleeping
Sleeping
Merge pull request #114 from solidprivacy-nl/wp/repository-convergence-bootstrap
Browse filesSCRUB-WP_REPOSITORY_CONVERGENCE_BOOTSTRAP — reset current truth before further development
- AGENTS.md +105 -141
- CHANGELOG.md +0 -0
- DECISION_LOG.md +131 -606
- PROJECT_PROMPT.md +201 -170
- PROJECT_PROMPT_SHORT.md +43 -45
- REPOSITORY_CONVERGENCE_DEBT_LEDGER.md +89 -0
- RISK_REGISTER.md +253 -124
- ROADMAP.md +226 -538
- WORKPACKAGES.md +0 -0
- handover/workpackages/20260904_2315_repository_convergence_bootstrap.md +278 -0
- history/CHANGELOG_PRE_CONVERGENCE_20260904.md +2256 -0
- tests/test_mvp_document_fidelity_pr_final_contracts.py +18 -17
- tests/test_premium_core_flow_ui_realignment_plan.py +24 -14
- tests/test_premium_staged_workspace_decision.py +29 -20
- tests/test_reinsert_auto_flow_app_verify_closeout.py +15 -8
- tests/test_repository_convergence_bootstrap_contracts.py +228 -0
- workpackage_claims/SCRUB-WP_REPOSITORY_CONVERGENCE_BOOTSTRAP.md +83 -0
AGENTS.md
CHANGED
|
@@ -10,7 +10,7 @@ Work only in:
|
|
| 10 |
solidprivacy-nl/scrub
|
| 11 |
```
|
| 12 |
|
| 13 |
-
|
| 14 |
|
| 15 |
## Required start sequence
|
| 16 |
|
|
@@ -21,196 +21,160 @@ Before starting any task, read in order:
|
|
| 21 |
3. `WORKPACKAGES.md`
|
| 22 |
4. `CHANGELOG.md`
|
| 23 |
|
| 24 |
-
|
| 25 |
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
- `STATUS_MONITORING_RUNBOOK.md`
|
| 29 |
-
- `RELEASE_NOTES.md`
|
| 30 |
-
- feature specs such as `SCRUB_KEY_SPEC.md` and `PDF_TEXT_REINSERT_UI_PLAN.md`
|
| 31 |
|
| 32 |
-
|
| 33 |
|
| 34 |
-
|
| 35 |
|
| 36 |
-
|
| 37 |
|
| 38 |
-
|
| 39 |
-
- status `completed`: stop and report that the package is already done;
|
| 40 |
-
- status `blocked`: stop and read the blocking reason;
|
| 41 |
-
- status `abandoned`: continue only with coordinator approval.
|
| 42 |
|
| 43 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
|
| 45 |
-
|
| 46 |
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
## Product direction
|
| 50 |
-
|
| 51 |
-
The product direction is:
|
| 52 |
|
| 53 |
```text
|
| 54 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
```
|
| 56 |
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
1. false negatives / missed sensitive data;
|
| 60 |
-
2. Scrub Key leakage or accidental sharing;
|
| 61 |
-
3. hidden document content and metadata leakage;
|
| 62 |
-
4. cloud-demo trust gap versus local-first promise;
|
| 63 |
-
5. placeholder corruption during AI roundtrip;
|
| 64 |
-
6. review UX limitations;
|
| 65 |
-
7. PDF-scope misunderstandings.
|
| 66 |
-
|
| 67 |
-
## Safety rules
|
| 68 |
-
|
| 69 |
-
Do not weaken privacy or review controls.
|
| 70 |
-
|
| 71 |
-
Do not silently change export semantics.
|
| 72 |
-
|
| 73 |
-
Do not introduce cloud document processing unless explicitly approved.
|
| 74 |
|
| 75 |
-
|
| 76 |
|
| 77 |
-
|
| 78 |
|
| 79 |
-
|
| 80 |
|
| 81 |
-
|
| 82 |
|
| 83 |
-
##
|
| 84 |
|
| 85 |
-
|
| 86 |
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
|
|
|
| 92 |
|
| 93 |
-
|
| 94 |
|
| 95 |
-
|
| 96 |
|
| 97 |
-
|
| 98 |
|
| 99 |
-
|
|
|
|
|
|
|
|
|
|
| 100 |
|
| 101 |
-
|
| 102 |
-
- helper modules in separate files;
|
| 103 |
-
- tests that do not touch the same UI flow;
|
| 104 |
-
- risk reviews;
|
| 105 |
-
- architecture plans;
|
| 106 |
-
- non-UI documentation.
|
| 107 |
|
| 108 |
-
|
| 109 |
|
| 110 |
-
|
| 111 |
-
- `fix_streamlit_nested_expanders.py`
|
| 112 |
-
- `fix_streamlit_pdf_text_reinsert.py`
|
| 113 |
-
- review table UI flow
|
| 114 |
-
- export/download UI flow
|
| 115 |
-
- shared workflow state
|
| 116 |
-
- Docker/runtime startup patch order
|
| 117 |
|
| 118 |
-
##
|
| 119 |
|
| 120 |
-
|
| 121 |
|
| 122 |
-
|
| 123 |
|
| 124 |
-
|
| 125 |
|
| 126 |
-
|
| 127 |
|
| 128 |
-
|
| 129 |
|
| 130 |
-
|
| 131 |
|
| 132 |
-
|
| 133 |
|
| 134 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 135 |
|
| 136 |
-
UI
|
| 137 |
|
| 138 |
-
##
|
| 139 |
|
| 140 |
-
|
| 141 |
|
| 142 |
-
|
| 143 |
-
|
| 144 |
-
|
| 145 |
-
|
| 146 |
-
|
|
|
|
| 147 |
|
| 148 |
-
Do not
|
| 149 |
|
| 150 |
-
##
|
| 151 |
|
| 152 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 153 |
|
| 154 |
-
|
| 155 |
-
handover/workpackages/
|
| 156 |
-
```
|
| 157 |
|
| 158 |
-
|
| 159 |
|
| 160 |
```text
|
| 161 |
handover/workpackages/YYYYMMDD_HHMM_<workpackage_slug>.md
|
| 162 |
```
|
| 163 |
|
| 164 |
-
|
| 165 |
-
|
| 166 |
-
- repository
|
| 167 |
-
- workpackage
|
| 168 |
-
- status;
|
| 169 |
-
-
|
| 170 |
-
- files changed;
|
| 171 |
-
- tests
|
| 172 |
-
- validation
|
| 173 |
-
- GitHub Actions
|
| 174 |
-
- Hugging Face sync
|
| 175 |
-
- app verification
|
| 176 |
-
- remaining risks;
|
| 177 |
-
- next recommended step.
|
| 178 |
-
|
| 179 |
-
For Codex or other parallel worker tasks, do not paste the full handover into the coordinator chat when the handover has been committed to the repository.
|
| 180 |
-
|
| 181 |
-
Instead, the final worker response should provide only:
|
| 182 |
-
|
| 183 |
-
- workpackage title;
|
| 184 |
-
- status;
|
| 185 |
-
- commit SHA or PR link;
|
| 186 |
-
- handover file path;
|
| 187 |
-
- short summary of files added/changed;
|
| 188 |
-
- tests/checks run;
|
| 189 |
-
- remaining risks;
|
| 190 |
- next recommended step.
|
| 191 |
|
| 192 |
-
|
| 193 |
|
| 194 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 195 |
|
| 196 |
-
|
| 197 |
-
- GitHub access failed;
|
| 198 |
-
- there is a conflict or permission issue;
|
| 199 |
-
- the coordinator explicitly asks for the full text.
|
| 200 |
-
|
| 201 |
-
## Conflict rule
|
| 202 |
-
|
| 203 |
-
Before editing shared documentation files, fetch the latest version.
|
| 204 |
-
|
| 205 |
-
If there is a SHA conflict, update conflict or missing permission, stop and report instead of overwriting another worker’s changes.
|
| 206 |
-
|
| 207 |
-
## Final response requirements
|
| 208 |
-
|
| 209 |
-
In the final response, summarize:
|
| 210 |
-
|
| 211 |
-
- what was changed;
|
| 212 |
-
- which files were touched;
|
| 213 |
-
- what tests/checks were run;
|
| 214 |
-
- what was intentionally not changed;
|
| 215 |
-
- remaining risks;
|
| 216 |
-
- next recommended step.
|
|
|
|
| 10 |
solidprivacy-nl/scrub
|
| 11 |
```
|
| 12 |
|
| 13 |
+
Use `solidprivacy-nl/solidprivacy` only when an explicitly shared SolidPrivacy workflow/component is required by the current workpackage. Do not modify unrelated repositories.
|
| 14 |
|
| 15 |
## Required start sequence
|
| 16 |
|
|
|
|
| 21 |
3. `WORKPACKAGES.md`
|
| 22 |
4. `CHANGELOG.md`
|
| 23 |
|
| 24 |
+
For consequential work also read:
|
| 25 |
|
| 26 |
+
1. `control/PROJECT_GOVERNANCE_BOOTSTRAP.md`
|
| 27 |
+
2. `control/SCRUB_RELEASE_ASSURANCE_CONTRACT_V1.md`
|
|
|
|
|
|
|
|
|
|
| 28 |
|
| 29 |
+
Read `RISK_REGISTER.md`, `DECISION_LOG.md` and relevant product contracts for the affected surface.
|
| 30 |
|
| 31 |
+
GitHub is the source of truth. If canonical docs conflict with current source/runtime evidence, surface and reconcile the conflict rather than silently selecting one.
|
| 32 |
|
| 33 |
+
## Engineering doctrine
|
| 34 |
|
| 35 |
+
Follow the SolidPrivacy Execution & Engineering Constitution:
|
|
|
|
|
|
|
|
|
|
| 36 |
|
| 37 |
+
- business outcome first;
|
| 38 |
+
- smallest complete solution;
|
| 39 |
+
- solid but simple;
|
| 40 |
+
- no overengineering;
|
| 41 |
+
- first-principles reasoning;
|
| 42 |
+
- proven/native solutions before custom machinery;
|
| 43 |
+
- one source of truth;
|
| 44 |
+
- remove obsolete/conflicting active paths;
|
| 45 |
+
- verify behavior and security;
|
| 46 |
+
- Done requires implementation, verification, cleanup and documentation alignment.
|
| 47 |
|
| 48 |
+
## Current strategic direction
|
| 49 |
|
| 50 |
+
The active line is:
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
|
| 52 |
```text
|
| 53 |
+
Repository Convergence
|
| 54 |
+
→ Scrub Private Application
|
| 55 |
+
→ Private Service
|
| 56 |
+
→ External Product & Service Assurance
|
| 57 |
+
→ Pilot
|
| 58 |
```
|
| 59 |
|
| 60 |
+
Normal new feature development is paused until `SCRUB_REPOSITORY_CONVERGED`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 61 |
|
| 62 |
+
After convergence, `main` becomes the active Scrub Private line.
|
| 63 |
|
| 64 |
+
Hugging Face is a synthetic/approved-test application-validation environment, not the final confidential-production trust environment.
|
| 65 |
|
| 66 |
+
Local/offline functionality is preserved/deferred. Do not restart old installer or historical Premium work merely because old files/issues mention it.
|
| 67 |
|
| 68 |
+
Do not build a new Evidence Framework; reuse and reconcile the existing validation systems.
|
| 69 |
|
| 70 |
+
## Worker claim rule
|
| 71 |
|
| 72 |
+
Before implementation, claim one specific workpackage in `workpackage_claims/` with:
|
| 73 |
|
| 74 |
+
- workpackage title;
|
| 75 |
+
- role;
|
| 76 |
+
- issue/branch where applicable;
|
| 77 |
+
- exact starting base;
|
| 78 |
+
- scope and exclusions;
|
| 79 |
+
- required validation.
|
| 80 |
|
| 81 |
+
Do not claim multiple overlapping shared-surface packages in one worker.
|
| 82 |
|
| 83 |
+
## Implementation and assurance separation
|
| 84 |
|
| 85 |
+
Consequential work uses two separated roles:
|
| 86 |
|
| 87 |
+
```text
|
| 88 |
+
implementation_operations
|
| 89 |
+
governance_release_assurance
|
| 90 |
+
```
|
| 91 |
|
| 92 |
+
Implementation may create a candidate but may not certify it.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 93 |
|
| 94 |
+
Assurance must use a fresh independent worker/session, reconstruct exact source/evidence, and record `PASS`, `FAIL` or `INDETERMINATE`. Before its initial verdict it must not rely on implementation handovers, self-assessments or conclusions. Assurance may not silently repair what it reviews.
|
| 95 |
|
| 96 |
+
A changed/repaired exact head requires a fresh assurance pass.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 97 |
|
| 98 |
+
## Workpackage sizing
|
| 99 |
|
| 100 |
+
Use small coherent root-cause packages.
|
| 101 |
|
| 102 |
+
Do not create one giant convergence PR. Do not fragment trivial administrative edits into meaningless ritual packages.
|
| 103 |
|
| 104 |
+
**Do not refactor for aesthetics.**
|
| 105 |
|
| 106 |
+
During Repository Convergence, a cleanup/refactor requires a concrete current reason such as duplicate/contradictory/dead behavior, privacy/security risk, obsolete compatibility, meaningful maintenance burden, runtime instability or evidence-authority ambiguity.
|
| 107 |
|
| 108 |
+
“A cleaner architecture” alone is not sufficient.
|
| 109 |
|
| 110 |
+
## Shared-surface sequencing
|
| 111 |
|
| 112 |
+
Avoid uncoordinated parallel edits to:
|
| 113 |
|
| 114 |
+
- `presidio_streamlit.py`;
|
| 115 |
+
- Streamlit/Docker startup behavior;
|
| 116 |
+
- review table/direct-correction flow;
|
| 117 |
+
- processing-generation state;
|
| 118 |
+
- export/download flow;
|
| 119 |
+
- Scrub Key/reinsert paths.
|
| 120 |
|
| 121 |
+
Keep shared runtime/UI integration sequential unless explicit coordination proves independence.
|
| 122 |
|
| 123 |
+
## Testing and deployment
|
| 124 |
|
| 125 |
+
Where tools allow, workers self-check:
|
| 126 |
|
| 127 |
+
1. relevant focused tests;
|
| 128 |
+
2. full GitHub Actions on the exact candidate when required;
|
| 129 |
+
3. failed-job logs;
|
| 130 |
+
4. exact-main Actions after authorized merge;
|
| 131 |
+
5. GitHub→Hugging Face sync when applicable;
|
| 132 |
+
6. deployed/live app verification when visible/runtime behavior changed.
|
| 133 |
|
| 134 |
+
Do not treat code inspection alone as proof.
|
| 135 |
|
| 136 |
+
## Documentation ownership
|
| 137 |
|
| 138 |
+
- `ROADMAP.md` = strategic stages, not implementation history.
|
| 139 |
+
- `WORKPACKAGES.md` = one current executable queue.
|
| 140 |
+
- `CHANGELOG.md` = current implementation history; exact pre-convergence history is preserved in `history/CHANGELOG_PRE_CONVERGENCE_20260904.md` and Git at the baseline SHA.
|
| 141 |
+
- `DECISION_LOG.md` = accepted current binding decisions.
|
| 142 |
+
- `RISK_REGISTER.md` = current risks.
|
| 143 |
+
- temporary audit/debt ledgers = execution evidence only; they must not become competing permanent status authorities.
|
| 144 |
|
| 145 |
+
## Handover
|
|
|
|
|
|
|
| 146 |
|
| 147 |
+
Every worker writes:
|
| 148 |
|
| 149 |
```text
|
| 150 |
handover/workpackages/YYYYMMDD_HHMM_<workpackage_slug>.md
|
| 151 |
```
|
| 152 |
|
| 153 |
+
Include:
|
| 154 |
+
|
| 155 |
+
- repository;
|
| 156 |
+
- workpackage;
|
| 157 |
+
- role/status;
|
| 158 |
+
- exact candidate/baseline identity where relevant;
|
| 159 |
+
- files added/changed;
|
| 160 |
+
- tests;
|
| 161 |
+
- validation;
|
| 162 |
+
- GitHub Actions;
|
| 163 |
+
- Hugging Face sync;
|
| 164 |
+
- app verification;
|
| 165 |
+
- remaining risks/blockers;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 166 |
- next recommended step.
|
| 167 |
|
| 168 |
+
## Safety
|
| 169 |
|
| 170 |
+
- Do not weaken human review/privacy controls.
|
| 171 |
+
- Do not silently change export or Scrub Key semantics.
|
| 172 |
+
- Treat false negatives as product-critical.
|
| 173 |
+
- Treat Scrub Keys as sensitive re-identification material.
|
| 174 |
+
- Use synthetic data only.
|
| 175 |
+
- Do not store secrets/tokens/real personal data.
|
| 176 |
+
- Preserve legal and clinical meaning.
|
| 177 |
+
- Do not introduce external/cloud document processing unless explicitly approved for the relevant product line.
|
| 178 |
+
- Do not claim perfect anonymisation/recall or production readiness from synthetic tests.
|
| 179 |
|
| 180 |
+
For Scrub Private, the target is no intentional persistent customer document content, no content-bearing ordinary logs, no document-content backup and no third-party document-processing egress. Minimal non-document control-plane metadata may persist only when justified by a real service requirement.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
CHANGELOG.md
CHANGED
|
The diff for this file is too large to render.
See raw diff
|
|
|
DECISION_LOG.md
CHANGED
|
@@ -1,739 +1,264 @@
|
|
| 1 |
-
#
|
| 2 |
|
| 3 |
-
|
| 4 |
|
| 5 |
-
|
| 6 |
|
| 7 |
```text
|
| 8 |
-
|
| 9 |
```
|
| 10 |
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
```text
|
| 14 |
-
Toevoegen page → Controleren page → Downloaden page
|
| 15 |
-
```
|
| 16 |
-
|
| 17 |
-
The three core stages are states of the same document, not three separate routed screens. Also reject a conventional long Streamlit form rebuilt as a tree of independent or nested expanders.
|
| 18 |
-
|
| 19 |
-
First-principles reason:
|
| 20 |
-
- the scarce resource is user attention during a privacy-sensitive review task;
|
| 21 |
-
- Scrub is iterative: review can legitimately send the user back to input/profile correction;
|
| 22 |
-
- the interface must preserve confidence about source identity, completed work and whether downstream state is still current;
|
| 23 |
-
- page routing maximizes isolation but removes spatial workflow context and adds state-restoration/routing complexity;
|
| 24 |
-
- one staged workspace preserves the cognitive benefit of one task at a time while representing document state directly;
|
| 25 |
-
- the model maps naturally to the merged source/processed/review/export generation lineage in `premium_core_flow_state.py`.
|
| 26 |
-
|
| 27 |
-
Binding interaction consequences:
|
| 28 |
-
- one dominant primary action per active stage;
|
| 29 |
-
- no permanent configuration sidebar in Standard;
|
| 30 |
-
- completed stage summaries contain orientation/trust information only, not mini-forms;
|
| 31 |
-
- future stages show status rather than disabled control forests;
|
| 32 |
-
- successful processing opens Controleren automatically;
|
| 33 |
-
- explicit review completion opens Downloaden automatically;
|
| 34 |
-
- earlier processing-affecting edits invalidate downstream review/export state before those results can be presented as current;
|
| 35 |
-
- no nested core-flow accordion hierarchy in Standard;
|
| 36 |
-
- stage surfaces should read as application panels even when Streamlit primitives are used internally.
|
| 37 |
-
|
| 38 |
-
Current workpackage consequence:
|
| 39 |
-
- insert `SCRUB-WP_PREMIUM_STAGED_WORKSPACE_DECISION_FREEZE` before further production integration in `SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION`;
|
| 40 |
-
- retain PR #85's compatible pure helper/test work;
|
| 41 |
-
- require PR #85 to incorporate the staged-workspace semantics before touching/merging production `presidio_streamlit.py` integration;
|
| 42 |
-
- continue sequentially with Input, Review, Export, Expert parity and live app closeout packages.
|
| 43 |
-
|
| 44 |
-
Safety boundary:
|
| 45 |
-
- this decision changes presentation architecture, not anonymization semantics;
|
| 46 |
-
- recognizers, thresholds, review authority, direct masking, export bytes/names/MIME, Scrub Key, reinsert, audit and human-review requirements remain unchanged;
|
| 47 |
-
- Standard is lower cognitive load, not lower safety.
|
| 48 |
-
|
| 49 |
-
Evidence:
|
| 50 |
-
- coordinator/user approval on 2026-08-08 Europe/Amsterdam after explicit first-principles comparison;
|
| 51 |
-
- `PREMIUM_STAGED_WORKSPACE_DECISION.md`;
|
| 52 |
-
- `PREMIUM_CORE_FLOW_UI_REALIGNMENT_PLAN.md`;
|
| 53 |
-
- merged Premium UI contract and state model.
|
| 54 |
|
| 55 |
---
|
| 56 |
|
| 57 |
-
## 2026-
|
| 58 |
|
| 59 |
-
Status: accepted
|
| 60 |
|
| 61 |
Decision:
|
| 62 |
|
| 63 |
```text
|
| 64 |
-
|
| 65 |
-
```
|
| 66 |
-
|
| 67 |
-
Scrub-specific strengthening:
|
| 68 |
|
| 69 |
-
|
| 70 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
```
|
| 72 |
|
| 73 |
-
|
| 74 |
-
- Scrub processes privacy-sensitive professional documents and creates re-identification-sensitive Scrub Keys;
|
| 75 |
-
- builder self-certification creates avoidable confirmation bias and weakens release confidence;
|
| 76 |
-
- the Weekly ETF donor architecture already establishes a mature coordinator/implementation/assurance split;
|
| 77 |
-
- the user's explicit requirement is stronger than ordinary review and requires conclusion isolation before the initial assurance decision.
|
| 78 |
-
|
| 79 |
-
Operating consequences:
|
| 80 |
-
- consequential implementation and verification are separate workpackages and separate workers/sessions;
|
| 81 |
-
- the user continues to give one instruction and receives one consolidated status;
|
| 82 |
-
- a failed or indeterminate candidate returns to implementation and requires a fresh assurance pass;
|
| 83 |
-
- implementation statuses are limited to `IMPLEMENTATION_IN_PROGRESS`, `IMPLEMENTATION_BLOCKED` and `RELEASE_CANDIDATE_READY`;
|
| 84 |
-
- assurance decisions are `PASS`, `FAIL` or `INDETERMINATE`;
|
| 85 |
-
- merge/deployment execution and independently confirmed outcome remain distinct statuses.
|
| 86 |
-
|
| 87 |
-
Initial maturity:
|
| 88 |
-
- `LEVEL_1_CHECKLIST`;
|
| 89 |
-
- target `LEVEL_2_MACHINE_EVIDENCE` through a later structured assurance-record package;
|
| 90 |
-
- no claim that documentation alone provides hard CI or post-action enforcement.
|
| 91 |
|
| 92 |
-
|
| 93 |
-
- canonical
|
| 94 |
-
-
|
| 95 |
-
-
|
|
|
|
| 96 |
|
| 97 |
-
|
| 98 |
|
| 99 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 100 |
|
| 101 |
-
|
| 102 |
|
| 103 |
```text
|
| 104 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 105 |
```
|
| 106 |
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
- live app evidence confirms that earlier Basic/Expert and decluttering work improved individual sections but left the whole product as a long form;
|
| 110 |
-
- premium and enterprise credibility require a coherent task hierarchy, not merely fewer open expanders;
|
| 111 |
-
- input alternatives, profile settings, review machinery, normal downloads, restore material and audit evidence serve different user goals and should not have equal visual weight;
|
| 112 |
-
- one active stage at a time reduces cognitive load and makes the current task explicit.
|
| 113 |
-
|
| 114 |
-
Presentation boundary:
|
| 115 |
-
|
| 116 |
-
- recommended global labels are `Standaard` and `Expert`, subject to contract freeze;
|
| 117 |
-
- Standard has no permanent configuration sidebar;
|
| 118 |
-
- Expert preserves full inspection, tuning, audit and troubleshooting;
|
| 119 |
-
- switching presentation changes visibility and grouping only and must preserve input, replacement decisions and session state.
|
| 120 |
-
|
| 121 |
-
Safety boundary:
|
| 122 |
-
|
| 123 |
-
- human review remains mandatory;
|
| 124 |
-
- the review table remains source of truth and fallback;
|
| 125 |
-
- no silent profile or recognizer changes;
|
| 126 |
-
- no export-byte, filename, MIME, Scrub Key, reinsert or audit semantic changes;
|
| 127 |
-
- no cloud document processing, telemetry or browser persistence;
|
| 128 |
-
- an explicit processing action requires a pure stale-state and transition model before UI integration.
|
| 129 |
-
|
| 130 |
-
Approved sequence:
|
| 131 |
-
|
| 132 |
-
1. `SCRUB-WP_PROCESSED_TEXT_SELECTION_CROSS_FLOW_REGRESSION`
|
| 133 |
-
2. `SCRUB-WP_PREMIUM_CORE_FLOW_UI_CONTRACT`
|
| 134 |
-
3. `SCRUB-WP_PREMIUM_CORE_FLOW_STATE_MODEL`
|
| 135 |
-
4. `SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION`
|
| 136 |
-
5. `SCRUB-WP_PREMIUM_INPUT_STAGE_SIMPLIFICATION`
|
| 137 |
-
6. `SCRUB-WP_PREMIUM_REVIEW_STAGE_SIMPLIFICATION`
|
| 138 |
-
7. `SCRUB-WP_PREMIUM_EXPORT_STAGE_SIMPLIFICATION`
|
| 139 |
-
8. `SCRUB-WP_PREMIUM_EXPERT_PARITY_REGRESSION`
|
| 140 |
-
9. `SCRUB-WP_PREMIUM_CORE_FLOW_APP_VERIFY_CLOSEOUT`
|
| 141 |
-
|
| 142 |
-
Evidence:
|
| 143 |
-
|
| 144 |
-
- coordinator/user feedback at 2026-08-05 10:49 Europe/Amsterdam;
|
| 145 |
-
- `BASIC_EXPERT_REVIEW_MODE_PLAN.md`;
|
| 146 |
-
- `MVP_UI_CLEANUP_AND_EXPORT_REDESIGN_PLAN.md`;
|
| 147 |
-
- `PREMIUM_CORE_FLOW_UI_REALIGNMENT_PLAN.md`.
|
| 148 |
-
|
| 149 |
-
## 2026-08-04 — D040 — Use a two-stage server-authoritative protocol for direct masking from processed text
|
| 150 |
-
|
| 151 |
-
Status: accepted product, UX and implementation-sequence decision
|
| 152 |
-
|
| 153 |
-
Decision:
|
| 154 |
-
|
| 155 |
-
```text
|
| 156 |
-
Allow a user to select an unmasked value in Verwerkte tekst, invoke a right-click or visible/keyboard masking action, inspect the server-validated exact-occurrence impact, choose a broad type and create one normal manual replacement row. Version one masks all safe exact occurrences only.
|
| 157 |
-
```
|
| 158 |
-
|
| 159 |
-
Protocol:
|
| 160 |
-
|
| 161 |
-
```text
|
| 162 |
-
inspect_selection
|
| 163 |
-
→ server validates UTF-16 offsets, text, document scope, processed hash, collisions and occurrence count
|
| 164 |
-
→ ready / confirmation_required / blocked inspection result
|
| 165 |
-
→ commit_manual_mask with a current single-use inspection ID
|
| 166 |
-
→ server revalidates and creates the bound manual row
|
| 167 |
-
```
|
| 168 |
-
|
| 169 |
-
Safety boundaries:
|
| 170 |
-
- one to five safe exact occurrences are ready;
|
| 171 |
-
- six to twenty require explicit confirmation;
|
| 172 |
-
- more than twenty are blocked from the quick path;
|
| 173 |
-
- embedded substrings, nested included replacement terms, marked-range intersections, duplicates, stale views and replays fail closed;
|
| 174 |
-
- the review table remains source of truth and `Gemiste waarde toevoegen` remains fallback;
|
| 175 |
-
- browser code never creates placeholders, mutates the table, writes a Scrub Key or builds an export;
|
| 176 |
-
- no external assets, telemetry, browser persistence or cloud processing;
|
| 177 |
-
- no occurrence-specific replacement, rich editor or Streamlit upgrade in this line.
|
| 178 |
-
|
| 179 |
-
Reason:
|
| 180 |
-
- direct correction where a false negative is noticed reduces navigation friction and copying mistakes;
|
| 181 |
-
- a server-derived impact step is required because browser-supplied counts and types are untrusted;
|
| 182 |
-
- all-exact behavior matches current replacement/export/Scrub Key/reinsert semantics;
|
| 183 |
-
- occurrence-specific behavior would require a separate span-aware architecture.
|
| 184 |
-
|
| 185 |
-
Approved sequence:
|
| 186 |
-
1. `SCRUB-WP_PROCESSED_TEXT_SELECTION_MASKING_CONTRACT`
|
| 187 |
-
2. `SCRUB-WP_PROCESSED_TEXT_SELECTION_MASKING_ACTION_MODEL`
|
| 188 |
-
3. `SCRUB-WP_PROCESSED_TEXT_SELECTION_COMPONENT_SPIKE`
|
| 189 |
-
4. `SCRUB-WP_PROCESSED_TEXT_SELECTION_TABLE_INTEGRATION`
|
| 190 |
-
5. `SCRUB-WP_PROCESSED_TEXT_SELECTION_CROSS_FLOW_REGRESSION`
|
| 191 |
-
6. `SCRUB-WP_PROCESSED_TEXT_SELECTION_APP_VERIFY`
|
| 192 |
-
|
| 193 |
-
Evidence:
|
| 194 |
-
- coordinator/user approval at 2026-08-04 00:09 Europe/Amsterdam;
|
| 195 |
-
- `PROCESSED_TEXT_SELECTION_MASKING_PLAN.md`;
|
| 196 |
-
- `PROCESSED_TEXT_SELECTION_MASKING_CONTRACT.md`;
|
| 197 |
-
- `test_cases/processed_text_selection_masking/contract.json`.
|
| 198 |
-
|
| 199 |
-
## 2026-08-03 — D039 — Add Zorgfilter v1 with clinical-context preservation
|
| 200 |
-
|
| 201 |
-
Status: accepted product and implementation-sequence decision
|
| 202 |
-
|
| 203 |
-
Decision:
|
| 204 |
-
|
| 205 |
-
```text
|
| 206 |
-
Add an explicit Dutch Zorg profile. Replace date of birth and patient/client identity by default. Show other exact care dates, provider identity, BIG/AGB, organizations and locations for review and select them by default. Preserve diagnosis, medication, dosage, laboratory results and observations. Surface rare-case indirect re-identification as residual-risk evidence rather than blindly masking clinical meaning.
|
| 207 |
-
```
|
| 208 |
-
|
| 209 |
-
Implementation sequence:
|
| 210 |
-
- policy and fully synthetic corpus first;
|
| 211 |
-
- current-engine baseline and gap triage second;
|
| 212 |
-
- recognizer contracts and pure recognizer implementation before UI;
|
| 213 |
-
- central profile configuration before adding the fourth Streamlit profile;
|
| 214 |
-
- cross-profile regression and live app verification before closeout.
|
| 215 |
-
|
| 216 |
-
Reason:
|
| 217 |
-
- Care documents contain both direct identifiers and essential clinical meaning.
|
| 218 |
-
- A broad medical-word filter would destroy usability and potentially clinical/legal context.
|
| 219 |
-
- The current `NL_HEALTHCARE_REFERENCE` category combines values with different privacy policies and must be split through evidence-driven work.
|
| 220 |
-
- A fourth UI label without dedicated detection evidence would be cosmetic and unsafe.
|
| 221 |
-
|
| 222 |
-
Boundaries:
|
| 223 |
-
- synthetic data only;
|
| 224 |
-
- human review remains required;
|
| 225 |
-
- no production-readiness claim from corpus or benchmark results;
|
| 226 |
-
- no silent profile switching;
|
| 227 |
-
- no cloud document processing;
|
| 228 |
-
- no export, Scrub Key or reinsert semantic change in the foundation package.
|
| 229 |
-
|
| 230 |
-
## 2026-08-03 — D038 — Use AI-first execution with human-controlled signing, security and release gates for Phase 9 desktop packaging
|
| 231 |
-
|
| 232 |
-
Status: accepted roadmap and execution-model decision; implementation remains gated
|
| 233 |
-
|
| 234 |
-
Decision:
|
| 235 |
-
|
| 236 |
-
```text
|
| 237 |
-
When the final local desktop/offline installer phase is explicitly opened, use scoped AI agents for deterministic packaging, CI, synthetic testing and release-candidate preparation. Target a signed Tauri Windows shell with a bundled PyInstaller onedir Python/Presidio sidecar, a low-friction setup.exe and a managed-deployment MSI. Retain human control over publisher identity, production signing, public release, security claims, real-user acceptance and safety-critical semantic changes.
|
| 238 |
-
```
|
| 239 |
-
|
| 240 |
-
Planning assumptions:
|
| 241 |
-
- 60–70% of first-cycle development/integration labor may be agent-executed;
|
| 242 |
-
- 75–90% of repetitive later release work may be automated;
|
| 243 |
-
- post-agent development/integration planning range: approximately EUR 8,000–24,000;
|
| 244 |
-
- independent security review, test hardware, signing and human release oversight remain retained costs.
|
| 245 |
-
|
| 246 |
-
Boundaries:
|
| 247 |
-
- Phase 9 remains gated by Phase 6 quality closeout and explicit coordinator approval.
|
| 248 |
-
- No installer, runtime, UI, dependency, export, Scrub Key, reinsert or cloud behavior changes in this decision package.
|
| 249 |
-
- Synthetic data only.
|
| 250 |
-
- No single agent receives unrestricted repository-write, signing-identity and public-release authority.
|
| 251 |
-
- Successful packaging alone does not justify a production-readiness or local-only security claim.
|
| 252 |
-
|
| 253 |
-
## 2026-07-28 — D037 — Validate document/key binding before every reinsert replacement
|
| 254 |
-
|
| 255 |
-
Status: accepted reinsert-integration decision
|
| 256 |
-
|
| 257 |
-
Decision:
|
| 258 |
-
|
| 259 |
-
```text
|
| 260 |
-
Validate the complete supported text surface against the supplied Scrub Key before any original value is restored. Permit verified bound matches and explicit legacy-v1.0 compatibility only. All mismatch, mixed-binding, missing-binding, invalid-digest and invalid-bound-key states fail closed with zero replacements and no partial DOCX output.
|
| 261 |
-
```
|
| 262 |
-
|
| 263 |
-
Reason:
|
| 264 |
-
|
| 265 |
-
- Structural key validity alone cannot prove that a key belongs to a document.
|
| 266 |
-
- Applying a wrong but valid key can silently restore incorrect confidential values.
|
| 267 |
-
- Validation must happen before mutation so document-level helpers cannot produce partial output.
|
| 268 |
-
- Legacy compatibility remains necessary, but it must never be presented as a verified document match.
|
| 269 |
-
- The document-first three-step UX remains simpler and does not require new execution gates.
|
| 270 |
|
| 271 |
---
|
| 272 |
|
| 273 |
-
## 2026-
|
| 274 |
|
| 275 |
-
Status: accepted
|
| 276 |
|
| 277 |
Decision:
|
| 278 |
|
| 279 |
```text
|
| 280 |
-
|
| 281 |
-
```
|
| 282 |
-
|
| 283 |
-
Reason:
|
| 284 |
-
|
| 285 |
-
- Review choices remain source of truth.
|
| 286 |
-
- Silent rewriting would change legal/readability semantics and user intent.
|
| 287 |
-
- A bound key must not claim verified document matching for an unbound mapping.
|
| 288 |
-
- Fail-visible key export is safer than a false binding claim.
|
| 289 |
|
| 290 |
-
|
| 291 |
-
|
| 292 |
-
# SolidPrivacy Scrub — Decision Log
|
| 293 |
-
|
| 294 |
-
This file records accepted strategic, product and architecture decisions.
|
| 295 |
-
|
| 296 |
-
---
|
| 297 |
-
|
| 298 |
-
## 2026-07-27 — D035 — Keep binding-model implementation pure until sequential export and reinsert integration
|
| 299 |
-
|
| 300 |
-
Status: accepted implementation decision
|
| 301 |
-
|
| 302 |
-
Decision:
|
| 303 |
-
|
| 304 |
-
```text
|
| 305 |
-
Implement document/Scrub-Key binding as a new pure helper module first. Do not alter current placeholder creation, Scrub Key export/import, deterministic replacement or Streamlit behavior in the model package. Export integration creates bound artifacts in the next package; reinsert integration enforces binding only after bound export is proven.
|
| 306 |
```
|
| 307 |
|
| 308 |
-
|
| 309 |
-
|
| 310 |
-
- Placeholder generation, key export and reinsert are shared safety-critical surfaces.
|
| 311 |
-
- Pure helpers can be validated completely against the frozen contract without silently changing current output semantics.
|
| 312 |
-
- Sequential integration preserves explicit migration and rollback boundaries.
|
| 313 |
|
| 314 |
-
|
| 315 |
-
|
| 316 |
-
-
|
| 317 |
-
-
|
| 318 |
-
-
|
| 319 |
-
-
|
| 320 |
-
-
|
| 321 |
-
-
|
| 322 |
-
-
|
| 323 |
-
- no cloud, AI, file persistence, signing or secret storage.
|
| 324 |
-
|
| 325 |
-
Evidence:
|
| 326 |
-
|
| 327 |
-
- `scrub_key_binding.py`
|
| 328 |
-
- `tests/test_scrub_key_binding_model.py`
|
| 329 |
-
- `output/validation/mvp_scrub_key_binding_model_validation.json`
|
| 330 |
|
| 331 |
---
|
| 332 |
|
| 333 |
-
## 2026-
|
| 334 |
|
| 335 |
-
Status: accepted
|
| 336 |
|
| 337 |
Decision:
|
| 338 |
|
| 339 |
```text
|
| 340 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 341 |
```
|
| 342 |
|
| 343 |
-
|
| 344 |
-
|
| 345 |
-
- Exact grammar and canonicalization are required before multiple shared placeholder, export and reinsert surfaces change.
|
| 346 |
-
- A fixed synthetic digest fixture makes implementation independently testable.
|
| 347 |
-
- Bound and legacy statuses must not be conflated.
|
| 348 |
-
- UI simplification must survive the security change without new source/key execution gates.
|
| 349 |
-
|
| 350 |
-
Boundaries:
|
| 351 |
-
|
| 352 |
-
- Contract/tests only in this package; no product behavior change.
|
| 353 |
-
- Mapping digest is not authenticity or a signature.
|
| 354 |
-
- No automatic placeholder repair or legacy upgrade.
|
| 355 |
-
- Preserve the three-step document-first reinsert flow and final confidential-download acknowledgement.
|
| 356 |
-
- Model implementation remains pure and Streamlit-free.
|
| 357 |
-
- Export and reinsert integration require later sequential packages.
|
| 358 |
-
- Human review remains mandatory; no production-readiness claim.
|
| 359 |
-
|
| 360 |
-
Evidence:
|
| 361 |
-
|
| 362 |
-
- `SCRUB_KEY_BINDING_CONTRACT.md`
|
| 363 |
-
- `test_cases/mvp_phase6/scrub_key_binding_contract.json`
|
| 364 |
-
- `tests/test_mvp_scrub_key_binding_contracts.py`
|
| 365 |
-
- `output/validation/mvp_scrub_key_binding_contract_validation.json`
|
| 366 |
|
| 367 |
---
|
| 368 |
|
| 369 |
-
## 2026-
|
| 370 |
|
| 371 |
-
Status: accepted
|
| 372 |
|
| 373 |
Decision:
|
| 374 |
|
| 375 |
-
``
|
| 376 |
-
|
| 377 |
-
``
|
| 378 |
-
|
| 379 |
-
|
| 380 |
-
|
| 381 |
-
- Generic placeholder namespaces allow a wrong valid key to restore wrong originals without an audit mismatch.
|
| 382 |
-
- A binding token inside placeholders survives pasted-text, TXT, DOCX and PDF-text roundtrips when the placeholders themselves survive.
|
| 383 |
-
- Labels, filenames, metadata, content hashes and placeholder-list hashes are not reliable cross-format AI-roundtrip bindings.
|
| 384 |
-
- A digest detects accidental edits but is not authenticity; malicious tampering requires later protected signing-key infrastructure.
|
| 385 |
-
|
| 386 |
-
Compatibility and UX boundaries:
|
| 387 |
-
|
| 388 |
-
- Introduce an explicit new bound-key contract; do not silently reinterpret legacy v1.0 keys.
|
| 389 |
-
- Legacy unbound keys may remain dual-readable with a visible unbound warning.
|
| 390 |
-
- Preserve the document-first three-step reinsert flow and the final confidential-download acknowledgement.
|
| 391 |
-
- Add no repeated confirmation buttons or checkboxes.
|
| 392 |
-
- Preserve unknown, duplicate and missing-placeholder audit reporting.
|
| 393 |
-
- No cloud processing, server secret, OCR or restored-PDF behavior.
|
| 394 |
-
- Human review remains mandatory; no production-readiness claim.
|
| 395 |
-
|
| 396 |
-
Approved sequence:
|
| 397 |
-
|
| 398 |
-
1. `SCRUB-WP_MVP_SCRUB_KEY_BINDING_CONTRACT_TESTS`
|
| 399 |
-
2. `SCRUB-WP_MVP_SCRUB_KEY_BINDING_MODEL_IMPLEMENTATION`
|
| 400 |
-
3. `SCRUB-WP_MVP_SCRUB_KEY_BINDING_EXPORT_INTEGRATION`
|
| 401 |
-
4. `SCRUB-WP_MVP_SCRUB_KEY_BINDING_REINSERT_INTEGRATION`
|
| 402 |
-
5. `SCRUB-WP_MVP_SCRUB_KEY_BINDING_APP_VERIFY`
|
| 403 |
-
|
| 404 |
-
Evidence:
|
| 405 |
-
|
| 406 |
-
- `MVP_SCRUB_KEY_DOCUMENT_BINDING_GAP_TRIAGE.md`
|
| 407 |
-
- `output/validation/mvp_scrub_key_document_binding_gap_triage.json`
|
| 408 |
-
- `output/validation/mvp_scrub_key_roundtrip_validation_report.json`
|
| 409 |
|
| 410 |
---
|
| 411 |
|
| 412 |
-
## 2026-
|
| 413 |
|
| 414 |
-
Status: accepted
|
| 415 |
|
| 416 |
Decision:
|
| 417 |
|
| 418 |
-
|
| 419 |
-
Treat the missing document/Scrub-Key binding as a critical Phase 6 finding. Do not implement an implicit heuristic or silently change the Scrub Key schema, export or reinsert semantics inside the validation package. Open a separate triage package to define the binding contract and migration boundaries first.
|
| 420 |
-
```
|
| 421 |
-
|
| 422 |
-
Reason:
|
| 423 |
-
|
| 424 |
-
- A wrong key with a disjoint placeholder namespace is visibly rejected through unknown/not-found audit evidence.
|
| 425 |
-
- A structurally valid wrong or tampered key with the same placeholder names restores wrong original values with no validation issue, unknown placeholder or missing-placeholder signal.
|
| 426 |
-
- Document labels are descriptive metadata and are not currently a verified binding mechanism.
|
| 427 |
-
- A safe correction may affect key creation, scrubbed output metadata, import compatibility and export semantics.
|
| 428 |
-
|
| 429 |
-
Boundaries:
|
| 430 |
|
| 431 |
-
|
| 432 |
-
- Do not guess whether a key belongs to a document.
|
| 433 |
-
- Do not auto-repair malformed placeholders.
|
| 434 |
-
- Preserve existing audit evidence and human review.
|
| 435 |
-
- Use synthetic data only; no production-readiness claim.
|
| 436 |
-
|
| 437 |
-
Evidence:
|
| 438 |
-
|
| 439 |
-
- `output/validation/mvp_scrub_key_roundtrip_validation_report.json`
|
| 440 |
-
- `test_cases/mvp_phase6/scrub_key_roundtrip_manifest.json`
|
| 441 |
-
- `tests/test_mvp_scrub_key_roundtrip_validation.py`
|
| 442 |
|
| 443 |
---
|
| 444 |
|
| 445 |
-
## 2026-
|
| 446 |
|
| 447 |
-
Status: accepted
|
| 448 |
|
| 449 |
Decision:
|
| 450 |
|
| 451 |
-
|
| 452 |
-
|
| 453 |
-
|
| 454 |
-
|
| 455 |
-
|
| 456 |
-
|
| 457 |
-
- Live Phase 6 verification confirmed that DOCX restoration works, but users can reasonably interpret an uploaded and visibly listed file as already accepted.
|
| 458 |
-
- Requiring a checkbox and action button after each upload creates hidden completion states and unnecessary form friction.
|
| 459 |
-
- The highest-risk user action is obtaining and handling the restored confidential output, so the explicit acknowledgement remains at that boundary.
|
| 460 |
-
- Automatic key validation does not weaken structural validation or change Scrub Key semantics.
|
| 461 |
-
|
| 462 |
-
Boundaries:
|
| 463 |
-
|
| 464 |
-
- Keep warnings about pseudonymisation, key sensitivity and local-only use visible.
|
| 465 |
-
- Invalid or ambiguous keys must still fail clearly and must not be used.
|
| 466 |
-
- Preserve result/audit warnings for unknown, duplicate and missing placeholders.
|
| 467 |
-
- Preserve existing helpers, output bytes, filenames and MIME types.
|
| 468 |
-
- Do not add cloud, AI, OCR, restored-PDF or key-storage behavior.
|
| 469 |
-
- Human review remains required and no production-readiness claim is created.
|
| 470 |
-
|
| 471 |
-
Evidence:
|
| 472 |
-
|
| 473 |
-
- User-confirmed restored synthetic DOCX containing body, table, header and footer values.
|
| 474 |
-
- `tests/test_reinsert_auto_flow.py`
|
| 475 |
-
- `tests/test_reinsert_auto_flow_ui.py`
|
| 476 |
-
- `output/validation/mvp_reinsert_auto_flow_validation.json`
|
| 477 |
|
| 478 |
---
|
| 479 |
|
| 480 |
-
## 2026-
|
| 481 |
|
| 482 |
-
Status:
|
| 483 |
|
| 484 |
-
|
| 485 |
|
| 486 |
-
|
| 487 |
-
Extend the existing local deterministic DOCX reinsert helper to process WordprocessingML text nodes in word/header*.xml and word/footer*.xml in addition to word/document.xml.
|
| 488 |
-
```
|
| 489 |
-
|
| 490 |
-
Reason:
|
| 491 |
-
|
| 492 |
-
- The scrubbed DOCX export already replaces reviewed values in body, table, header and footer paragraphs.
|
| 493 |
-
- The Phase 6 matrix showed that reinsert restored body/table values but left header/footer placeholders behind.
|
| 494 |
-
- The gap belongs to document fidelity and reinsert scope, not detection or recognizer behavior.
|
| 495 |
-
|
| 496 |
-
Boundaries:
|
| 497 |
-
|
| 498 |
-
- Process only existing body, header and footer WordprocessingML text nodes.
|
| 499 |
-
- Preserve unrelated OOXML package parts byte-for-byte where they are not rewritten.
|
| 500 |
-
- Do not claim support for comments, tracked-change-only parts, footnotes/endnotes, text boxes, metadata or placeholders split across text nodes.
|
| 501 |
-
- Do not add OCR or restored-PDF behavior.
|
| 502 |
-
- Keep processing local, deterministic and Scrub Key driven.
|
| 503 |
-
|
| 504 |
-
Evidence:
|
| 505 |
-
|
| 506 |
-
- `output/validation/mvp_phase6_document_hygiene_fidelity_hardening_report.json`
|
| 507 |
-
- `output/validation/mvp_phase6_false_negative_gap_triage.json`
|
| 508 |
|
| 509 |
---
|
| 510 |
|
| 511 |
-
## 2026-07-
|
| 512 |
|
| 513 |
-
Status: accepted
|
| 514 |
|
| 515 |
Decision:
|
| 516 |
|
| 517 |
-
|
| 518 |
-
Do not open a recognizer or threshold implementation package from the first corrected Phase 6 synthetic matrix.
|
| 519 |
-
```
|
| 520 |
-
|
| 521 |
-
Reason:
|
| 522 |
-
|
| 523 |
-
- The corrected matrix contains no reproducible detection false negative, misclassification or legal-role over-masking evidence.
|
| 524 |
-
- The DOCX result concerns header/footer reinsert fidelity and helper scope.
|
| 525 |
-
- The PDF result reflects the approved restored-TXT-only/no-OCR product boundary.
|
| 526 |
-
- Treating either item as a recognizer problem would target the wrong layer and weaken evidence discipline.
|
| 527 |
-
|
| 528 |
-
Consequences:
|
| 529 |
-
|
| 530 |
-
- Route both findings to `SCRUB-WP_MVP_DOCUMENT_HYGIENE_FIDELITY_HARDENING`.
|
| 531 |
-
- Preserve current recognizer and threshold behavior.
|
| 532 |
-
- Keep the PDF limitation explicit; do not infer OCR or restored-PDF authorization.
|
| 533 |
-
- Preserve human review and the no-production-readiness-claim boundary.
|
| 534 |
-
|
| 535 |
-
Evidence:
|
| 536 |
-
|
| 537 |
-
- `output/validation/mvp_phase6_false_negative_gap_triage.json`
|
| 538 |
-
- `output/validation/mvp_phase6_synthetic_validation_report.json`
|
| 539 |
|
| 540 |
---
|
| 541 |
|
| 542 |
-
## 2026-07-
|
| 543 |
|
| 544 |
-
Status: accepted
|
| 545 |
|
| 546 |
Decision:
|
| 547 |
|
| 548 |
-
|
| 549 |
-
Close the verified MVP UI simplification line as the default development focus and activate Phase 6 end-to-end workflow validation and trust hardening.
|
| 550 |
-
```
|
| 551 |
-
|
| 552 |
-
Reason:
|
| 553 |
-
|
| 554 |
-
```text
|
| 555 |
-
The current import, review, manual correction and export interface has been live-app verified and works as expected. The next material risk reduction comes from proving the supported workflow with synthetic evidence, then fixing only reproducible trust gaps.
|
| 556 |
-
```
|
| 557 |
-
|
| 558 |
-
Consequences:
|
| 559 |
-
|
| 560 |
-
- Start with `SCRUB-WP_MVP_E2E_SYNTHETIC_VALIDATION_MATRIX`.
|
| 561 |
-
- Open recognizer, document-hygiene, Scrub Key/roundtrip or audit fixes only from reproducible evidence.
|
| 562 |
-
- Do not start another broad UI package by default.
|
| 563 |
-
- Phase 7 pilots remain parked until the Phase 6 quality gate is explicitly approved.
|
| 564 |
-
- Local installer/packaging work remains deferred.
|
| 565 |
-
- Human review remains mandatory; no production-readiness claim is created by this decision.
|
| 566 |
|
| 567 |
---
|
| 568 |
|
| 569 |
-
## 2026-07-
|
| 570 |
-
|
| 571 |
-
Status: accepted planning recommendation, implementation pending contract tests
|
| 572 |
-
|
| 573 |
-
Decision:
|
| 574 |
-
|
| 575 |
-
```text
|
| 576 |
-
Use Basiscontrole / Expertcontrole as the planning names for the two normal review-mode layers.
|
| 577 |
-
Basiscontrole is the default MVP path.
|
| 578 |
-
Expertcontrole exposes the full inspection/audit machinery.
|
| 579 |
-
Mode switching changes visibility only, not processing or export semantics.
|
| 580 |
-
```
|
| 581 |
-
|
| 582 |
-
Reason:
|
| 583 |
-
|
| 584 |
-
```text
|
| 585 |
-
The MVP interface needs a true less-is-more default path without weakening legal/privacy review controls. Basiscontrole communicates lower cognitive load while keeping the workflow framed as a serious control task.
|
| 586 |
-
```
|
| 587 |
|
| 588 |
-
|
| 589 |
|
| 590 |
-
-
|
| 591 |
-
- The review table remains source of truth internally.
|
| 592 |
-
- Mode switching must not change recognizer behavior, replacement logic, export output, Scrub Key JSON, reinsert behavior or audit generation.
|
| 593 |
-
- Implementation requires contract tests first.
|
| 594 |
|
| 595 |
---
|
| 596 |
|
| 597 |
-
## 2026-
|
| 598 |
-
|
| 599 |
-
Status: accepted product-direction decision
|
| 600 |
|
| 601 |
-
|
| 602 |
|
| 603 |
-
|
| 604 |
-
Pause new recall/benchmark follow-up packages temporarily and prioritize MVP UI cleanup/export redesign.
|
| 605 |
-
```
|
| 606 |
|
| 607 |
-
|
| 608 |
-
|
| 609 |
-
|
| 610 |
-
|
| 611 |
-
|
| 612 |
-
|
| 613 |
-
Consequence:
|
| 614 |
-
|
| 615 |
-
```text
|
| 616 |
-
Next packages focus on export/download UX and hiding/collapsing debug details without weakening safety controls.
|
| 617 |
-
```
|
| 618 |
-
|
| 619 |
-
Implications:
|
| 620 |
|
| 621 |
-
|
| 622 |
-
- Export/download UX is now the active next user-visible improvement line.
|
| 623 |
-
- Technical/audit details must remain available but move out of the primary flow where appropriate.
|
| 624 |
-
- Scrub Key must stay clearly separated and visibly sensitive.
|
| 625 |
-
- Export semantics must not change silently.
|
| 626 |
-
- The review table remains source of truth and fallback.
|
| 627 |
-
- No Streamlit implementation is approved by this planning decision; implementation requires separate workpackages.
|
| 628 |
|
| 629 |
---
|
| 630 |
|
| 631 |
-
## 2026-
|
| 632 |
-
|
| 633 |
-
Status: accepted tests/specification decision
|
| 634 |
-
|
| 635 |
-
Decision:
|
| 636 |
-
|
| 637 |
-
```text
|
| 638 |
-
PERSON-name recognizer implementation may only start after contract tests are green.
|
| 639 |
-
```
|
| 640 |
-
|
| 641 |
-
Reason:
|
| 642 |
|
| 643 |
-
|
| 644 |
-
Value-only matching and role preservation are safety-critical.
|
| 645 |
-
```
|
| 646 |
|
| 647 |
-
|
| 648 |
|
| 649 |
-
|
| 650 |
-
Implementation package must satisfy the contract fixture before benchmark review.
|
| 651 |
-
```
|
| 652 |
|
| 653 |
---
|
| 654 |
|
| 655 |
-
## 2026-
|
| 656 |
-
|
| 657 |
-
Status: accepted planning/specification decision
|
| 658 |
|
| 659 |
-
|
| 660 |
-
|
| 661 |
-
```text
|
| 662 |
-
PERSON-name improvement will proceed test-first.
|
| 663 |
-
```
|
| 664 |
|
| 665 |
-
|
| 666 |
|
| 667 |
```text
|
| 668 |
-
|
|
|
|
|
|
|
|
|
|
| 669 |
```
|
| 670 |
|
| 671 |
-
|
| 672 |
-
|
| 673 |
-
```text
|
| 674 |
-
Contract tests are required before recognizer implementation.
|
| 675 |
-
```
|
| 676 |
|
| 677 |
---
|
| 678 |
|
| 679 |
-
## 2026-
|
| 680 |
-
|
| 681 |
-
Status: accepted bounded UX refinement decision
|
| 682 |
-
|
| 683 |
-
Decision:
|
| 684 |
-
|
| 685 |
-
```text
|
| 686 |
-
In the central side-by-side review surface, synchronized scrolling should be on by default and should not be exposed as a visible checkbox.
|
| 687 |
-
```
|
| 688 |
|
| 689 |
-
|
| 690 |
|
| 691 |
-
|
| 692 |
|
| 693 |
---
|
| 694 |
|
| 695 |
-
## 2026-06-
|
| 696 |
-
|
| 697 |
-
Status: accepted bounded UX implementation decision; refined by D023 for visible control behavior
|
| 698 |
-
|
| 699 |
-
Decision:
|
| 700 |
|
| 701 |
-
|
| 702 |
-
After visual review of the isolated synchronized-scroll prototype, bounded synchronized scrolling may be integrated into the existing side-by-side review surface.
|
| 703 |
-
```
|
| 704 |
|
| 705 |
-
|
| 706 |
|
| 707 |
-
|
| 708 |
-
- Do not change replacement behavior.
|
| 709 |
-
- Do not mutate review table state.
|
| 710 |
-
- Do not write or change Scrub Key data.
|
| 711 |
-
- Do not change export/download behavior.
|
| 712 |
-
- Do not change reinsert behavior.
|
| 713 |
-
- Do not use real data.
|
| 714 |
|
| 715 |
---
|
| 716 |
|
| 717 |
-
#
|
| 718 |
-
|
| 719 |
-
Status: accepted product/UX direction
|
| 720 |
-
|
| 721 |
-
Decision:
|
| 722 |
-
|
| 723 |
-
```text
|
| 724 |
-
The review UX should move toward one unified side-by-side main review surface: source text on the left, processed/checked text on the right, with optional highlights integrated in the processed-text pane. The product should not keep adding separate helper panels or duplicate preview expanders for every review feature.
|
| 725 |
-
```
|
| 726 |
-
|
| 727 |
-
Implications:
|
| 728 |
-
|
| 729 |
-
- Future review UX work should centralize around source-vs-processed comparison.
|
| 730 |
-
- The review table remains source of truth and fallback.
|
| 731 |
-
- Serial review remains a guided review layer, not a replacement of the table.
|
| 732 |
-
- The old replacement decision helper panel must not return as normal user-facing UI.
|
| 733 |
-
- Do not start panel removal, click-to-mark, advanced editor, full-document marking, Scrub Key writes, export blocking or reinsert behavior changes without separate approved packages.
|
| 734 |
|
| 735 |
-
|
| 736 |
|
| 737 |
-
|
| 738 |
|
| 739 |
-
|
|
|
|
| 1 |
+
# SolidPrivacy Scrub — Current Decision Log
|
| 2 |
|
| 3 |
+
This file records **accepted decisions that remain materially binding on current/future product work**.
|
| 4 |
|
| 5 |
+
Detailed pre-convergence decision history remains recoverable from exact baseline:
|
| 6 |
|
| 7 |
```text
|
| 8 |
+
54c73e0ebf5a3a3ed7039a50596fb57694add3cd
|
| 9 |
```
|
| 10 |
|
| 11 |
+
Git and `CHANGELOG.md` are the historical provenance layers. This file must not become another implementation-history ledger.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
|
| 13 |
---
|
| 14 |
|
| 15 |
+
## D044 — 2026-09-04 — Repository Convergence before further normal feature development; Scrub Private becomes the active future production line
|
| 16 |
|
| 17 |
+
Status: **accepted strategic/architecture direction; implementation of the bootstrap candidate remains subject to normal independent assurance**
|
| 18 |
|
| 19 |
Decision:
|
| 20 |
|
| 21 |
```text
|
| 22 |
+
Pause normal new feature development.
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
+
Preserve exact current source by SHA
|
| 25 |
+
→ reconstruct active/reachable current truth
|
| 26 |
+
→ classify CANONICAL / RECONCILE / RETIRE / VARIANT-SPECIFIC
|
| 27 |
+
→ remove only proven debt
|
| 28 |
+
→ reconcile current documentation/issues/evidence authority
|
| 29 |
+
→ verify one clean baseline
|
| 30 |
+
→ after convergence, continue main as Scrub Private
|
| 31 |
+
→ finish Private-oriented application behavior on Hugging Face with synthetic data
|
| 32 |
+
→ add production Private Service controls
|
| 33 |
+
→ external assurance
|
| 34 |
+
→ pilot Legal first, then Zorg based on evidence.
|
| 35 |
```
|
| 36 |
|
| 37 |
+
Reasons:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
|
| 39 |
+
- current repository contains mature functionality that must not be rebuilt from stale assumptions;
|
| 40 |
+
- canonical docs/workpackages/issues contain multiple generations of status and local-first deployment assumptions;
|
| 41 |
+
- active technical legacy and valid-but-hosted-incompatible functionality must be distinguished before removal;
|
| 42 |
+
- version control preserves prior/full-feature functionality more safely than an in-repository source clone;
|
| 43 |
+
- the shortest path to the business outcome is to clean current truth first, then adapt only the trust boundary required by Scrub Private.
|
| 44 |
|
| 45 |
+
Binding consequences:
|
| 46 |
|
| 47 |
+
- exact Git SHA is preservation authority; optional tags are only human-readable markers;
|
| 48 |
+
- no `/app_v2`, `/scrub-new`, duplicate Streamlit main or other source clone;
|
| 49 |
+
- `ROADMAP.md` contains five strategic stages only;
|
| 50 |
+
- `WORKPACKAGES.md` contains one current executable queue only;
|
| 51 |
+
- the temporary convergence debt ledger is not a new permanent source of truth;
|
| 52 |
+
- no cleanup refactor is justified by aesthetics alone;
|
| 53 |
+
- no new Evidence Framework; reconcile/reuse existing synthetic/benchmark/E2E/security evidence;
|
| 54 |
+
- human review and existing privacy/safety semantics remain binding;
|
| 55 |
+
- Hugging Face is a synthetic/approved-test application-validation environment, not confidential-production infrastructure assurance;
|
| 56 |
+
- Local/offline capability remains recoverable/deferred and is not the active delivery line;
|
| 57 |
+
- after `SCRUB_REPOSITORY_CONVERGED`, `main` becomes the active Scrub Private development line.
|
| 58 |
|
| 59 |
+
Scrub Private content-plane direction:
|
| 60 |
|
| 61 |
```text
|
| 62 |
+
no intentional persistent customer document content
|
| 63 |
+
no content-bearing ordinary application logs
|
| 64 |
+
no document-content backup
|
| 65 |
+
no third-party document-processing egress
|
| 66 |
+
minimal non-document control-plane persistence only when a real service requirement justifies it
|
| 67 |
```
|
| 68 |
|
| 69 |
+
This decision does not itself remove current replacement-memory or external-processing functionality. Those are variant-specific candidates for a later, separately assured Private adaptation after the clean baseline.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
|
| 71 |
---
|
| 72 |
|
| 73 |
+
## D043 — 2026-08-08 — Premium Standard is a single-page staged document workspace
|
| 74 |
|
| 75 |
+
Status: accepted and implemented direction; preserve unless current runtime evidence justifies change.
|
| 76 |
|
| 77 |
Decision:
|
| 78 |
|
| 79 |
```text
|
| 80 |
+
One document. One workspace. Three stages. One active task.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 81 |
|
| 82 |
+
Toevoegen → Controleren → Downloaden
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 83 |
```
|
| 84 |
|
| 85 |
+
Binding behavior:
|
|
|
|
|
|
|
|
|
|
|
|
|
| 86 |
|
| 87 |
+
- all three stages remain represented in one workspace;
|
| 88 |
+
- exactly one Standard stage is dominant at a time;
|
| 89 |
+
- completed stages collapse to compact summaries;
|
| 90 |
+
- future stages are visible but passive;
|
| 91 |
+
- successful completion advances safely;
|
| 92 |
+
- explicit return to earlier stages is allowed;
|
| 93 |
+
- processing-affecting changes invalidate downstream review/export state fail-closed;
|
| 94 |
+
- Standard is lower cognitive load, not lower safety;
|
| 95 |
+
- three routed pages and a nested-expander long form are rejected as the Standard core-flow target.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 96 |
|
| 97 |
---
|
| 98 |
|
| 99 |
+
## D042 — 2026-08-06 — Separate implementation from blind independent release assurance
|
| 100 |
|
| 101 |
+
Status: accepted and binding governance decision.
|
| 102 |
|
| 103 |
Decision:
|
| 104 |
|
| 105 |
```text
|
| 106 |
+
implementation_operations
|
| 107 |
+
→ identifiable exact candidate
|
| 108 |
+
→ governance_release_assurance fresh blind reconstruction
|
| 109 |
+
→ PASS | FAIL | INDETERMINATE
|
| 110 |
+
→ authorized action
|
| 111 |
+
→ independent post-action confirmation
|
| 112 |
```
|
| 113 |
|
| 114 |
+
Implementation may not certify its own consequential candidate. Assurance may not repair what it reviews and may not read implementation conclusions before its initial verdict.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 115 |
|
| 116 |
---
|
| 117 |
|
| 118 |
+
## D041 — 2026-08-05 — Application-wide Standard/Expert premium core-flow shell
|
| 119 |
|
| 120 |
+
Status: accepted and implemented direction.
|
| 121 |
|
| 122 |
Decision:
|
| 123 |
|
| 124 |
+
- top-level `Anonimiseren | Terugzetten` workflows;
|
| 125 |
+
- global `Standaard | Expert` presentation;
|
| 126 |
+
- Standard uses the staged `Toevoegen → Controleren → Downloaden` workspace;
|
| 127 |
+
- Expert operates over the same authoritative source/processing/review state;
|
| 128 |
+
- presentation-only switching must not silently change profile/settings, reprocess or reset valid state;
|
| 129 |
+
- human review and export/Scrub Key/reinsert semantics are not weakened by presentation simplification.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 130 |
|
| 131 |
---
|
| 132 |
|
| 133 |
+
## D040 — 2026-08-04 — Server-authoritative direct masking from processed text
|
| 134 |
|
| 135 |
+
Status: accepted and implemented product decision.
|
| 136 |
|
| 137 |
Decision:
|
| 138 |
|
| 139 |
+
A user may select a missed sensitive value in processed text, but browser selection is only an input event. The server validates scope, offsets, exact occurrences, collisions and stale/replay state before creating one normal document-bound manual replacement row.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 140 |
|
| 141 |
+
The review table/authoritative review state remains source of truth. Direct masking may not write Scrub Keys or exports independently.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 142 |
|
| 143 |
---
|
| 144 |
|
| 145 |
+
## D039 — 2026-08-03 — Zorg profile with clinical-context preservation
|
| 146 |
|
| 147 |
+
Status: accepted and implemented product decision.
|
| 148 |
|
| 149 |
Decision:
|
| 150 |
|
| 151 |
+
- direct patient/client identity and date of birth: replace;
|
| 152 |
+
- other exact care dates/provider identity and dedicated care identifiers: review according to policy, selected by default where defined;
|
| 153 |
+
- diagnosis, medication, dosage, laboratory results, observations and clinically meaningful context: preserve;
|
| 154 |
+
- rare indirect-identification risk: surface for review/audit rather than blindly destroying clinical meaning;
|
| 155 |
+
- synthetic corpus/evidence and human review remain mandatory; no production-safety claim from corpus scores alone.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 156 |
|
| 157 |
---
|
| 158 |
|
| 159 |
+
## D038 — 2026-08-03 — Local desktop packaging is a deferred product option, not current execution authority
|
| 160 |
|
| 161 |
+
Status: historical decision retained only where still applicable.
|
| 162 |
|
| 163 |
+
The previously selected signed Tauri/PyInstaller local packaging direction remains recoverable if Local becomes an active product requirement later.
|
| 164 |
|
| 165 |
+
D044 supersedes its roadmap priority. Installer work is not part of the current execution line.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 166 |
|
| 167 |
---
|
| 168 |
|
| 169 |
+
## D037 — 2026-07-28 — Validate document/key binding before every reinsert replacement
|
| 170 |
|
| 171 |
+
Status: accepted and binding safety decision.
|
| 172 |
|
| 173 |
Decision:
|
| 174 |
|
| 175 |
+
Validate the complete supported document text surface against the supplied Scrub Key before restoring any original value. Bound mismatch, mixed binding, missing binding or invalid mapping digest fails closed with zero replacements. Legacy unbound behavior remains explicitly unverified where supported.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 176 |
|
| 177 |
---
|
| 178 |
|
| 179 |
+
## D036 — 2026-07-27 — Preserve intentional custom replacement text; fail verified key export rather than silently rewriting it
|
| 180 |
|
| 181 |
+
Status: accepted and binding review/export decision.
|
| 182 |
|
| 183 |
Decision:
|
| 184 |
|
| 185 |
+
Do not silently convert arbitrary user-chosen replacement text into bound placeholders merely to make Scrub Key export succeed. Preserve the reviewed replacement decision and fail visibly/block verified key export when a fully bound mapping cannot be established.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 186 |
|
| 187 |
---
|
| 188 |
|
| 189 |
+
## D035 — 2026-07-27 — Implement Scrub Key binding in pure model first, then integrate sequentially
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 190 |
|
| 191 |
+
Status: accepted implementation/safety pattern; completed historically.
|
| 192 |
|
| 193 |
+
Decision remains relevant as a precedent: safety-critical shared placeholder/key/reinsert changes should be isolated in pure/testable helpers before sequential integration into export/reinsert when new changes are genuinely required.
|
|
|
|
|
|
|
|
|
|
| 194 |
|
| 195 |
---
|
| 196 |
|
| 197 |
+
## D034 — 2026-07-27 — Bound-placeholder and mapping-digest contract
|
|
|
|
|
|
|
| 198 |
|
| 199 |
+
Status: accepted and implemented safety contract.
|
| 200 |
|
| 201 |
+
Binding direction:
|
|
|
|
|
|
|
| 202 |
|
| 203 |
+
- document-specific binding ID in automatic/manual bound placeholders;
|
| 204 |
+
- schema-1.1 bound key direction;
|
| 205 |
+
- canonical SHA-256 mapping digest;
|
| 206 |
+
- explicit legacy-unbound compatibility state;
|
| 207 |
+
- mismatch/mixed/missing/digest-invalid bound states fail closed before replacement.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 208 |
|
| 209 |
+
Do not shorten or mutate authoritative bound tokens merely for visual convenience; compact aliases are presentation-only.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 210 |
|
| 211 |
---
|
| 212 |
|
| 213 |
+
## D033 — 2026-07-27 — Document-specific placeholder namespaces bind new Scrub Keys to their document
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 214 |
|
| 215 |
+
Status: accepted and implemented architecture decision.
|
|
|
|
|
|
|
| 216 |
|
| 217 |
+
Each new bound Scrub Key/document uses one locally generated non-sensitive binding ID carried by automatic/manual placeholders and the corresponding key. A mapping digest complements this for accidental corruption detection. The binding token, not filenames/labels/content heuristics, is the cross-format document/key association.
|
| 218 |
|
| 219 |
+
Legacy unbound keys remain explicitly distinguishable and must not be silently reinterpreted as verified bound matches.
|
|
|
|
|
|
|
| 220 |
|
| 221 |
---
|
| 222 |
|
| 223 |
+
## D031 — 2026-07-27 — Reinsert is document-first and automatically processes valid source/key inputs
|
|
|
|
|
|
|
| 224 |
|
| 225 |
+
Status: accepted and implemented UX/safety decision.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 226 |
|
| 227 |
+
Current reinsert direction:
|
| 228 |
|
| 229 |
```text
|
| 230 |
+
source document/text
|
| 231 |
+
→ corresponding Scrub Key
|
| 232 |
+
→ deterministic validation/reinsert
|
| 233 |
+
→ restored download
|
| 234 |
```
|
| 235 |
|
| 236 |
+
Source type recognition and structural key validation may run automatically when valid inputs are present. Do not reintroduce redundant hidden confirmation buttons/checkboxes merely to create a sense of safety. Keep the meaningful confidentiality acknowledgement at the restored-output boundary and fail invalid/ambiguous keys visibly.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 237 |
|
| 238 |
---
|
| 239 |
|
| 240 |
+
## D030 — 2026-07-17 — Supported DOCX reinsert includes body, tables, headers and footers
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 241 |
|
| 242 |
+
Status: accepted and implemented document-fidelity decision.
|
| 243 |
|
| 244 |
+
Restore supported placeholders in existing `word/document.xml`, `word/header*.xml` and `word/footer*.xml` text nodes. Do not imply support for comments, tracked-change-only parts, footnotes/endnotes, text boxes, metadata or split placeholders unless a separate evidence-backed package adds and verifies that support.
|
| 245 |
|
| 246 |
---
|
| 247 |
|
| 248 |
+
## D021 — 2026-06-14 — Unified side-by-side review is the target review surface
|
|
|
|
|
|
|
|
|
|
|
|
|
| 249 |
|
| 250 |
+
Status: accepted and still materially binding product/UX decision.
|
|
|
|
|
|
|
| 251 |
|
| 252 |
+
The main review experience centers on source text/document versus processed/checked text. Highlights are a visual aid, not an alternate mutation authority. The authoritative review table/state remains the fallback/source of truth, and future review work should avoid proliferating duplicate helper panels.
|
| 253 |
|
| 254 |
+
This direction is implemented within the later D041/D043 Premium staged workspace.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 255 |
|
| 256 |
---
|
| 257 |
|
| 258 |
+
# Current decision discipline
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 259 |
|
| 260 |
+
New decisions belong here only when they remain materially binding on current/future work.
|
| 261 |
|
| 262 |
+
Do not copy workpackage status, test runs or implementation narrative into this file.
|
| 263 |
|
| 264 |
+
If a decision is superseded, mark the superseding decision explicitly rather than leaving two competing current rules.
|
PROJECT_PROMPT.md
CHANGED
|
@@ -1,12 +1,12 @@
|
|
| 1 |
# SolidPrivacy Scrub — Project Prompt / Worker Instructions
|
| 2 |
|
| 3 |
-
This file
|
| 4 |
|
| 5 |
-
For ChatGPT Project Instructions, use the shorter bootstrap prompt in `PROJECT_PROMPT_SHORT.md`. The short prompt points workers back to this file and the other
|
| 6 |
|
| 7 |
---
|
| 8 |
|
| 9 |
-
## Repository
|
| 10 |
|
| 11 |
Work only in:
|
| 12 |
|
|
@@ -14,267 +14,298 @@ Work only in:
|
|
| 14 |
solidprivacy-nl/scrub
|
| 15 |
```
|
| 16 |
|
| 17 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
|
| 19 |
---
|
| 20 |
|
| 21 |
-
##
|
| 22 |
|
| 23 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
-
|
|
|
|
|
|
|
| 26 |
|
| 27 |
1. `PROJECT_PROMPT.md`
|
| 28 |
2. `ROADMAP.md`
|
| 29 |
3. `WORKPACKAGES.md`
|
| 30 |
4. `CHANGELOG.md`
|
| 31 |
|
| 32 |
-
Then read
|
| 33 |
|
| 34 |
-
- `RISK_REGISTER.md`
|
| 35 |
-
- `DECISION_LOG.md`
|
| 36 |
-
- `RELEASE_NOTES.md`
|
| 37 |
-
- `STATUS_MONITORING_RUNBOOK.md`
|
| 38 |
-
-
|
|
|
|
| 39 |
|
| 40 |
-
Use
|
| 41 |
|
| 42 |
-
- `PROJECT_PROMPT.md`
|
| 43 |
-
- `
|
| 44 |
-
- `
|
| 45 |
-
- `
|
| 46 |
-
- `
|
| 47 |
-
- `
|
| 48 |
-
- `
|
| 49 |
-
- `DECISION_LOG.md` = accepted strategic/product/architecture decisions.
|
| 50 |
|
| 51 |
-
|
| 52 |
|
| 53 |
---
|
| 54 |
|
| 55 |
-
##
|
| 56 |
|
| 57 |
-
|
| 58 |
|
| 59 |
```text
|
| 60 |
-
|
| 61 |
-
governance_release_assurance
|
| 62 |
```
|
| 63 |
|
| 64 |
-
|
| 65 |
|
| 66 |
-
|
| 67 |
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
|
| 72 |
-
|
| 73 |
-
2. `control/SCRUB_RELEASE_ASSURANCE_CONTRACT_V1.md`
|
| 74 |
|
| 75 |
-
|
| 76 |
|
| 77 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 78 |
|
| 79 |
-
|
| 80 |
|
| 81 |
-
|
| 82 |
-
2. Scrub Key leakage or misuse;
|
| 83 |
-
3. hidden document content and metadata;
|
| 84 |
-
4. cloud/demo trust gap versus local-first promise;
|
| 85 |
-
5. placeholder corruption during AI roundtrip;
|
| 86 |
-
6. UI review limitations that prevent reliable human review.
|
| 87 |
|
| 88 |
-
|
| 89 |
|
| 90 |
-
|
| 91 |
-
2. Check dependencies in `WORKPACKAGES.md`.
|
| 92 |
-
3. Check `RISK_REGISTER.md` when the work touches detection, Scrub Key, export, document parsing, UI review or deployment.
|
| 93 |
-
4. Check `DECISION_LOG.md` when the work touches strategy, architecture or explicit product boundaries.
|
| 94 |
-
5. Prefer helper modules and tests before UI changes.
|
| 95 |
-
6. Avoid parallel edits to the same UI patch area.
|
| 96 |
-
7. Add or update tests where meaningful.
|
| 97 |
-
8. Update `CHANGELOG.md` for implementation history.
|
| 98 |
-
9. Update `RELEASE_NOTES.md` for user-visible product changes.
|
| 99 |
-
10. Update `ROADMAP.md` only when strategy, phase status, risk priority or sequence changes.
|
| 100 |
-
11. Update `WORKPACKAGES.md` when execution status, dependencies or next workpackages change.
|
| 101 |
-
12. End with a handover summary.
|
| 102 |
-
13. Write the handover summary to `handover/workpackages/`.
|
| 103 |
|
| 104 |
---
|
| 105 |
|
| 106 |
-
##
|
| 107 |
|
| 108 |
-
|
| 109 |
|
| 110 |
-
|
| 111 |
-
|
| 112 |
-
|
| 113 |
-
|
| 114 |
-
- benchmark data design;
|
| 115 |
-
- risk reviews;
|
| 116 |
-
- non-UI architecture work.
|
| 117 |
-
|
| 118 |
-
Do not do in parallel without explicit coordination:
|
| 119 |
-
|
| 120 |
-
- `presidio_streamlit.py`;
|
| 121 |
-
- `fix_streamlit_nested_expanders.py`;
|
| 122 |
-
- any additional Streamlit patch file that edits the same UI flow;
|
| 123 |
-
- review table UI flow;
|
| 124 |
-
- export/download UI flow;
|
| 125 |
-
- shared workflow state;
|
| 126 |
-
- Docker/runtime startup patch order.
|
| 127 |
-
|
| 128 |
-
When in doubt, keep UI integration sequential.
|
| 129 |
|
| 130 |
-
-
|
| 131 |
|
| 132 |
-
##
|
| 133 |
|
| 134 |
-
|
| 135 |
|
| 136 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 137 |
|
| 138 |
-
|
| 139 |
-
2. GitHub to Hugging Face sync status for the relevant commit.
|
| 140 |
-
3. Failed-job logs when Actions are red.
|
| 141 |
-
4. Whether a fix workpackage is needed before asking for app verification.
|
| 142 |
|
| 143 |
-
|
| 144 |
|
| 145 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 146 |
|
| 147 |
-
|
| 148 |
-
- missing permissions or inaccessible logs;
|
| 149 |
-
- subjective UX confirmation;
|
| 150 |
-
- explicit approval for gated workpackages.
|
| 151 |
|
| 152 |
-
|
| 153 |
|
| 154 |
---
|
| 155 |
|
| 156 |
-
##
|
| 157 |
|
| 158 |
-
|
| 159 |
|
| 160 |
-
|
| 161 |
|
| 162 |
-
|
| 163 |
-
|
| 164 |
-
|
| 165 |
-
|
| 166 |
-
|
| 167 |
-
|
| 168 |
-
|
| 169 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 170 |
|
| 171 |
-
|
| 172 |
|
| 173 |
-
|
| 174 |
|
| 175 |
---
|
| 176 |
|
| 177 |
-
##
|
| 178 |
|
| 179 |
-
|
| 180 |
|
| 181 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 182 |
|
| 183 |
-
|
| 184 |
-
- workpackage title;
|
| 185 |
-
- status;
|
| 186 |
-
- files added;
|
| 187 |
-
- files changed;
|
| 188 |
-
- tests added/updated;
|
| 189 |
-
- validation status;
|
| 190 |
-
- GitHub Actions status if known;
|
| 191 |
-
- Hugging Face sync status if known;
|
| 192 |
-
- app verification status if known;
|
| 193 |
-
- remaining risks or follow-up actions;
|
| 194 |
-
- next recommended step.
|
| 195 |
|
| 196 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 197 |
|
| 198 |
-
|
| 199 |
-
handover/workpackages/
|
| 200 |
-
```
|
| 201 |
|
| 202 |
-
|
| 203 |
|
| 204 |
-
|
| 205 |
-
handover/workpackages/YYYYMMDD_HHMM_<workpackage_slug>.md
|
| 206 |
-
```
|
| 207 |
|
| 208 |
-
|
| 209 |
|
| 210 |
-
|
| 211 |
-
# Handover — <Workpackage title>
|
| 212 |
|
| 213 |
-
|
| 214 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 215 |
|
| 216 |
-
|
| 217 |
|
| 218 |
-
|
| 219 |
|
| 220 |
-
|
| 221 |
|
| 222 |
-
-
|
| 223 |
|
| 224 |
-
##
|
| 225 |
|
| 226 |
-
|
| 227 |
|
| 228 |
-
|
| 229 |
|
| 230 |
-
|
| 231 |
|
| 232 |
-
|
| 233 |
|
| 234 |
-
|
| 235 |
-
- Hugging Face sync: <green/red/unknown>
|
| 236 |
-
- App verification: <confirmed/pending/not applicable>
|
| 237 |
|
| 238 |
-
|
| 239 |
|
| 240 |
-
|
| 241 |
|
| 242 |
-
|
| 243 |
|
| 244 |
-
|
|
|
|
| 245 |
```
|
| 246 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 247 |
---
|
| 248 |
|
| 249 |
-
## Safety and
|
| 250 |
|
| 251 |
-
- Do not
|
| 252 |
-
- Do not silently change export semantics.
|
| 253 |
-
-
|
| 254 |
-
-
|
|
|
|
|
|
|
| 255 |
- Do not store secrets, tokens or real personal data.
|
| 256 |
-
-
|
| 257 |
-
-
|
| 258 |
-
-
|
| 259 |
-
- Treat the Scrub Key as sensitive data because it can re-identify scrubbed content.
|
| 260 |
-
- Treat false negatives as a product-critical risk, not as a cosmetic bug.
|
| 261 |
|
| 262 |
-
|
| 263 |
|
| 264 |
-
|
| 265 |
|
| 266 |
-
|
| 267 |
|
| 268 |
-
|
| 269 |
-
|
| 270 |
-
|
|
|
|
|
|
|
| 271 |
|
| 272 |
-
|
| 273 |
|
| 274 |
-
|
|
|
|
|
|
|
| 275 |
|
| 276 |
-
|
|
|
|
|
|
|
| 277 |
|
| 278 |
```text
|
| 279 |
-
|
| 280 |
```
|
|
|
|
|
|
|
|
|
| 1 |
# SolidPrivacy Scrub — Project Prompt / Worker Instructions
|
| 2 |
|
| 3 |
+
This file contains the stable worker instructions for SolidPrivacy Scrub.
|
| 4 |
|
| 5 |
+
For ChatGPT Project Instructions, use the shorter bootstrap prompt in `PROJECT_PROMPT_SHORT.md`. The short prompt points workers back to this file and the other canonical control files.
|
| 6 |
|
| 7 |
---
|
| 8 |
|
| 9 |
+
## 1. Repository scope
|
| 10 |
|
| 11 |
Work only in:
|
| 12 |
|
|
|
|
| 14 |
solidprivacy-nl/scrub
|
| 15 |
```
|
| 16 |
|
| 17 |
+
Use `solidprivacy-nl/solidprivacy` only for explicitly shared SolidPrivacy privacy/AI workflows, regulatory knowledge, orchestration, evaluation frameworks and reusable privacy components when the workpackage calls for it.
|
| 18 |
+
|
| 19 |
+
Do not modify unrelated repositories.
|
| 20 |
+
|
| 21 |
+
GitHub is the source of truth.
|
| 22 |
|
| 23 |
---
|
| 24 |
|
| 25 |
+
## 2. Engineering constitution
|
| 26 |
|
| 27 |
+
The canonical expanded engineering doctrine is the SolidPrivacy **Execution & Engineering Constitution**.
|
| 28 |
+
|
| 29 |
+
Binding principles include:
|
| 30 |
+
|
| 31 |
+
- business outcome first;
|
| 32 |
+
- smallest complete solution;
|
| 33 |
+
- solid but simple;
|
| 34 |
+
- no overengineering;
|
| 35 |
+
- first-principles reasoning;
|
| 36 |
+
- prefer proven/native solutions;
|
| 37 |
+
- critique and refine material work;
|
| 38 |
+
- keep executing once the objective and constraints are clear;
|
| 39 |
+
- progress reporting must not replace execution;
|
| 40 |
+
- solve root causes and keep one source of truth;
|
| 41 |
+
- verify behavior, security and maintainability;
|
| 42 |
+
- remove obsolete/conflicting active paths;
|
| 43 |
+
- Done means outcome + implementation + verification + cleanup + documentation alignment.
|
| 44 |
+
|
| 45 |
+
Project requirements may strengthen this doctrine, not weaken it.
|
| 46 |
+
|
| 47 |
+
---
|
| 48 |
|
| 49 |
+
## 3. Required start sequence
|
| 50 |
+
|
| 51 |
+
At the start of every work session, read in this exact order:
|
| 52 |
|
| 53 |
1. `PROJECT_PROMPT.md`
|
| 54 |
2. `ROADMAP.md`
|
| 55 |
3. `WORKPACKAGES.md`
|
| 56 |
4. `CHANGELOG.md`
|
| 57 |
|
| 58 |
+
Then read when relevant:
|
| 59 |
|
| 60 |
+
- `RISK_REGISTER.md` — current product/privacy/security risks;
|
| 61 |
+
- `DECISION_LOG.md` — accepted strategic and architecture decisions;
|
| 62 |
+
- `RELEASE_NOTES.md` — user-visible product changes;
|
| 63 |
+
- `STATUS_MONITORING_RUNBOOK.md` — Actions/Hugging Face monitoring;
|
| 64 |
+
- `control/PROJECT_GOVERNANCE_BOOTSTRAP.md` and `control/SCRUB_RELEASE_ASSURANCE_CONTRACT_V1.md` for consequential work;
|
| 65 |
+
- relevant specifications such as Scrub Key, reinsert, document-hygiene or domain-profile contracts.
|
| 66 |
|
| 67 |
+
Use the canonical files as follows:
|
| 68 |
|
| 69 |
+
- `PROJECT_PROMPT.md` — stable worker/governance/safety rules;
|
| 70 |
+
- `ROADMAP.md` — strategic product direction and stage order;
|
| 71 |
+
- `WORKPACKAGES.md` — one current executable queue and gates;
|
| 72 |
+
- `CHANGELOG.md` — historical implementation record;
|
| 73 |
+
- `RELEASE_NOTES.md` — user-facing release summary;
|
| 74 |
+
- `RISK_REGISTER.md` — current risk picture;
|
| 75 |
+
- `DECISION_LOG.md` — accepted current decisions.
|
|
|
|
| 76 |
|
| 77 |
+
If these files conflict with current source/runtime evidence, treat the conflict as a defect. Do not silently choose whichever statement is convenient.
|
| 78 |
|
| 79 |
---
|
| 80 |
|
| 81 |
+
## 4. Current product direction
|
| 82 |
|
| 83 |
+
Core product workflow:
|
| 84 |
|
| 85 |
```text
|
| 86 |
+
Scrub → Review → Scrub Key → AI / external use → Reinsert → Export → Audit
|
|
|
|
| 87 |
```
|
| 88 |
|
| 89 |
+
Scrub is a reviewed privacy-processing / pseudonymisation product for confidential professional documents. Legal and Zorg are the priority validation domains. Human review remains required; the product must preserve legal and clinical meaning rather than blindly mask context.
|
| 90 |
|
| 91 |
+
### Current strategic stage
|
| 92 |
|
| 93 |
+
```text
|
| 94 |
+
Stage 1 — Repository Convergence
|
| 95 |
+
```
|
| 96 |
|
| 97 |
+
Normal new feature development is paused until the repository has one verified clean current truth.
|
|
|
|
| 98 |
|
| 99 |
+
Current strategic sequence:
|
| 100 |
|
| 101 |
+
```text
|
| 102 |
+
Repository Convergence
|
| 103 |
+
→ Scrub Private Application
|
| 104 |
+
→ Private Service
|
| 105 |
+
→ External Product & Service Assurance
|
| 106 |
+
→ Pilot
|
| 107 |
+
```
|
| 108 |
|
| 109 |
+
After convergence, `main` becomes the active Scrub Private development line.
|
| 110 |
|
| 111 |
+
Hugging Face is a synthetic/approved-test **application validation environment**, not the final confidential-production trust environment.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 112 |
|
| 113 |
+
Local/offline functionality and prior installer work remain recoverable/deferred. They are not the active delivery line and must not be resumed merely because historical plans mention them.
|
| 114 |
|
| 115 |
+
Do not build a new Evidence Framework. Reuse and reconcile the existing synthetic corpus, benchmark, gap-triage, E2E, Scrub Key, document and AppTest evidence systems.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 116 |
|
| 117 |
---
|
| 118 |
|
| 119 |
+
## 5. Implementation versus independent release assurance
|
| 120 |
|
| 121 |
+
Consequential work uses the canonical two-role model:
|
| 122 |
|
| 123 |
+
```text
|
| 124 |
+
implementation_operations
|
| 125 |
+
governance_release_assurance
|
| 126 |
+
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 127 |
|
| 128 |
+
The user remains the coordinator-facing principal.
|
| 129 |
|
| 130 |
+
### Implementation role
|
| 131 |
|
| 132 |
+
`implementation_operations`:
|
| 133 |
|
| 134 |
+
- implements the workpackage;
|
| 135 |
+
- creates an identifiable exact candidate;
|
| 136 |
+
- adds/updates meaningful tests;
|
| 137 |
+
- records implementation evidence and handover;
|
| 138 |
+
- may only claim an implementation state such as `IMPLEMENTATION_IN_PROGRESS`, `IMPLEMENTATION_BLOCKED` or `RELEASE_CANDIDATE_READY`;
|
| 139 |
+
- may not certify its own consequential candidate.
|
| 140 |
|
| 141 |
+
### Assurance role
|
|
|
|
|
|
|
|
|
|
| 142 |
|
| 143 |
+
`governance_release_assurance`:
|
| 144 |
|
| 145 |
+
- works independently from a fresh worker/session;
|
| 146 |
+
- reconstructs the requested outcome and exact candidate;
|
| 147 |
+
- may not silently repair the candidate;
|
| 148 |
+
- before its initial verdict may inspect authoritative criteria, exact source/diff and raw machine/deployment evidence, but not implementation handovers, self-assessments or conclusions;
|
| 149 |
+
- records exactly `PASS`, `FAIL` or `INDETERMINATE`;
|
| 150 |
+
- may inspect the implementation handover only after the initial verdict for disclosure/administrative completeness.
|
| 151 |
|
| 152 |
+
A repaired candidate receives a fresh independent assurance pass.
|
|
|
|
|
|
|
|
|
|
| 153 |
|
| 154 |
+
Consequential implementation and verification are separate workpackages/workers.
|
| 155 |
|
| 156 |
---
|
| 157 |
|
| 158 |
+
## 6. Workpackage discipline
|
| 159 |
|
| 160 |
+
Work in small, testable, coherent root-cause packages.
|
| 161 |
|
| 162 |
+
For each workpackage:
|
| 163 |
|
| 164 |
+
1. confirm title, role and scope;
|
| 165 |
+
2. check dependencies/current queue in `WORKPACKAGES.md`;
|
| 166 |
+
3. check `RISK_REGISTER.md` for affected product/privacy risks;
|
| 167 |
+
4. check `DECISION_LOG.md` for relevant boundaries;
|
| 168 |
+
5. reconstruct current source behavior before changing it;
|
| 169 |
+
6. prefer pure helpers/tests before risky UI integration when new logic is genuinely required;
|
| 170 |
+
7. add/update meaningful tests;
|
| 171 |
+
8. update `CHANGELOG.md` for implementation history;
|
| 172 |
+
9. update `RELEASE_NOTES.md` only for user-visible behavior changes;
|
| 173 |
+
10. update `ROADMAP.md` only when strategy/stage order changes;
|
| 174 |
+
11. update `WORKPACKAGES.md` when current execution status/next work changes;
|
| 175 |
+
12. independently verify consequential candidates;
|
| 176 |
+
13. end with a handover and write it under `handover/workpackages/`.
|
| 177 |
|
| 178 |
+
Do not create abstractions, services, configuration layers or compatibility mechanisms without a concrete current requirement.
|
| 179 |
|
| 180 |
+
During Repository Convergence, no cleanup/refactor is justified by aesthetics alone. Require evidence of duplicate/contradictory/dead behavior, privacy/security risk, obsolete compatibility, meaningful maintenance burden, runtime instability or another concrete current problem.
|
| 181 |
|
| 182 |
---
|
| 183 |
|
| 184 |
+
## 7. Parallelization and shared-surface rule
|
| 185 |
|
| 186 |
+
Safe to parallelize only when changes are genuinely independent, such as:
|
| 187 |
|
| 188 |
+
- separate helper modules;
|
| 189 |
+
- independent test/documentation work;
|
| 190 |
+
- benchmark-data analysis;
|
| 191 |
+
- risk review;
|
| 192 |
+
- non-overlapping architecture analysis.
|
| 193 |
|
| 194 |
+
Do not make parallel uncoordinated edits to shared state/runtime/UI surfaces, especially:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 195 |
|
| 196 |
+
- `presidio_streamlit.py`;
|
| 197 |
+
- Streamlit startup/patch behavior;
|
| 198 |
+
- review table/direct-correction flow;
|
| 199 |
+
- export/download flow;
|
| 200 |
+
- Scrub Key/reinsert shared paths;
|
| 201 |
+
- processing-generation state;
|
| 202 |
+
- Docker/runtime startup order.
|
| 203 |
|
| 204 |
+
When in doubt, keep shared-surface integration sequential.
|
|
|
|
|
|
|
| 205 |
|
| 206 |
+
---
|
| 207 |
|
| 208 |
+
## 8. Testing, deployment and monitoring
|
|
|
|
|
|
|
| 209 |
|
| 210 |
+
Implementation workers must self-check where connector permissions allow.
|
| 211 |
|
| 212 |
+
After meaningful implementation:
|
|
|
|
| 213 |
|
| 214 |
+
1. verify relevant local/pure tests where execution is available;
|
| 215 |
+
2. verify the full GitHub Actions suite for the exact candidate where required;
|
| 216 |
+
3. inspect failed-job logs rather than inferring causes;
|
| 217 |
+
4. after authorized merge, verify exact-main Actions;
|
| 218 |
+
5. verify GitHub→Hugging Face synchronization when applicable;
|
| 219 |
+
6. request human/live app verification only when UI/runtime behavior changed or subjective UX confirmation is required.
|
| 220 |
|
| 221 |
+
Use `STATUS_MONITORING_RUNBOOK.md` for the monitoring procedure.
|
| 222 |
|
| 223 |
+
Do not claim functional success merely because code appears correct.
|
| 224 |
|
| 225 |
+
For UI behavior, automated tests are necessary but may not replace required deployed/live verification.
|
| 226 |
|
| 227 |
+
---
|
| 228 |
|
| 229 |
+
## 9. Documentation discipline
|
| 230 |
|
| 231 |
+
`CHANGELOG.md` is historical implementation provenance. Do not rewrite history simply because old entries describe obsolete states.
|
| 232 |
|
| 233 |
+
`ROADMAP.md` must remain strategic and concise.
|
| 234 |
|
| 235 |
+
`WORKPACKAGES.md` must contain one real current executable queue. Do not stack multiple historical “current override” sections.
|
| 236 |
|
| 237 |
+
`DECISION_LOG.md` records accepted decisions; `RISK_REGISTER.md` records current risk; do not create competing permanent status documents.
|
| 238 |
|
| 239 |
+
Temporary audit/debt ledgers may exist during a workpackage, but they are execution evidence, not new canonical authorities.
|
|
|
|
|
|
|
| 240 |
|
| 241 |
+
---
|
| 242 |
|
| 243 |
+
## 10. Handover discipline
|
| 244 |
|
| 245 |
+
Every worker must write a handover to:
|
| 246 |
|
| 247 |
+
```text
|
| 248 |
+
handover/workpackages/YYYYMMDD_HHMM_<workpackage_slug>.md
|
| 249 |
```
|
| 250 |
|
| 251 |
+
The handover must explicitly state:
|
| 252 |
+
|
| 253 |
+
- repository worked in;
|
| 254 |
+
- workpackage title;
|
| 255 |
+
- role;
|
| 256 |
+
- status;
|
| 257 |
+
- exact candidate/baseline identity where relevant;
|
| 258 |
+
- files added;
|
| 259 |
+
- files changed;
|
| 260 |
+
- tests added/updated;
|
| 261 |
+
- validation status;
|
| 262 |
+
- GitHub Actions status;
|
| 263 |
+
- Hugging Face sync status;
|
| 264 |
+
- app verification status;
|
| 265 |
+
- remaining risks/blockers;
|
| 266 |
+
- next recommended step.
|
| 267 |
+
|
| 268 |
+
Do not label a consequential package `completed` merely because implementation code exists. Distinguish implementation readiness, assurance result, merge/action and independently confirmed outcome.
|
| 269 |
+
|
| 270 |
---
|
| 271 |
|
| 272 |
+
## 11. Safety and privacy rules
|
| 273 |
|
| 274 |
+
- Do not weaken human review or privacy controls.
|
| 275 |
+
- Do not silently change export semantics, filenames, MIME or Scrub Key semantics.
|
| 276 |
+
- Treat false negatives as product-critical.
|
| 277 |
+
- Treat the Scrub Key as sensitive re-identification material.
|
| 278 |
+
- Do not introduce external/cloud document processing unless explicitly approved for the relevant product line.
|
| 279 |
+
- Use synthetic data only for repository tests and public/reference-environment verification.
|
| 280 |
- Do not store secrets, tokens or real personal data.
|
| 281 |
+
- Preserve legal and clinical meaning.
|
| 282 |
+
- Do not claim perfect anonymisation, perfect recall or production safety from synthetic benchmarks.
|
| 283 |
+
- Fail visibly/closed where binding/state validation requires it.
|
|
|
|
|
|
|
| 284 |
|
| 285 |
+
### Scrub Private boundary
|
| 286 |
|
| 287 |
+
For the future Private line, customer document content includes uploaded/extracted text, detected values, replacement mappings, processed/restored documents and Scrub Keys.
|
| 288 |
|
| 289 |
+
Target policy:
|
| 290 |
|
| 291 |
+
- no intentional persistent server-side customer document content;
|
| 292 |
+
- no content-bearing ordinary application logs;
|
| 293 |
+
- no document-content backups;
|
| 294 |
+
- no third-party document-processing egress;
|
| 295 |
+
- minimal non-document control-plane metadata only where a real service requirement justifies persistence.
|
| 296 |
|
| 297 |
+
Production infrastructure/security claims must be separately verified; Hugging Face application tests do not prove provider-level retention properties.
|
| 298 |
|
| 299 |
+
---
|
| 300 |
+
|
| 301 |
+
## 12. Current execution rule
|
| 302 |
|
| 303 |
+
Follow `WORKPACKAGES.md`.
|
| 304 |
+
|
| 305 |
+
During Repository Convergence:
|
| 306 |
|
| 307 |
```text
|
| 308 |
+
Preserve → reconstruct → remove only proven debt → reconcile → verify → freeze
|
| 309 |
```
|
| 310 |
+
|
| 311 |
+
Do not resume historical Premium stage packages, installer work, VPS infrastructure, pilots or scale features unless the current roadmap/workpackage gate explicitly releases them.
|
PROJECT_PROMPT_SHORT.md
CHANGED
|
@@ -12,77 +12,75 @@ Work only in:
|
|
| 12 |
solidprivacy-nl/scrub
|
| 13 |
```
|
| 14 |
|
| 15 |
-
|
|
|
|
|
|
|
| 16 |
|
| 17 |
1. `PROJECT_PROMPT.md`
|
| 18 |
2. `ROADMAP.md`
|
| 19 |
3. `WORKPACKAGES.md`
|
| 20 |
4. `CHANGELOG.md`
|
| 21 |
|
| 22 |
-
|
|
|
|
|
|
|
| 23 |
|
| 24 |
1. `control/PROJECT_GOVERNANCE_BOOTSTRAP.md`
|
| 25 |
2. `control/SCRUB_RELEASE_ASSURANCE_CONTRACT_V1.md`
|
| 26 |
|
| 27 |
-
Implementation may only prepare
|
| 28 |
|
| 29 |
-
Follow the current
|
| 30 |
|
| 31 |
-
|
| 32 |
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
|
|
|
|
|
|
|
|
|
| 37 |
|
| 38 |
-
|
| 39 |
|
| 40 |
-
|
| 41 |
-
- update `ROADMAP.md` only when strategy or phase order changes;
|
| 42 |
-
- update `WORKPACKAGES.md` when execution status or next workpackages change;
|
| 43 |
-
- verify GitHub Actions tests;
|
| 44 |
-
- verify GitHub to Hugging Face sync;
|
| 45 |
-
- ask for app verification when UI behavior changed.
|
| 46 |
|
| 47 |
-
|
| 48 |
|
| 49 |
-
``
|
| 50 |
-
|
| 51 |
-
|
| 52 |
|
| 53 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 54 |
|
| 55 |
```text
|
| 56 |
handover/workpackages/YYYYMMDD_HHMM_<workpackage_slug>.md
|
| 57 |
```
|
| 58 |
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
- repository worked in;
|
| 62 |
-
- workpackage title;
|
| 63 |
-
- status;
|
| 64 |
-
- files added/changed;
|
| 65 |
-
- tests;
|
| 66 |
-
- validation status;
|
| 67 |
-
- GitHub Actions status;
|
| 68 |
-
- Hugging Face sync status;
|
| 69 |
-
- app verification status;
|
| 70 |
-
- remaining risks;
|
| 71 |
-
- next recommended step.
|
| 72 |
|
| 73 |
-
Safety
|
| 74 |
|
| 75 |
-
-
|
| 76 |
-
-
|
| 77 |
-
-
|
| 78 |
-
-
|
| 79 |
-
-
|
| 80 |
-
-
|
|
|
|
|
|
|
|
|
|
| 81 |
|
| 82 |
-
|
| 83 |
|
| 84 |
```text
|
| 85 |
-
Scrub → Review → Scrub Key → AI → Reinsert → Export → Audit
|
| 86 |
```
|
| 87 |
-
|
| 88 |
-
Immediate line: v12 Review UX, then v13 Scrub Key / Reinsert.
|
|
|
|
| 12 |
solidprivacy-nl/scrub
|
| 13 |
```
|
| 14 |
|
| 15 |
+
Use `solidprivacy-nl/solidprivacy` only for explicitly shared SolidPrivacy privacy/AI workflows/components when the current workpackage calls for it. Do not modify unrelated repositories.
|
| 16 |
+
|
| 17 |
+
GitHub is the source of truth. At the start of every work session, read in order:
|
| 18 |
|
| 19 |
1. `PROJECT_PROMPT.md`
|
| 20 |
2. `ROADMAP.md`
|
| 21 |
3. `WORKPACKAGES.md`
|
| 22 |
4. `CHANGELOG.md`
|
| 23 |
|
| 24 |
+
Follow the SolidPrivacy Execution & Engineering Constitution: business outcome first, smallest complete/proven solution, solid but simple, no overengineering, one source of truth, verify behavior, remove obsolete/conflicting active paths, and do not call work Done until implementation, verification, cleanup and documentation agree.
|
| 25 |
+
|
| 26 |
+
For consequential work, also read:
|
| 27 |
|
| 28 |
1. `control/PROJECT_GOVERNANCE_BOOTSTRAP.md`
|
| 29 |
2. `control/SCRUB_RELEASE_ASSURANCE_CONTRACT_V1.md`
|
| 30 |
|
| 31 |
+
Implementation may only prepare an exact release candidate. A separate fresh `governance_release_assurance` worker/session independently reconstructs and verifies it before reading implementation conclusions, then records `PASS`, `FAIL` or `INDETERMINATE`. Governance may not silently repair the candidate.
|
| 32 |
|
| 33 |
+
Follow the one current execution queue in `WORKPACKAGES.md`. Do not execute historical queues merely because old documentation/issues mention them.
|
| 34 |
|
| 35 |
+
Current strategic sequence:
|
| 36 |
|
| 37 |
+
```text
|
| 38 |
+
Repository Convergence
|
| 39 |
+
→ Scrub Private Application
|
| 40 |
+
→ Private Service
|
| 41 |
+
→ External Product & Service Assurance
|
| 42 |
+
→ Pilot
|
| 43 |
+
```
|
| 44 |
|
| 45 |
+
Normal new feature work is paused during Repository Convergence. Preserve current capability, remove only proven debt, align current truth, verify, and freeze one exact clean-baseline SHA.
|
| 46 |
|
| 47 |
+
Do not build a new Evidence Framework. Reuse/reconcile the existing synthetic corpora, benchmarks, E2E, Scrub Key, document and AppTest evidence.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 48 |
|
| 49 |
+
Hugging Face is a synthetic/approved-test application-validation surface, not the final confidential-production trust environment. Local/offline functionality is preserved/deferred, not the active delivery line.
|
| 50 |
|
| 51 |
+
Use small coherent workpackages. Do not refactor for aesthetics. Avoid parallel edits to shared UI/state/runtime surfaces, especially `presidio_streamlit.py`, review, export, Scrub Key/reinsert, processing state and Docker/startup behavior.
|
| 52 |
+
|
| 53 |
+
After meaningful changes:
|
| 54 |
|
| 55 |
+
- update `CHANGELOG.md`;
|
| 56 |
+
- update `ROADMAP.md` only for strategy/stage changes;
|
| 57 |
+
- update `WORKPACKAGES.md` when current execution changes;
|
| 58 |
+
- verify GitHub Actions;
|
| 59 |
+
- verify GitHub→Hugging Face sync when applicable;
|
| 60 |
+
- request live app verification when visible/runtime behavior changed.
|
| 61 |
+
|
| 62 |
+
Every worker must write:
|
| 63 |
|
| 64 |
```text
|
| 65 |
handover/workpackages/YYYYMMDD_HHMM_<workpackage_slug>.md
|
| 66 |
```
|
| 67 |
|
| 68 |
+
including repository, workpackage, role/status, exact candidate where relevant, files, tests, validation, Actions, HF sync, app verification, risks/blockers and next step.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 69 |
|
| 70 |
+
Safety:
|
| 71 |
|
| 72 |
+
- do not weaken privacy or human-review controls;
|
| 73 |
+
- do not silently change export or Scrub Key semantics;
|
| 74 |
+
- false negatives are product-critical;
|
| 75 |
+
- Scrub Keys are sensitive re-identification material;
|
| 76 |
+
- use synthetic data only;
|
| 77 |
+
- do not store secrets/tokens/real personal data;
|
| 78 |
+
- preserve legal and clinical meaning;
|
| 79 |
+
- do not introduce external/cloud document processing unless explicitly approved for the relevant product line;
|
| 80 |
+
- do not claim perfect anonymisation/recall or production safety from synthetic tests.
|
| 81 |
|
| 82 |
+
Core workflow:
|
| 83 |
|
| 84 |
```text
|
| 85 |
+
Scrub → Review → Scrub Key → AI / external use → Reinsert → Export → Audit
|
| 86 |
```
|
|
|
|
|
|
REPOSITORY_CONVERGENCE_DEBT_LEDGER.md
ADDED
|
@@ -0,0 +1,89 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# SolidPrivacy Scrub — Repository Convergence Debt Ledger
|
| 2 |
+
|
| 3 |
+
> **TEMPORARY EXECUTION ARTIFACT — NON-AUTHORITATIVE AFTER CONVERGENCE**
|
| 4 |
+
>
|
| 5 |
+
> This ledger supports `SCRUB-WP_REPOSITORY_CONVERGENCE_BOOTSTRAP` and later convergence work. It is not a new permanent source of product truth. At convergence closeout, accepted decisions belong in `DECISION_LOG.md`, current risks in `RISK_REGISTER.md`, execution state in `WORKPACKAGES.md`, and implementation history in `CHANGELOG.md`.
|
| 6 |
+
|
| 7 |
+
Date: 2026-09-04
|
| 8 |
+
Repository: `solidprivacy-nl/scrub`
|
| 9 |
+
Pre-convergence baseline: `54c73e0ebf5a3a3ed7039a50596fb57694add3cd`
|
| 10 |
+
|
| 11 |
+
## Classification model
|
| 12 |
+
|
| 13 |
+
- `CANONICAL` — supported current behavior; retain, test and document accurately.
|
| 14 |
+
- `RECONCILE` — valid responsibility whose implementation, tests, configuration, documentation or status does not match current reality.
|
| 15 |
+
- `RETIRE` — superseded, dead, contradictory or unnecessarily active path; remove only after evidence shows it is safe to do so.
|
| 16 |
+
- `VARIANT-SPECIFIC` — potentially valid functionality whose requirements conflict with Scrub Private; preserve recoverability through Git but exclude from the Private line unless explicitly re-approved.
|
| 17 |
+
|
| 18 |
+
Classification applies to capabilities/contracts/runtime paths, not mechanically to whole files.
|
| 19 |
+
|
| 20 |
+
## Current capability/path ledger
|
| 21 |
+
|
| 22 |
+
Rows describing documentary debt explicitly distinguish the pre-bootstrap finding from the state already resolved in this bootstrap candidate. `RECONCILE — RESOLVED IN BOOTSTRAP CANDIDATE` means no further bootstrap cleanup is authorized for that item unless fresh assurance finds a concrete defect. Unresolved technical, issue-state and validation-hierarchy findings remain future work.
|
| 23 |
+
|
| 24 |
+
| Capability / path | Current role and evidence | Classification | Target | Protecting evidence / tests | Risk | Smallest next action |
|
| 25 |
+
|---|---|---|---|---|---|---|
|
| 26 |
+
| Premium staged workspace and generation-bound state (`premium_core_flow_state.py`, Premium Streamlit helpers, current `presidio_streamlit.py`) | Current main contains `Anonimiseren | Terugzetten`, `Standaard | Expert`, staged `Toevoegen → Controleren → Downloaden`, generation-bound analysis/review state and fail-closed export eligibility. | `CANONICAL` | Retain as current UI/state architecture. | Premium state/AppTest/integration suites. | Rebuilding from stale workpackages could reintroduce already-fixed state defects. | Preserve; do not restart old Input/Review/Export packages without current evidence. |
|
| 27 |
+
| Human review authority, review table and direct correction | Review/include state remains authoritative; direct masking/manual additions feed normal replacement rows rather than bypassing review/export authority. | `CANONICAL` | Retain mandatory human review and authoritative review state. | Review/direct-selection/manual-row/cross-flow tests. | Privacy control could be weakened by UI simplification or alternate mutation paths. | Preserve. |
|
| 28 |
+
| Scrub Key binding and reinsert | Current architecture has bound placeholders, schema-1.1 keys, mapping digest, fail-closed wrong/mixed binding checks and controlled reinsert with explicit legacy handling. | `CANONICAL` | Retain; no redesign without evidence of a concrete flaw. | Scrub Key binding, import/export, TXT/DOCX/PDF-to-TXT reinsert and roundtrip suites. | Re-identification material is safety-critical. | Preserve. |
|
| 29 |
+
| Legal recognition / Dutch deterministic recognition | Current runtime uses deterministic Dutch recognizers plus generic NER and Legal profile behavior. Latest main includes Dutch address-span precision repair. | `CANONICAL` | Retain current behavior; future fixes remain evidence-driven. | Legal/custom-recognizer, recall, address-span and cross-profile tests. | False negatives remain product-critical. | Preserve and verify through canonical validation path. |
|
| 30 |
+
| Zorg profile, policy and recognizers | Dedicated Zorg policy/recognizers, synthetic corpus, baseline, gap triage and cross-profile evidence exist and are integrated. | `CANONICAL` | Retain; preserve clinical meaning and human review. | Care policy/corpus/recognizer/cross-profile/long-form tests. | Under-detection or clinical over-masking remains critical. | Preserve. |
|
| 31 |
+
| TXT/DOCX/text-PDF document handling and DOCX hygiene/fidelity | Supported import/export/reinsert/hygiene paths exist; PDF remains text-based with explicit limitations. | `CANONICAL` | Retain supported boundaries; do not add OCR/restored-PDF merely during cleanup. | Document tools, hygiene, Phase-6 E2E and fidelity tests. | Hidden content and unsupported-format assumptions. | Preserve. |
|
| 32 |
+
| Recognizer-backed recall benchmark (`recall_benchmark_runner.py`, `recall_benchmark_report.py`, `.github/workflows/recall-benchmark-report.yml`, `corpus/**`) | Current workflow executes corpus/runner/report tests and runs the actual recognition stack to produce JSON/Markdown diagnostic artifacts. | `CANONICAL` candidate | Treat as primary current recognizer-backed diagnostic release evidence unless later audit finds a newer equivalent. | Recall benchmark corpus/runner/report tests and workflow. | Competing benchmark generations can confuse release evidence. | Document release role and compare older scorecard/residual-risk helpers before de-authorizing anything. |
|
| 33 |
+
| Older entity-scorecard / residual-risk helper generation (`benchmark/build_entity_scorecard.py`, `benchmark/build_residual_risk_report.py`) | Useful report-only/supplied-prediction diagnostics explicitly state coverage limits and no production gate. | `RECONCILE` | Keep as supplemental diagnostic if still useful; do not present as equivalent to recognizer-backed corpus execution. | Scorecard/residual-risk tests. | Different generations may appear to be competing sources of accuracy truth. | Classify canonical vs supplemental in a narrow validation-hierarchy package; no new framework. |
|
| 34 |
+
| Phase-6 synthetic E2E validation | Exercises TXT/DOCX/PDF workflow, manual additions, Scrub Key, export/reinsert and known limitations. | `CANONICAL` | Retain as workflow-level product evidence complementary to recognizer benchmarks. | `tests/test_mvp_phase6_*` and committed validation artifacts. | Could be mistaken for complete production readiness. | Preserve limitations and human-review boundary. |
|
| 35 |
+
| Persistent replacement memory (`replacement_memory.py` + Expert UI) | Current code deliberately persists original/replacement/entity values to `/data/replacement_memory.json` when persistent storage exists and exposes save/load/clear controls. | `VARIANT-SPECIFIC` | Preserve recoverability for possible Local use; Scrub Private must not persist these mappings server-side. | Existing replacement-memory tests plus future Private no-persistence contract. | Direct conflict with Private content-plane retention boundary. | Do not change in bootstrap. After clean baseline, remove/exclude from Private in a separate consequential package. |
|
| 36 |
+
| Azure AI Language document recognition | Current Expert analyzer list still includes Azure AI Language and current helper can construct Azure recognizer clients. | `VARIANT-SPECIFIC` | Exclude from Scrub Private baseline; Private document processing stays local. | Current analyzer-selection tests; future Private no-egress tests. | Third-party document-content egress conflicts with intended service trust boundary. | Do not change in bootstrap. Remove/exclude after clean baseline. |
|
| 37 |
+
| OpenAI/Azure OpenAI synthesis operator | Current Expert operator includes `synthesize`; helper sends document-bearing prompt to external provider. | `VARIANT-SPECIFIC` | Exclude from Scrub Private baseline unless explicitly approved later. | Current operator/state tests; future Private no-egress tests. | External content processing and unnecessary trust-boundary complexity. | Do not change in bootstrap. Remove/exclude after clean baseline. |
|
| 38 |
+
| Content-bearing prompt print in `create_fake_data()` | Current helper prints the generated synthesis prompt before external call. | `RECONCILE` for current full-feature line; Private blocker | No document text/prompts in ordinary application logs. | Future narrow no-content-log regression. | Sensitive content can enter logs. | Remove with Private external-synthesis cleanup; do not build a logging framework. |
|
| 39 |
+
| Legacy startup patch script invocation in Dockerfile | Docker CMD still runs `fix_streamlit_nested_expanders.py` and `fix_streamlit_pdf_text_reinsert.py`. Current Premium/direct-source markers make both scripts exit without mutating source. | `RETIRE` candidate | Production/current runtime should not invoke obsolete no-op source mutation machinery. | Patch-script contract tests, Premium source markers, full suite and runtime startup tests. | Hidden startup complexity and future accidental mutation. | Prove both current no-op paths are unnecessary, then remove invocation/scripts in isolated package. |
|
| 40 |
+
| `fix_streamlit_nested_expanders.py` historical mutation implementation | Contains large historical patch logic but exits immediately on current Premium source. | `RETIRE` candidate | Remove after startup-path proof; Git history retains implementation. | Existing patch tests and Premium regression suites. | Dead complex code can be accidentally reactivated or mistaken for current UI authority. | Isolated technical package after bootstrap. |
|
| 41 |
+
| `fix_streamlit_pdf_text_reinsert.py` historical mutation implementation | Current direct-source marker makes script exit without mutation. | `RETIRE` candidate | Remove after startup-path proof. | Reinsert direct-source/patch tests. | Dead startup code and duplicate authority. | Isolated technical package after bootstrap. |
|
| 42 |
+
| Docker/runtime production hardening | Current Dockerfile uses Python 3.10 base, build tools, legacy startup patch calls and permissive Streamlit XSRF/CORS flags. This is not itself evidence of a current HF product defect. | `RECONCILE` / later service work | Keep convergence scoped; production immutability, least privilege, egress and secure server config belong primarily to Private Service stage. | Runtime/startup tests and later service security evidence. | Premature hardening work can derail functional convergence; leaving insecure production config later would be unacceptable. | Record only; defer major hardening until Private Service unless a current safety defect is demonstrated. |
|
| 43 |
+
| `openai` and `azure-ai-textanalytics` dependencies | Required only by current variant-specific external processing paths. | `RECONCILE` | Remove when the corresponding Private-incompatible runtime paths are removed; do not leave unused dependencies. | Full dependency/import/runtime suite. | Unused attack/maintenance surface. | Defer to the same Private cleanup package as external processing. |
|
| 44 |
+
| `PROJECT_PROMPT.md` product-direction section | Pre-bootstrap finding: product direction still encoded local-first sequencing. Current bootstrap candidate now preserves stable governance/safety rules while routing execution through Repository Convergence → Scrub Private Application → Private Service → External Assurance → Pilot; HF is synthetic/approved-test validation and Local is deferred. | `RECONCILE — RESOLVED IN BOOTSTRAP CANDIDATE` | Adopt the candidate wording on independent PASS/merge. | Repository-convergence contract tests plus direct document review. | Future drift could again route workers to obsolete strategy. | No further bootstrap action; revisit only if later convergence work changes the strategy. |
|
| 45 |
+
| `AGENTS.md` product-direction section | Pre-bootstrap finding: agent routing still encoded the local-first path. Current bootstrap candidate aligns claim/governance discipline with Repository Convergence and preserves the two-role assurance model. | `RECONCILE — RESOLVED IN BOOTSTRAP CANDIDATE` | Keep agent routing aligned with the canonical roadmap/workpackage queue. | Repository-convergence contract tests plus direct document review. | Agent workers could otherwise restart superseded packages. | No further bootstrap action. |
|
| 46 |
+
| `ROADMAP.md` | Pre-bootstrap finding: strategy, long implementation history and old nine-phase/local-installer path were mixed. Current bootstrap candidate contains the concise five-stage roadmap with Stage 1 Repository Convergence as `CURRENT`. | `RECONCILE — RESOLVED IN BOOTSTRAP CANDIDATE` | On PASS/merge, keep ROADMAP strategic and move detailed execution to WORKPACKAGES/CHANGELOG. | Formal-stage parsing contract and direct roadmap review. | Reintroducing implementation history as current strategy would recreate source-of-truth drift. | No further bootstrap action; Pass B only if later technical convergence changes current truth. |
|
| 47 |
+
| `WORKPACKAGES.md` | Pre-bootstrap finding: multiple stacked `Current execution status override` sections and superseded Premium queues coexisted. Current bootstrap candidate explicitly contains the current executable queue only, with Repository Convergence active and later stages blocked by gates. | `RECONCILE — RESOLVED IN BOOTSTRAP CANDIDATE` | On PASS/merge, maintain one current executable queue. Historical work remains in CHANGELOG/handovers/Git. | Repository-convergence queue contract and direct workpackage review. | Stacked historical queues can misroute future workers. | No further bootstrap action; derive later technical packages only from current evidence. |
|
| 48 |
+
| `CHANGELOG.md` | Pre-bootstrap finding: the root changelog mixed a large historical record with language that could be misread as current state. Current bootstrap candidate starts a post-convergence implementation log and points to `history/CHANGELOG_PRE_CONVERGENCE_20260904.md`; that archive is byte-identical to the baseline changelog blob. | `RECONCILE — RESOLVED IN BOOTSTRAP CANDIDATE` | Keep root CHANGELOG as implementation history from the reset onward while preserving the archived pre-convergence provenance. | Archive blob identity plus convergence changelog contract. | Losing provenance or treating historical candidate status as current authority would both be defects. | No further bootstrap action. |
|
| 49 |
+
| `DECISION_LOG.md` | Pre-bootstrap finding: the strategic pivot was not yet recorded. Current bootstrap candidate contains D044 and retains concise still-binding Premium, review, Zorg, Scrub Key, reinsert and DOCX decisions. | `RECONCILE — RESOLVED IN BOOTSTRAP CANDIDATE` | On PASS/merge, keep only materially binding current/future decisions here; use Git/CHANGELOG for history. | D044/still-binding-decision contracts plus direct review. | Missing or duplicate decisions can create competing architecture authority. | No further bootstrap action unless a later accepted decision supersedes current direction. |
|
| 50 |
+
| `RISK_REGISTER.md` | Pre-bootstrap finding: historical execution narrative obscured live risk and R5 assumed local-first trust. Current bootstrap candidate preserves critical R1/R2/R10 and aligns R5 to Scrub Private retention/egress, with explicit source-of-truth and startup-mutation risks. | `RECONCILE — RESOLVED IN BOOTSTRAP CANDIDATE` | Keep current risks/mitigations visible without turning the register into an execution history. | Risk continuity/current-truth contracts plus direct review. | Stale risk framing can weaken or misroute safety work. | No further bootstrap action; update only as risk reality changes. |
|
| 51 |
+
| Open GitHub issues #112/#109/#107/#106 and earlier Premium/governance cycle issues | Several describe candidates that have since PASSed/merged; open state does not match main. #105/#96 additionally require a final consolidated deployed live retest after their child repairs. | `RECONCILE` | Reconstruct each against current main; close stale completed issues, preserve/consolidate any still-unverified live gate. | Raw PR/Actions/assurance comments + current runtime evidence. | Issue state can misroute future work. | Reconcile after bootstrap candidate is accepted; do not falsely claim unperformed live verification. |
|
| 52 |
+
| Hugging Face role in project docs | Pre-bootstrap finding: HF was described inconsistently against the prior local-first direction. Current bootstrap candidate consistently defines HF as a synthetic/approved-test application-validation surface, not confidential-production infrastructure assurance. | `RECONCILE — RESOLVED IN BOOTSTRAP CANDIDATE` | Preserve this bounded claim unless the deployment strategy changes. | Repository-convergence contracts plus current project-doc review. | Overclaiming provider-level retention/security would create false assurance. | No further bootstrap action. |
|
| 53 |
+
| Local/offline installer programme | Valuable prior planning remains recoverable, but current bootstrap candidate explicitly demotes Local/offline packaging from the active delivery line and records it as a possible future product variant only if business evidence reopens it. | `VARIANT-SPECIFIC — DEFERRED / BOOTSTRAP ROUTING RESOLVED` | Preserve history; do not execute installer work unless a future accepted requirement reopens it. | D038/D044 plus roadmap/workpackage contracts. | Maintaining two active delivery lines would create distraction and duplicate architecture. | No further bootstrap action. |
|
| 54 |
+
|
| 55 |
+
## Current issue-reconciliation notes
|
| 56 |
+
|
| 57 |
+
These notes are findings, not closure authority by themselves.
|
| 58 |
+
|
| 59 |
+
- #112: independent assurance for PR #111 recorded PASS; PR #111 is merged into current main; exact-main Tests and HF sync are green. Issue is stale as an active candidate gate.
|
| 60 |
+
- #109: independent assurance for PR #108 recorded PASS; PR #108 is merged. Issue is stale as an active candidate gate.
|
| 61 |
+
- #107/#106: implementation issues whose corresponding repairs are merged; active issue state requires reconciliation.
|
| 62 |
+
- #105/#96: their original App Shell state defect was repaired, but live verification later exposed marker/address defects. Those child defects were subsequently repaired by PR #108/#111. Evidence found so far does **not** prove the final consolidated deployed live retest required by the parent issues was completed. Do not close that residual gate as verified without evidence.
|
| 63 |
+
- Earlier #74–#100 Premium/governance repair/assurance issues are historical candidate cycles and should be checked for stale open state, not treated as current implementation direction.
|
| 64 |
+
|
| 65 |
+
## Current validation-hierarchy hypothesis
|
| 66 |
+
|
| 67 |
+
Subject to a narrow follow-up verification:
|
| 68 |
+
|
| 69 |
+
### Candidate canonical release/product evidence
|
| 70 |
+
|
| 71 |
+
- full `python -m pytest -q tests` regression;
|
| 72 |
+
- recognizer-backed `recall_benchmark_runner.py` / `recall_benchmark_report.py` corpus workflow;
|
| 73 |
+
- Zorg baseline/recognizer/cross-profile evidence where domain-specific;
|
| 74 |
+
- Phase-6 synthetic E2E workflow validation;
|
| 75 |
+
- Scrub Key security/roundtrip tests;
|
| 76 |
+
- document hygiene/fidelity tests;
|
| 77 |
+
- Premium AppTest/state tests.
|
| 78 |
+
|
| 79 |
+
### Supplemental diagnostic evidence
|
| 80 |
+
|
| 81 |
+
- entity scorecard and older report-only residual-risk builders when used within their explicit coverage limits.
|
| 82 |
+
|
| 83 |
+
### Explicit non-claim
|
| 84 |
+
|
| 85 |
+
None of the above alone proves production safety or removes the mandatory human-review requirement.
|
| 86 |
+
|
| 87 |
+
## Bootstrap boundary
|
| 88 |
+
|
| 89 |
+
This ledger authorizes no runtime change by itself. The bootstrap package may only reset current strategy/execution documentation and contract tests. Technical changes must be split into evidence-backed consequential workpackages and receive normal independent assurance.
|
RISK_REGISTER.md
CHANGED
|
@@ -1,267 +1,396 @@
|
|
| 1 |
-
# SolidPrivacy Scrub — Risk Register
|
| 2 |
|
| 3 |
-
This register tracks product, privacy, security and
|
| 4 |
|
| 5 |
-
|
| 6 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
Impact values: `critical`, `high`, `medium`, `low`.
|
| 8 |
|
| 9 |
---
|
| 10 |
|
| 11 |
## R1 — False negatives / missed sensitive data
|
| 12 |
|
| 13 |
-
Status: mitigating
|
| 14 |
-
Impact: critical
|
| 15 |
|
| 16 |
Risk:
|
| 17 |
|
| 18 |
```text
|
| 19 |
-
Sensitive data remains in
|
| 20 |
```
|
| 21 |
|
| 22 |
-
Current
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
-
|
| 25 |
|
| 26 |
-
|
| 27 |
-
- No production benchmark gate exists.
|
| 28 |
-
- No production safety claim is supported.
|
| 29 |
-
- Human review remains necessary.
|
| 30 |
|
| 31 |
---
|
| 32 |
|
| 33 |
-
## R2 — Scrub Key leakage or
|
| 34 |
|
| 35 |
-
Status: mitigating
|
| 36 |
-
Impact: critical
|
| 37 |
|
| 38 |
Risk:
|
| 39 |
|
| 40 |
```text
|
| 41 |
-
|
| 42 |
```
|
| 43 |
|
| 44 |
-
Current
|
| 45 |
|
| 46 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
|
| 48 |
---
|
| 49 |
|
| 50 |
-
## R3 — Placeholder corruption during AI roundtrip
|
| 51 |
|
| 52 |
-
Status: mitigating
|
| 53 |
-
Impact: high
|
| 54 |
|
| 55 |
Risk:
|
| 56 |
|
| 57 |
```text
|
| 58 |
-
An AI
|
| 59 |
```
|
| 60 |
|
| 61 |
-
Current
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 62 |
|
| 63 |
---
|
| 64 |
|
| 65 |
-
## R4 — Hidden document content and
|
| 66 |
|
| 67 |
-
Status: mitigating
|
| 68 |
-
Impact: high
|
| 69 |
|
| 70 |
Risk:
|
| 71 |
|
| 72 |
```text
|
| 73 |
-
DOCX metadata, comments, tracked changes,
|
| 74 |
```
|
| 75 |
|
| 76 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 77 |
|
| 78 |
---
|
| 79 |
|
| 80 |
-
## R5 —
|
| 81 |
|
| 82 |
-
Status:
|
| 83 |
-
Impact:
|
| 84 |
|
| 85 |
Risk:
|
| 86 |
|
| 87 |
```text
|
| 88 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 89 |
```
|
| 90 |
|
| 91 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 92 |
|
| 93 |
---
|
| 94 |
|
| 95 |
-
## R6 — Review UX and
|
| 96 |
|
| 97 |
-
Status: mitigating
|
| 98 |
-
Impact: high
|
| 99 |
|
| 100 |
Risk:
|
| 101 |
|
| 102 |
```text
|
| 103 |
-
The
|
| 104 |
```
|
| 105 |
|
| 106 |
-
Current
|
| 107 |
-
|
| 108 |
-
- Review table remains source of truth and fallback.
|
| 109 |
-
- Side-by-side review surface, synced scrolling and collapsible review table are live and verified.
|
| 110 |
-
- Export/download UX is directly implemented in `presidio_streamlit.py` and live verified.
|
| 111 |
-
- `REVIEW_DEBUG_ELEMENTS_COLLAPSE_PLAN.md` narrows review UI cleanup to a small interface pass, not a new review loop.
|
| 112 |
-
- `WP_REVIEW_DEBUG_ELEMENTS_COLLAPSE_IMPLEMENTATION` made the existing step-by-step review aid collapsed by default and removed debug/governance wording from the primary UI.
|
| 113 |
-
- `WP_MVP_FAST_MANUAL_MASK_ENTRY` adds a verified simple user-facing path to add missed values to the existing replacement table.
|
| 114 |
-
- The contract, action model and component spike are now connected through `SCRUB-WP_PROCESSED_TEXT_SELECTION_TABLE_INTEGRATION`. The review table remains source of truth; the component is only an input route into normal bound manual rows.
|
| 115 |
-
- Live app evidence on 2026-08-05 confirms that local decluttering is not sufficient: the application still exposes input, settings, review, downloads, Scrub Key and audit as one long form. `PREMIUM_CORE_FLOW_UI_REALIGNMENT_PLAN.md` mitigates this through a global Standard/Expert model, top-level workflow navigation, one active stage and progressive disclosure.
|
| 116 |
|
| 117 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 118 |
|
| 119 |
-
|
| 120 |
-
- Additional copy polish may still be needed, but it should remain separate and small.
|
| 121 |
-
- Production table integration preserves collision/replay/stale guards, hidden-marker protection, immediate rerun and edit-aware undo. Deployment synchronization and live browser/app verification are green. Display-only placeholder compaction is implemented with lossless source tokens, exact UTF-16 mapping and protected compact spans; the underlying 80-bit binding remains unchanged. Focused deployment/app verification of the compact view is required before the remaining high-risk export/Scrub Key/reinsert cross-flow regression.
|
| 122 |
-
- Implementation must avoid weakening review controls or hiding audit details; one final confidentiality acknowledgement remains at download.
|
| 123 |
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
- The general UI baseline is completed and app-verified. The automatic reinsert-flow package is permitted as a narrow exception because live Phase 6 validation exposed a concrete workflow blocker; broader UI work remains gated.
|
| 127 |
|
| 128 |
---
|
| 129 |
|
| 130 |
## R7 — PDF limitations misunderstood by users
|
| 131 |
|
| 132 |
-
Status: mitigating
|
| 133 |
-
Impact: high
|
| 134 |
|
| 135 |
Risk:
|
| 136 |
|
| 137 |
```text
|
| 138 |
-
Users
|
| 139 |
```
|
| 140 |
|
| 141 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 142 |
|
| 143 |
---
|
| 144 |
|
| 145 |
-
## R8 —
|
| 146 |
|
| 147 |
-
Status: mitigating
|
| 148 |
-
Impact:
|
| 149 |
|
| 150 |
Risk:
|
| 151 |
|
| 152 |
```text
|
| 153 |
-
|
| 154 |
```
|
| 155 |
|
| 156 |
-
Current
|
| 157 |
|
| 158 |
-
-
|
| 159 |
-
-
|
| 160 |
-
-
|
| 161 |
-
- `MVP_UI_CLEANUP_AND_EXPORT_REDESIGN_PLAN.md` states that technical/audit details must remain available but become secondary.
|
| 162 |
-
- Export/download UX now groups document downloads, key file, and audit/technical files while keeping audit details available.
|
| 163 |
-
- The step-by-step review aid is now secondary by default, while review table and audit controls remain available.
|
| 164 |
-
- The verified manual missed-value entry is intentionally placed in the primary review path because it directly supports faster anonymization.
|
| 165 |
-
- The approved selection-driven route may reduce navigation friction, but every accepted action must still become a visible normal replacement-table row and must not mutate export or Scrub Key state directly from the browser component.
|
| 166 |
|
| 167 |
-
|
| 168 |
|
| 169 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 170 |
|
| 171 |
---
|
| 172 |
|
| 173 |
-
## R9 — Dutch
|
| 174 |
|
| 175 |
-
Status: mitigating
|
| 176 |
-
Impact: high
|
| 177 |
|
| 178 |
Risk:
|
| 179 |
|
| 180 |
```text
|
| 181 |
-
|
| 182 |
```
|
| 183 |
|
| 184 |
-
Current
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 185 |
|
| 186 |
---
|
| 187 |
|
| 188 |
-
## R10 —
|
| 189 |
|
| 190 |
-
Status: mitigating
|
| 191 |
-
Impact: critical
|
| 192 |
|
| 193 |
Risk:
|
| 194 |
|
| 195 |
```text
|
| 196 |
-
A care document retains patient
|
| 197 |
```
|
| 198 |
|
| 199 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 200 |
|
| 201 |
-
-
|
| 202 |
-
|
| 203 |
-
|
| 204 |
-
- current-engine baseline before recognizer changes;
|
| 205 |
-
- separate care taxonomy and recognizers;
|
| 206 |
-
- negative tests for medical numbers, dosages, times, vital signs and laboratory values;
|
| 207 |
-
- cross-profile regression before UI promotion;
|
| 208 |
-
- human review and residual-risk evidence remain mandatory.
|
| 209 |
|
| 210 |
-
|
|
|
|
|
|
|
|
|
|
| 211 |
|
| 212 |
```text
|
| 213 |
-
|
| 214 |
-
Other exact care dates and provider identity: review, selected by default.
|
| 215 |
-
Clinical meaning: preserve.
|
| 216 |
-
Rare-case indirect identification: audit warning, not blind masking.
|
| 217 |
```
|
| 218 |
|
| 219 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 220 |
|
| 221 |
-
|
| 222 |
|
| 223 |
-
|
| 224 |
-
- 14/81 were found under the intended entity type;
|
| 225 |
-
- 11 were misclassified and 56 missed;
|
| 226 |
-
- only 4/42 review-selected values were found;
|
| 227 |
-
- one AGB value collided with BSN recognition;
|
| 228 |
-
- no designated clinical preserve phrase was overlapped by the current custom rules.
|
| 229 |
|
| 230 |
-
|
| 231 |
|
| 232 |
-
|
| 233 |
|
| 234 |
-
|
|
|
|
| 235 |
|
| 236 |
-
|
| 237 |
|
| 238 |
-
|
|
|
|
|
|
|
| 239 |
|
| 240 |
-
|
| 241 |
|
| 242 |
-
|
|
|
|
| 243 |
|
| 244 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 245 |
|
| 246 |
-
|
| 247 |
|
| 248 |
---
|
| 249 |
|
| 250 |
-
#
|
| 251 |
|
| 252 |
-
Disallowed claims:
|
| 253 |
|
| 254 |
```text
|
| 255 |
-
|
| 256 |
-
|
| 257 |
-
|
| 258 |
-
|
| 259 |
-
|
| 260 |
```
|
| 261 |
|
| 262 |
-
Allowed
|
| 263 |
|
| 264 |
```text
|
| 265 |
-
Scrub helpt
|
| 266 |
-
Technische en auditdetails blijven beschikbaar voor controle.
|
| 267 |
```
|
|
|
|
|
|
|
|
|
| 1 |
+
# SolidPrivacy Scrub — Current Risk Register
|
| 2 |
|
| 3 |
+
This register tracks the **current** product, privacy, security, trust and execution risks for SolidPrivacy Scrub.
|
| 4 |
|
| 5 |
+
Detailed pre-convergence risk-history remains recoverable from exact baseline:
|
| 6 |
|
| 7 |
+
```text
|
| 8 |
+
54c73e0ebf5a3a3ed7039a50596fb57694add3cd
|
| 9 |
+
```
|
| 10 |
+
|
| 11 |
+
Git/CHANGELOG preserve execution history. This file should show the live risk picture rather than accumulate workpackage narrative.
|
| 12 |
+
|
| 13 |
+
Status values: `open`, `mitigating`, `accepted`, `closed`.
|
| 14 |
Impact values: `critical`, `high`, `medium`, `low`.
|
| 15 |
|
| 16 |
---
|
| 17 |
|
| 18 |
## R1 — False negatives / missed sensitive data
|
| 19 |
|
| 20 |
+
Status: `mitigating`
|
| 21 |
+
Impact: `critical`
|
| 22 |
|
| 23 |
Risk:
|
| 24 |
|
| 25 |
```text
|
| 26 |
+
Sensitive data remains in scrubbed output and the user wrongly trusts the result.
|
| 27 |
```
|
| 28 |
|
| 29 |
+
Current controls/evidence:
|
| 30 |
+
|
| 31 |
+
- layered deterministic Dutch recognizers plus generic NER;
|
| 32 |
+
- dedicated Legal/Zorg rules and profile policy;
|
| 33 |
+
- recognizer-backed synthetic corpus benchmark/reporting;
|
| 34 |
+
- domain-specific baseline/gap-triage/cross-profile tests;
|
| 35 |
+
- manual missed-value addition and direct processed-text correction;
|
| 36 |
+
- mandatory human review;
|
| 37 |
+
- residual-risk/known-limitation language.
|
| 38 |
+
|
| 39 |
+
Current gaps:
|
| 40 |
+
|
| 41 |
+
- no accepted real-world production recall guarantee exists;
|
| 42 |
+
- synthetic corpora do not prove complete production coverage;
|
| 43 |
+
- human review remains necessary;
|
| 44 |
+
- new recognizer changes must remain evidence-driven so precision fixes do not silently reduce recall.
|
| 45 |
|
| 46 |
+
Required principle:
|
| 47 |
|
| 48 |
+
> Do not claim perfect recall or production safety from synthetic benchmark results.
|
|
|
|
|
|
|
|
|
|
| 49 |
|
| 50 |
---
|
| 51 |
|
| 52 |
+
## R2 — Scrub Key leakage, mismatch or misuse
|
| 53 |
|
| 54 |
+
Status: `mitigating`
|
| 55 |
+
Impact: `critical`
|
| 56 |
|
| 57 |
Risk:
|
| 58 |
|
| 59 |
```text
|
| 60 |
+
A Scrub Key leaks, is paired with the wrong document, is corrupted/tampered with, or is otherwise mishandled, enabling incorrect or unauthorized re-identification.
|
| 61 |
```
|
| 62 |
|
| 63 |
+
Current controls/evidence:
|
| 64 |
|
| 65 |
+
- explicit sensitive-key treatment in UI;
|
| 66 |
+
- document-bound placeholders;
|
| 67 |
+
- mapping digest;
|
| 68 |
+
- structural key validation;
|
| 69 |
+
- fail-closed wrong/mixed/missing-binding handling;
|
| 70 |
+
- controlled TXT/DOCX/PDF-to-TXT reinsert paths;
|
| 71 |
+
- adversarial/roundtrip tests;
|
| 72 |
+
- legacy unbound state kept explicitly unverified.
|
| 73 |
+
|
| 74 |
+
Current gaps:
|
| 75 |
+
|
| 76 |
+
- Scrub Key remains inherently sensitive re-identification material;
|
| 77 |
+
- cryptographic authenticity/signing is not implied by the current mapping digest;
|
| 78 |
+
- user/operational handling remains part of the threat model.
|
| 79 |
|
| 80 |
---
|
| 81 |
|
| 82 |
+
## R3 — Placeholder corruption during AI/external roundtrip
|
| 83 |
|
| 84 |
+
Status: `mitigating`
|
| 85 |
+
Impact: `high`
|
| 86 |
|
| 87 |
Risk:
|
| 88 |
|
| 89 |
```text
|
| 90 |
+
An external AI/process rewrites, translates, merges, deletes or corrupts placeholders, causing deterministic reinsert to fail or restore incompletely.
|
| 91 |
```
|
| 92 |
|
| 93 |
+
Current controls/evidence:
|
| 94 |
+
|
| 95 |
+
- stable bound placeholder grammar;
|
| 96 |
+
- mapping/document binding;
|
| 97 |
+
- roundtrip/adversarial mutation tests;
|
| 98 |
+
- unknown/missing placeholder diagnostics;
|
| 99 |
+
- fail-visible reinsert behavior.
|
| 100 |
+
|
| 101 |
+
Current gaps:
|
| 102 |
+
|
| 103 |
+
- malformed near-placeholders outside strict grammar may remain a diagnostic limitation;
|
| 104 |
+
- Scrub must not silently guess/repair values where binding cannot be proven.
|
| 105 |
|
| 106 |
---
|
| 107 |
|
| 108 |
+
## R4 — Hidden document content, metadata and unsupported-part leakage
|
| 109 |
|
| 110 |
+
Status: `mitigating`
|
| 111 |
+
Impact: `high`
|
| 112 |
|
| 113 |
Risk:
|
| 114 |
|
| 115 |
```text
|
| 116 |
+
DOCX metadata, comments, tracked changes, unsupported XML parts, headers/footers or other hidden content retains sensitive information outside the supported scrub/reinsert surface.
|
| 117 |
```
|
| 118 |
|
| 119 |
+
Current controls/evidence:
|
| 120 |
+
|
| 121 |
+
- DOCX hygiene/audit reporting;
|
| 122 |
+
- body/table/header/footer supported handling;
|
| 123 |
+
- synthetic fidelity/hygiene tests;
|
| 124 |
+
- explicit unsupported-part warnings;
|
| 125 |
+
- PDF support is text-based with explicit product limitations.
|
| 126 |
+
|
| 127 |
+
Current gaps:
|
| 128 |
+
|
| 129 |
+
- unsupported DOCX parts remain a known boundary;
|
| 130 |
+
- hygiene reporting must not be presented as a complete clean-document guarantee where unsupported content may exist.
|
| 131 |
|
| 132 |
---
|
| 133 |
|
| 134 |
+
## R5 — Scrub Private content-retention / external-egress trust boundary
|
| 135 |
|
| 136 |
+
Status: `open`
|
| 137 |
+
Impact: `critical`
|
| 138 |
|
| 139 |
Risk:
|
| 140 |
|
| 141 |
```text
|
| 142 |
+
A managed Private service retains customer document content, mappings or Scrub Keys, logs content, backs it up, or sends document content to a third party contrary to the intended service promise.
|
| 143 |
+
```
|
| 144 |
+
|
| 145 |
+
Current evidence on pre-convergence/full-feature application:
|
| 146 |
+
|
| 147 |
+
- `replacement_memory.py` intentionally persists original/replacement/entity values to `/data/replacement_memory.json` when persistent storage exists;
|
| 148 |
+
- current Expert functionality includes Azure AI Language and OpenAI/Azure synthesis paths;
|
| 149 |
+
- synthesis helper currently prints a content-bearing prompt.
|
| 150 |
+
|
| 151 |
+
These are not evidence that the current prototype is defective for every deployment model; they are **variant-specific conflicts** with Scrub Private.
|
| 152 |
+
|
| 153 |
+
Scrub Private target:
|
| 154 |
+
|
| 155 |
+
```text
|
| 156 |
+
no intentional persistent server-side customer document content
|
| 157 |
+
no content-bearing ordinary application logs
|
| 158 |
+
no document-content backup
|
| 159 |
+
no third-party document-processing egress
|
| 160 |
+
minimal non-document control-plane persistence only when required
|
| 161 |
```
|
| 162 |
|
| 163 |
+
Mitigation sequence:
|
| 164 |
+
|
| 165 |
+
- first complete Repository Convergence without destroying potentially useful Local/full-feature behavior;
|
| 166 |
+
- then remove/exclude hosted-incompatible persistence/egress from the Private line in separately tested/assured workpackages;
|
| 167 |
+
- prove application-level behavior on HF with synthetic data;
|
| 168 |
+
- prove infrastructure/service retention properties later on controlled production infrastructure.
|
| 169 |
+
|
| 170 |
+
Do not use HF application tests to claim provider-level zero retention.
|
| 171 |
|
| 172 |
---
|
| 173 |
|
| 174 |
+
## R6 — Review UX, stale state and workflow clarity
|
| 175 |
|
| 176 |
+
Status: `mitigating`
|
| 177 |
+
Impact: `high`
|
| 178 |
|
| 179 |
Risk:
|
| 180 |
|
| 181 |
```text
|
| 182 |
+
The user misses a sensitive value, changes state without realizing downstream output is stale, or the interface obscures whether review is complete/current.
|
| 183 |
```
|
| 184 |
|
| 185 |
+
Current controls/evidence:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 186 |
|
| 187 |
+
- staged Premium workspace;
|
| 188 |
+
- Standard/Expert modes over shared authoritative state;
|
| 189 |
+
- generation-bound analysis/review/export state;
|
| 190 |
+
- fail-closed stale export gating;
|
| 191 |
+
- explicit re-completion after review edits;
|
| 192 |
+
- source-versus-processed review;
|
| 193 |
+
- authoritative replacement table;
|
| 194 |
+
- direct/manual missed-value correction;
|
| 195 |
+
- AppTest and integration coverage.
|
| 196 |
|
| 197 |
+
Current gap:
|
|
|
|
|
|
|
|
|
|
| 198 |
|
| 199 |
+
- parent Premium live-verification issues contain stale/unfinished closeout state; current deployed behavior must be reconciled against merged marker/address repairs before declaring those historical gates closed.
|
|
|
|
|
|
|
| 200 |
|
| 201 |
---
|
| 202 |
|
| 203 |
## R7 — PDF limitations misunderstood by users
|
| 204 |
|
| 205 |
+
Status: `mitigating`
|
| 206 |
+
Impact: `high`
|
| 207 |
|
| 208 |
Risk:
|
| 209 |
|
| 210 |
```text
|
| 211 |
+
Users interpret text-based PDF support as OCR or complete restored-PDF fidelity.
|
| 212 |
```
|
| 213 |
|
| 214 |
+
Current boundary:
|
| 215 |
+
|
| 216 |
+
- text-based PDF extraction is supported;
|
| 217 |
+
- OCR is not implied;
|
| 218 |
+
- restored PDF generation is not implied;
|
| 219 |
+
- reinsert output may be restored text rather than a reconstructed PDF.
|
| 220 |
+
|
| 221 |
+
Copy, audit and pilot guidance must keep these limitations explicit.
|
| 222 |
|
| 223 |
---
|
| 224 |
|
| 225 |
+
## R8 — Audit/evidence clarity and false confidence
|
| 226 |
|
| 227 |
+
Status: `mitigating`
|
| 228 |
+
Impact: `high`
|
| 229 |
|
| 230 |
Risk:
|
| 231 |
|
| 232 |
```text
|
| 233 |
+
Multiple benchmark/report generations are mistaken for equivalent safety evidence, or diagnostic metrics are presented as production-readiness/individual-document safety scores.
|
| 234 |
```
|
| 235 |
|
| 236 |
+
Current controls:
|
| 237 |
|
| 238 |
+
- synthetic-only/report-only warnings in benchmark/residual-risk tooling;
|
| 239 |
+
- known limitations and human-review requirement;
|
| 240 |
+
- separate domain and workflow validation systems.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 241 |
|
| 242 |
+
Current convergence requirement:
|
| 243 |
|
| 244 |
+
Classify existing evidence paths as:
|
| 245 |
+
|
| 246 |
+
```text
|
| 247 |
+
CANONICAL RELEASE VALIDATION
|
| 248 |
+
SUPPLEMENTAL DIAGNOSTIC
|
| 249 |
+
HISTORICAL / SUPERSEDED
|
| 250 |
+
```
|
| 251 |
+
|
| 252 |
+
Do not create a replacement Evidence Framework merely to make the hierarchy look cleaner.
|
| 253 |
+
|
| 254 |
+
Disallowed product behavior:
|
| 255 |
+
|
| 256 |
+
- per-document “94% safe” or similar false precision;
|
| 257 |
+
- production-safety claim solely from synthetic scores.
|
| 258 |
|
| 259 |
---
|
| 260 |
|
| 261 |
+
## R9 — Dutch Legal recognition precision/recall and context damage
|
| 262 |
|
| 263 |
+
Status: `mitigating`
|
| 264 |
+
Impact: `high`
|
| 265 |
|
| 266 |
Risk:
|
| 267 |
|
| 268 |
```text
|
| 269 |
+
Legal identifiers/addresses/names are missed or misclassified, or ordinary legal/professional context is over-masked and loses meaning.
|
| 270 |
```
|
| 271 |
|
| 272 |
+
Current controls/evidence:
|
| 273 |
+
|
| 274 |
+
- Dutch deterministic recognizers and Legal profile;
|
| 275 |
+
- synthetic legal examples/corpus;
|
| 276 |
+
- address-span precision repair on current main;
|
| 277 |
+
- preserve/known-trap evidence;
|
| 278 |
+
- manual/direct review correction;
|
| 279 |
+
- human review.
|
| 280 |
+
|
| 281 |
+
Future fixes remain test-first and must preserve recall/context.
|
| 282 |
|
| 283 |
---
|
| 284 |
|
| 285 |
+
## R10 — Zorg under-detection and clinical over-masking
|
| 286 |
|
| 287 |
+
Status: `mitigating`
|
| 288 |
+
Impact: `critical`
|
| 289 |
|
| 290 |
Risk:
|
| 291 |
|
| 292 |
```text
|
| 293 |
+
A care document retains patient/trajectory identifiers or Scrub removes clinical meaning and makes the document misleading/unusable.
|
| 294 |
```
|
| 295 |
|
| 296 |
+
Current controls/evidence:
|
| 297 |
+
|
| 298 |
+
- explicit Zorg profile/policy;
|
| 299 |
+
- eight synthetic care-document families plus long-form variants;
|
| 300 |
+
- current-engine baseline before dedicated rules;
|
| 301 |
+
- gap triage;
|
| 302 |
+
- dedicated care recognizer contracts/implementation;
|
| 303 |
+
- collision/negative tests;
|
| 304 |
+
- cross-profile regression;
|
| 305 |
+
- clinical preserve expectations;
|
| 306 |
+
- human review.
|
| 307 |
+
|
| 308 |
+
Current gap:
|
| 309 |
+
|
| 310 |
+
Synthetic and bounded app evidence does not establish complete production recall/precision or rare-case indirect-identification safety.
|
| 311 |
+
|
| 312 |
+
Required principle:
|
| 313 |
+
|
| 314 |
+
> Preserve diagnosis, medication, dosage, laboratory values, observations and useful care context unless there is a specific evidence-backed reason to treat a value as identifying.
|
| 315 |
|
| 316 |
+
---
|
| 317 |
+
|
| 318 |
+
## R11 — Repository/source-of-truth drift
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 319 |
|
| 320 |
+
Status: `mitigating`
|
| 321 |
+
Impact: `high`
|
| 322 |
+
|
| 323 |
+
Risk:
|
| 324 |
|
| 325 |
```text
|
| 326 |
+
Workers follow stale ROADMAP/WORKPACKAGES/issues or duplicate already-built functionality because repository documentation and actual main no longer describe one coherent current truth.
|
|
|
|
|
|
|
|
|
|
| 327 |
```
|
| 328 |
|
| 329 |
+
Observed pre-convergence symptoms:
|
| 330 |
+
|
| 331 |
+
- old nine-phase/local-installer roadmap still active in docs;
|
| 332 |
+
- multiple stacked `Current execution status override` sections in WORKPACKAGES;
|
| 333 |
+
- open issues describing candidate states that already PASSed/merged;
|
| 334 |
+
- historical Premium packages still presented as future work despite current main containing integrated staged behavior;
|
| 335 |
+
- several generations of benchmark/report tooling with unclear release authority.
|
| 336 |
+
|
| 337 |
+
Current mitigation:
|
| 338 |
+
|
| 339 |
+
- Repository Convergence is the active execution line;
|
| 340 |
+
- exact pre-convergence SHA preserved;
|
| 341 |
+
- temporary capability/debt ledger;
|
| 342 |
+
- ROADMAP/WORKPACKAGES/PROJECT_PROMPT reset;
|
| 343 |
+
- stale issue/evidence hierarchy reconciliation before clean-baseline declaration;
|
| 344 |
+
- no new feature work until `SCRUB_REPOSITORY_CONVERGED`.
|
| 345 |
|
| 346 |
+
Exit condition:
|
| 347 |
|
| 348 |
+
One clean exact SHA whose source, tests, current issues and canonical docs agree materially.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 349 |
|
| 350 |
+
---
|
| 351 |
|
| 352 |
+
## R12 — Legacy runtime mutation / hidden startup authority
|
| 353 |
|
| 354 |
+
Status: `mitigating`
|
| 355 |
+
Impact: `medium`
|
| 356 |
|
| 357 |
+
Risk:
|
| 358 |
|
| 359 |
+
```text
|
| 360 |
+
Historical source-patch machinery remains in the startup path and can be mistaken for current product authority or accidentally reactivated.
|
| 361 |
+
```
|
| 362 |
|
| 363 |
+
Current evidence:
|
| 364 |
|
| 365 |
+
- Docker still invokes `fix_streamlit_nested_expanders.py` and `fix_streamlit_pdf_text_reinsert.py`;
|
| 366 |
+
- current Premium/direct-source markers cause both to exit without changing source.
|
| 367 |
|
| 368 |
+
Mitigation direction:
|
| 369 |
+
|
| 370 |
+
- verify no supported current path depends on the mutation scripts;
|
| 371 |
+
- retire invocation/obsolete scripts in a narrow consequential package if that proof holds;
|
| 372 |
+
- retain product-level behavior tests rather than tests whose only purpose is to keep dead patch machinery alive.
|
| 373 |
|
| 374 |
+
Do not combine this with hosted persistence/egress changes.
|
| 375 |
|
| 376 |
---
|
| 377 |
|
| 378 |
+
# Product-claim boundary
|
| 379 |
|
| 380 |
+
Disallowed claims include:
|
| 381 |
|
| 382 |
```text
|
| 383 |
+
Scrub vindt altijd alle persoonsgegevens.
|
| 384 |
+
Scrub garandeert volledige anonymisering.
|
| 385 |
+
De synthetische benchmark bewijst production readiness.
|
| 386 |
+
Hugging Face tests bewijzen zero retention op provider-infrastructuurniveau.
|
| 387 |
+
Een scrubbed document is veilig zonder menselijke controle.
|
| 388 |
```
|
| 389 |
|
| 390 |
+
Allowed direction:
|
| 391 |
|
| 392 |
```text
|
| 393 |
+
Scrub helpt gevoelige waarden te detecteren, controleren en pseudonimiseren terwijl context behouden blijft. Menselijke review en zichtbare beperkingen blijven onderdeel van het veiligheidsmodel.
|
|
|
|
| 394 |
```
|
| 395 |
+
|
| 396 |
+
For Scrub Private, content-retention/security claims must match independently verified application and service behavior exactly.
|
ROADMAP.md
CHANGED
|
@@ -1,682 +1,370 @@
|
|
| 1 |
-
# SolidPrivacy Scrub —
|
| 2 |
|
| 3 |
-
This document is the
|
| 4 |
|
| 5 |
Use it together with:
|
| 6 |
|
| 7 |
-
- `WORKPACKAGES.md`
|
| 8 |
-
- `
|
| 9 |
-
- `
|
| 10 |
-
- `
|
| 11 |
-
- `
|
| 12 |
-
- `PROJECT_PROMPT.md` for worker rules and project governance.
|
| 13 |
|
| 14 |
-
Last
|
| 15 |
|
| 16 |
---
|
| 17 |
|
| 18 |
-
##
|
| 19 |
|
| 20 |
-
|
| 21 |
|
| 22 |
-
|
| 23 |
-
implementation_operations
|
| 24 |
-
→ identifiable release candidate
|
| 25 |
-
→ governance_release_assurance blind reconstruction
|
| 26 |
-
→ PASS / FAIL / INDETERMINATE
|
| 27 |
-
→ authorized action
|
| 28 |
-
→ independent post-action confirmation
|
| 29 |
-
```
|
| 30 |
-
|
| 31 |
-
The user remains the single coordinator-facing principal. Implementation cannot certify its own candidate, and governance cannot silently repair it. Scrub adds a blind-review boundary: before its initial decision, governance may inspect source, criteria and raw machine evidence but not implementation handovers, self-assessments or conclusions.
|
| 32 |
-
|
| 33 |
-
The current maturity is `LEVEL_1_CHECKLIST`, with a later target of `LEVEL_2_MACHINE_EVIDENCE`.
|
| 34 |
-
|
| 35 |
-
The governance/cross-flow gate that previously blocked the Premium UI line is closed. Issue #70 is closed, PR #73 is merged, the Premium UI contract is merged through PR #80, and the pure Premium core-flow state model is merged through PR #82. The active line is therefore Premium UI implementation, subject to independent assurance for each consequential candidate.
|
| 36 |
|
| 37 |
-
|
| 38 |
-
-
|
| 39 |
-
-
|
| 40 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
Scrub is evolving from a technical Presidio demo into a trustworthy Dutch professional document scrubber for confidential legal and care documents.
|
| 45 |
-
|
| 46 |
-
The key promise remains:
|
| 47 |
|
| 48 |
```text
|
| 49 |
-
|
| 50 |
-
The user remains in control.
|
| 51 |
-
The document stays readable.
|
| 52 |
-
Residual risk is visible.
|
| 53 |
```
|
| 54 |
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
```text
|
| 58 |
-
Make the MVP workflow good before investing further in pilot operations, marketing, pricing, sales, promotion or installer work.
|
| 59 |
-
```
|
| 60 |
-
|
| 61 |
-
The active MVP workflow is:
|
| 62 |
-
|
| 63 |
-
```text
|
| 64 |
-
Import -> Scrub -> Review -> Replace -> Scrub Key -> Reinsert -> Export -> Audit
|
| 65 |
-
```
|
| 66 |
|
| 67 |
---
|
| 68 |
|
| 69 |
-
## 2.
|
| 70 |
-
|
| 71 |
-
The product is not just a generic anonymizer.
|
| 72 |
|
| 73 |
-
|
| 74 |
|
| 75 |
-
|
| 76 |
-
How can a professional safely use or share confidential documents without losing context, legal meaning, control or auditability?
|
| 77 |
-
```
|
| 78 |
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
1. high recall for sensitive data;
|
| 82 |
-
2. context preservation;
|
| 83 |
-
3. human review;
|
| 84 |
-
4. easy and consistent replacement logic;
|
| 85 |
-
5. secure Scrub Key lifecycle;
|
| 86 |
-
6. reliable de-anonymization/reinsert;
|
| 87 |
-
7. document hygiene;
|
| 88 |
-
8. safe export;
|
| 89 |
-
9. audit and residual-risk reporting;
|
| 90 |
-
10. clear interface and workflow confidence.
|
| 91 |
-
|
| 92 |
-
Legal and professional context must be preserved. Scrub should replace sensitive values, not legal meaning.
|
| 93 |
-
|
| 94 |
-
---
|
| 95 |
-
|
| 96 |
-
## 3. Strategic workflow
|
| 97 |
-
|
| 98 |
-
The product workflow direction remains:
|
| 99 |
-
|
| 100 |
-
```text
|
| 101 |
-
Scrub -> Review -> Scrub Key -> AI -> Reinsert -> Export -> Audit
|
| 102 |
-
```
|
| 103 |
-
|
| 104 |
-
The roadmap is risk-driven rather than feature-driven.
|
| 105 |
-
|
| 106 |
-
Highest priority risks:
|
| 107 |
-
|
| 108 |
-
1. false negatives;
|
| 109 |
-
2. Scrub Key leakage or accidental sharing;
|
| 110 |
3. hidden document content and metadata leakage;
|
| 111 |
-
4. placeholder corruption during AI roundtrip;
|
| 112 |
-
5. review
|
| 113 |
-
6.
|
| 114 |
-
7.
|
| 115 |
|
| 116 |
-
|
| 117 |
-
|
| 118 |
-
## 4. Current implementation status
|
| 119 |
-
|
| 120 |
-
Recent work completed or recorded:
|
| 121 |
-
|
| 122 |
-
```text
|
| 123 |
-
WP19-WP24 — recall/trust foundation.
|
| 124 |
-
WP25-WP29C — Scrub Key safety/test line.
|
| 125 |
-
WP30-WP34 — placeholder robustness helper/test line.
|
| 126 |
-
WP35-WP39 — DOCX hygiene line through clean-DOCX export policy and report-only audit UI.
|
| 127 |
-
WP40-WP43 — review UX/frontend decision line.
|
| 128 |
-
WP_REPLACE_LOGIC_UI_PRODUCT_ROLLBACK_VERIFY — product-rejected helper panel hidden and verified.
|
| 129 |
-
WP_REPLACE_LOGIC_UI_REDESIGN_PLAN — replacement review redesigned around simple user choices.
|
| 130 |
-
WP_SIDE_BY_SIDE_REVIEW_ROADMAP_ANCHOR — unified side-by-side review UX direction anchored.
|
| 131 |
-
WP45-WP49 — local runtime/packaging decision line; installer deferred.
|
| 132 |
-
WP50-WP51 — pilot/ICP thinking artifacts; parked for now.
|
| 133 |
-
WP_RECALL_PERSON_NAME_* — diagnostic, contract and helper-level PERSON-name work completed; benchmark follow-up temporarily parked unless a concrete blocker appears.
|
| 134 |
-
SCRUB-WP_EXECUTION_INTERFACE_SIMPLIFICATION_* through SCRUB-WP_MANUAL_CORRECTION_PANEL_DENSITY_SIMPLIFICATION_* — the current MVP UI simplification line is completed, synchronized and live-app verified.
|
| 135 |
-
SCRUB-WP_MVP_DOCUMENT_HYGIENE_FIDELITY_HARDENING — merged, synchronized and live-app verified for DOCX body, table, header and footer restoration.
|
| 136 |
-
SCRUB-WP_MVP_REINSERT_AUTO_FLOW_SIMPLIFICATION_IMPLEMENTATION — completed, synchronized and live-app verified.
|
| 137 |
-
SCRUB-WP_MVP_SCRUB_KEY_ROUNDTRIP_VALIDATION — completed with 15/15 cases passing and a critical document/key-binding gap routed to triage.
|
| 138 |
-
SCRUB-WP_MVP_SCRUB_KEY_DOCUMENT_BINDING_GAP_TRIAGE — completed with a bound-placeholder plus mapping-digest recommendation and a test-first implementation sequence.
|
| 139 |
-
SCRUB-WP_MVP_SCRUB_KEY_BINDING_CONTRACT_TESTS — completed with frozen placeholder, digest, legacy and fail-closed model contracts.
|
| 140 |
-
SCRUB-WP_MVP_SCRUB_KEY_BINDING_MODEL_IMPLEMENTATION — completed with pure binding-ID, placeholder, digest, bound-key and document/key validation helpers.
|
| 141 |
-
SCRUB-WP_MVP_SCRUB_KEY_BINDING_EXPORT_INTEGRATION — completed with bound placeholders and schema-1.1 key export.
|
| 142 |
-
SCRUB-WP_MVP_SCRUB_KEY_BINDING_REINSERT_INTEGRATION — implemented with dual-read import, fail-closed binding enforcement and explicit legacy compatibility; app verification pending.
|
| 143 |
-
SCRUB-WP_AI_FIRST_DESKTOP_PACKAGING_ROADMAP_ALIGNMENT — completed as a documentation-only Phase 9 execution-model refinement; installer implementation remains gated.
|
| 144 |
-
SCRUB-WP_PROCESSED_TEXT_SELECTION_CROSS_FLOW_REGRESSION — independently assured and governance gate closed through issue #70/#73 recovery.
|
| 145 |
-
SCRUB-WP_PREMIUM_CORE_FLOW_UI_CONTRACT — merged through PR #80; contract only, no production UI behavior.
|
| 146 |
-
SCRUB-WP_PREMIUM_CORE_FLOW_STATE_MODEL — merged through PR #82; pure state model only, no production UI behavior.
|
| 147 |
-
SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION — active in draft PR #85; current candidate contains reusable presentation primitives/tests but production `presidio_streamlit.py` integration has not started.
|
| 148 |
-
SCRUB-WP_PREMIUM_STAGED_WORKSPACE_DECISION_FREEZE — urgent architecture/queue intervention approved by the coordinator on 2026-08-08; must be incorporated before PR #85 proceeds to production UI integration.
|
| 149 |
-
```
|
| 150 |
-
|
| 151 |
-
Important UX status:
|
| 152 |
-
|
| 153 |
-
```text
|
| 154 |
-
The review table remains source of truth and fallback.
|
| 155 |
-
The old replacement decision helper panel must not return as normal user-facing UI.
|
| 156 |
-
The unified side-by-side review, manual missed-value entry, compact placeholder display and grouped export flow form the verified functional baseline.
|
| 157 |
-
Direct live-app evidence confirms that this baseline still presents too much of the workflow as one long form.
|
| 158 |
-
The Premium Standard target is one document workspace with three persistent stages and one active task, not another isolated expander-cleanup pass and not three routed core-flow pages.
|
| 159 |
-
```
|
| 160 |
|
| 161 |
---
|
| 162 |
|
| 163 |
-
##
|
| 164 |
|
| 165 |
-
|
| 166 |
|
| 167 |
-
|
| 168 |
-
- anonymization;
|
| 169 |
-
- review;
|
| 170 |
-
- replacement logic;
|
| 171 |
-
- Scrub Key handling;
|
| 172 |
-
- de-anonymization/reinsert;
|
| 173 |
-
- export;
|
| 174 |
-
- audit/residual-risk reporting;
|
| 175 |
-
- interface clarity.
|
| 176 |
|
| 177 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 178 |
|
| 179 |
-
|
| 180 |
|
| 181 |
---
|
| 182 |
|
| 183 |
-
#
|
| 184 |
-
|
| 185 |
-
The current phase order is:
|
| 186 |
-
|
| 187 |
-
```text
|
| 188 |
-
Phase 1 — Trust & recall foundation
|
| 189 |
-
Phase 2 — Scrub Key security and lifecycle
|
| 190 |
-
Phase 3 — Placeholder robustness for AI roundtrip
|
| 191 |
-
Phase 4 — Hidden content and document hygiene
|
| 192 |
-
Phase 5 — Document-centric review UX
|
| 193 |
-
Phase 6 — MVP workflow validation and trust hardening
|
| 194 |
-
Phase 7 — Pilot validation: Legal vs Zorg — parked until MVP quality gate passes
|
| 195 |
-
Phase 8 — Scale features: profiles, batch, CLI, enterprise
|
| 196 |
-
Phase 9 — Final local desktop/offline installer path
|
| 197 |
-
```
|
| 198 |
-
|
| 199 |
-
### Phase 5/6 — Review UX direction
|
| 200 |
-
|
| 201 |
-
The review UX target is now:
|
| 202 |
-
|
| 203 |
-
```text
|
| 204 |
-
Source text left | Processed/checked text right
|
| 205 |
-
| Optional highlights integrated in the processed text
|
| 206 |
-
```
|
| 207 |
-
|
| 208 |
-
This is governed by:
|
| 209 |
|
| 210 |
-
|
| 211 |
-
- `SIDE_BY_SIDE_REVIEW_UX_DIRECTION.md`;
|
| 212 |
-
- `REPLACE_LOGIC_UI_REDESIGN_PLAN.md`;
|
| 213 |
-
- `MVP_UI_CLEANUP_AND_EXPORT_REDESIGN_PLAN.md`;
|
| 214 |
-
- `PREMIUM_CORE_FLOW_UI_REALIGNMENT_PLAN.md`;
|
| 215 |
-
- `PREMIUM_STAGED_WORKSPACE_DECISION.md`.
|
| 216 |
|
| 217 |
-
|
| 218 |
|
| 219 |
-
|
| 220 |
-
- review table remains source of truth and fallback;
|
| 221 |
-
- serial review remains a guided review layer, not a table replacement;
|
| 222 |
-
- highlights are visual aid, not mutation mechanism;
|
| 223 |
-
- no repeated per-highlight `Gemarkeerd` labels as the long-term design;
|
| 224 |
-
- no click-to-mark, advanced editor or full-document marking in this phase;
|
| 225 |
-
- no Scrub Key/export/reinsert behavior changes from review UX work without separate approval;
|
| 226 |
-
- technical/debug-like details should move to secondary audit/advanced layers, not disappear.
|
| 227 |
-
|
| 228 |
-
### Premium core-flow UI realignment — binding staged-workspace decision
|
| 229 |
-
|
| 230 |
-
Direct coordinator/user evidence on 2026-08-05 established that the remaining interface problem is structural:
|
| 231 |
-
|
| 232 |
-
```text
|
| 233 |
-
The app still behaves visually as one long Streamlit form.
|
| 234 |
-
Input, settings, review, corrections, downloads, Scrub Key and audit controls compete on the same page.
|
| 235 |
-
```
|
| 236 |
-
|
| 237 |
-
The approved target remains:
|
| 238 |
-
|
| 239 |
-
```text
|
| 240 |
-
Top-level workflow: Anonimiseren | Terugzetten
|
| 241 |
-
Global presentation: Standaard | Expert
|
| 242 |
-
Core stages: Toevoegen → Controleren → Downloaden
|
| 243 |
-
One primary action per active stage
|
| 244 |
-
Progressive and conditional disclosure for settings, other formats, Scrub Key and audit evidence
|
| 245 |
-
```
|
| 246 |
|
| 247 |
-
|
| 248 |
|
| 249 |
-
|
| 250 |
|
| 251 |
```text
|
| 252 |
-
|
| 253 |
-
→
|
| 254 |
-
→
|
| 255 |
-
→
|
| 256 |
-
→
|
| 257 |
-
→
|
| 258 |
-
→
|
| 259 |
-
→
|
| 260 |
-
→
|
| 261 |
```
|
| 262 |
|
| 263 |
-
|
| 264 |
|
| 265 |
-
|
| 266 |
-
|
| 267 |
-
``
|
| 268 |
-
|
| 269 |
-
|
|
|
|
|
|
|
|
|
|
| 270 |
|
| 271 |
-
|
| 272 |
|
| 273 |
```text
|
| 274 |
-
|
| 275 |
-
+ multiple independent/nested expanders
|
| 276 |
```
|
| 277 |
|
| 278 |
-
|
| 279 |
|
| 280 |
-
|
| 281 |
|
| 282 |
-
|
| 283 |
|
| 284 |
-
``
|
| 285 |
-
Initial:
|
| 286 |
-
▼ 1 Toevoegen
|
| 287 |
-
2 Controleren — beschikbaar na verwerking
|
| 288 |
-
3 Downloaden — beschikbaar na controle
|
| 289 |
-
|
| 290 |
-
After processing:
|
| 291 |
-
✓ 1 Toevoegen — contract.docx · Juridisch
|
| 292 |
-
▼ 2 Controleren
|
| 293 |
-
3 Downloaden — beschikbaar na controle
|
| 294 |
-
|
| 295 |
-
After review:
|
| 296 |
-
✓ 1 Toevoegen — contract.docx · Juridisch
|
| 297 |
-
✓ 2 Controleren — 14 gecontroleerd · 1 handmatig toegevoegd
|
| 298 |
-
▼ 3 Downloaden
|
| 299 |
-
```
|
| 300 |
|
| 301 |
-
|
| 302 |
|
| 303 |
-
|
| 304 |
|
| 305 |
-
###
|
| 306 |
|
| 307 |
-
The
|
| 308 |
|
| 309 |
```text
|
| 310 |
-
|
| 311 |
-
|
| 312 |
-
|
| 313 |
-
|
| 314 |
-
|
| 315 |
-
|
| 316 |
-
|
| 317 |
-
7. SCRUB-WP_PREMIUM_EXPORT_STAGE_SIMPLIFICATION
|
| 318 |
-
8. SCRUB-WP_PREMIUM_EXPERT_PARITY_REGRESSION
|
| 319 |
-
9. SCRUB-WP_PREMIUM_CORE_FLOW_APP_VERIFY_CLOSEOUT
|
| 320 |
```
|
| 321 |
|
| 322 |
-
|
| 323 |
-
|
| 324 |
-
Do not combine input, review and export restructuring into one patch. Do not run shared Streamlit UI packages in parallel. Each consequential production UI candidate receives separate `governance_release_assurance` under the enforced contract.
|
| 325 |
-
|
| 326 |
-
### Phase 6 — MVP workflow validation and trust hardening
|
| 327 |
-
|
| 328 |
-
Goal:
|
| 329 |
-
|
| 330 |
-
```text
|
| 331 |
-
Use the online/web prototype and GitHub workflow to validate product behavior before pilot expansion or installer investment.
|
| 332 |
-
```
|
| 333 |
|
| 334 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 335 |
|
| 336 |
-
|
| 337 |
-
- import and export confidence;
|
| 338 |
-
- Scrub Key warning/acknowledgement verification;
|
| 339 |
-
- DOCX hygiene reporting;
|
| 340 |
-
- unified side-by-side review direction;
|
| 341 |
-
- easier replace/review logic;
|
| 342 |
-
- residual-risk and audit reporting;
|
| 343 |
-
- MVP interface clarity;
|
| 344 |
-
- professional export/download flow;
|
| 345 |
-
- Hugging Face app verification after GitHub Actions/sync when UI changes.
|
| 346 |
|
| 347 |
-
|
| 348 |
|
| 349 |
-
|
| 350 |
|
| 351 |
-
|
| 352 |
|
| 353 |
-
|
| 354 |
|
| 355 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 356 |
|
| 357 |
-
|
| 358 |
|
| 359 |
-
|
| 360 |
|
| 361 |
```text
|
| 362 |
-
|
| 363 |
-
2. SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION — draft PR #85; resume production integration only after #1 is incorporated
|
| 364 |
-
3. SCRUB-WP_PREMIUM_INPUT_STAGE_SIMPLIFICATION
|
| 365 |
-
4. SCRUB-WP_PREMIUM_REVIEW_STAGE_SIMPLIFICATION
|
| 366 |
-
5. SCRUB-WP_PREMIUM_EXPORT_STAGE_SIMPLIFICATION
|
| 367 |
-
6. SCRUB-WP_PREMIUM_EXPERT_PARITY_REGRESSION
|
| 368 |
-
7. SCRUB-WP_PREMIUM_CORE_FLOW_APP_VERIFY_CLOSEOUT
|
| 369 |
```
|
| 370 |
|
| 371 |
-
|
| 372 |
-
|
| 373 |
-
Execution principles:
|
| 374 |
|
| 375 |
-
-
|
| 376 |
-
- preserve legal meaning and keep human review mandatory;
|
| 377 |
-
- the Premium shell is presentation/state orchestration, not a license to change recognition or export semantics;
|
| 378 |
-
- one document, one workspace, three persistent stages, one active task;
|
| 379 |
-
- no three routed pages for `Toevoegen → Controleren → Downloaden` in Standard;
|
| 380 |
-
- no nested-expander recreation of the current long form;
|
| 381 |
-
- create recognizer or document-processing fixes only from reproducible safety evidence;
|
| 382 |
-
- live UI work must be sequential where it touches shared Streamlit/review/export surfaces;
|
| 383 |
-
- do not make production-readiness claims from prototype evidence.
|
| 384 |
|
| 385 |
-
##
|
| 386 |
|
| 387 |
-
|
| 388 |
|
| 389 |
-
|
| 390 |
-
select an unmasked value
|
| 391 |
-
→ right-click
|
| 392 |
-
→ choose a masking type
|
| 393 |
-
→ add through the existing manual replacement path
|
| 394 |
-
```
|
| 395 |
|
| 396 |
-
|
| 397 |
|
| 398 |
-
|
| 399 |
|
| 400 |
```text
|
| 401 |
-
|
| 402 |
-
|
| 403 |
-
|
| 404 |
-
|
| 405 |
-
|
| 406 |
-
|
|
|
|
|
|
|
|
|
|
| 407 |
```
|
| 408 |
|
| 409 |
-
|
| 410 |
-
|
| 411 |
-
Recall/benchmark work is reopened only where reproducible safety evidence requires it.
|
| 412 |
-
|
| 413 |
-
Do not start local packaging next steps such as `WP48B` or `WP49B` by default. They require explicit coordinator approval.
|
| 414 |
|
| 415 |
-
|
| 416 |
|
| 417 |
-
-
|
| 418 |
-
|
| 419 |
-
|
| 420 |
-
|
| 421 |
-
|
| 422 |
-
|
| 423 |
-
-
|
| 424 |
-
-
|
| 425 |
-
-
|
| 426 |
-
- documentation;
|
| 427 |
-
- benchmark data design;
|
| 428 |
-
- risk reviews;
|
| 429 |
-
- non-UI architecture work.
|
| 430 |
-
|
| 431 |
-
Do not run in parallel:
|
| 432 |
-
|
| 433 |
-
- multiple changes to `fix_streamlit_nested_expanders.py` or `presidio_streamlit.py`;
|
| 434 |
-
- multiple changes to the same Streamlit patch file;
|
| 435 |
-
- `serial_review_panel_ui.py` UI edits without coordination;
|
| 436 |
-
- `review_highlight_toggle_panel_ui.py` UI edits without coordination;
|
| 437 |
-
- export/download flow changes;
|
| 438 |
-
- review table flow changes;
|
| 439 |
-
- synchronized scroll implementation;
|
| 440 |
-
- custom HTML/component rendering implementation;
|
| 441 |
-
- Docker/runtime startup patch-order changes;
|
| 442 |
-
- installer/packaging work unless explicitly approved;
|
| 443 |
-
- Phase 7 follow-up while the MVP quality gate is not passed.
|
| 444 |
-
|
| 445 |
-
Use `workpackage_claims/` before starting a package.
|
| 446 |
-
|
| 447 |
-
---
|
| 448 |
-
|
| 449 |
-
## 9. Product architecture target
|
| 450 |
|
| 451 |
-
|
| 452 |
|
| 453 |
-
|
| 454 |
-
|
| 455 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 456 |
|
| 457 |
-
|
| 458 |
|
| 459 |
```text
|
| 460 |
-
|
| 461 |
-
one persistent document workspace;
|
| 462 |
-
three core Standard stages with exactly one active at a time;
|
| 463 |
-
global Standard/Expert presentation;
|
| 464 |
-
tested safety/state boundaries;
|
| 465 |
-
local-first direction;
|
| 466 |
-
and a clear primary export with secondary restore/audit layers.
|
| 467 |
```
|
| 468 |
|
| 469 |
-
Do not migrate frontend, add a full document editor, introduce cloud document processing, or alter export/Scrub Key/reinsert semantics without a dedicated approved package.
|
| 470 |
-
|
| 471 |
---
|
| 472 |
|
| 473 |
-
##
|
| 474 |
|
| 475 |
-
|
| 476 |
|
| 477 |
-
|
| 478 |
-
Hugging Face / web prototype = synthetic and approved non-confidential validation
|
| 479 |
-
Local signed desktop runtime = confidential production-oriented trust environment
|
| 480 |
-
```
|
| 481 |
|
| 482 |
-
|
| 483 |
|
| 484 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 485 |
|
| 486 |
-
|
| 487 |
|
| 488 |
-
|
| 489 |
-
one signed setup.exe for low-friction installation
|
| 490 |
-
+ one signed MSI for managed organizational deployment
|
| 491 |
-
+ Tauri Windows shell
|
| 492 |
-
+ bundled local Python/Presidio engine as a PyInstaller onedir sidecar
|
| 493 |
-
+ all required models and assets available locally
|
| 494 |
-
+ loopback-only communication on the same PC
|
| 495 |
-
+ no required cloud calls, runtime downloads or document telemetry
|
| 496 |
-
```
|
| 497 |
|
| 498 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 499 |
|
| 500 |
-
|
| 501 |
|
| 502 |
-
|
| 503 |
|
| 504 |
```text
|
| 505 |
-
|
| 506 |
-
- 60–70% of development and integration labor may be agent-executed.
|
| 507 |
-
- expected development/integration budget after agent substitution: approximately EUR 8,000–24,000.
|
| 508 |
-
- agent/build-compute allowance: approximately EUR 1,000–4,000.
|
| 509 |
-
|
| 510 |
-
Subsequent release cycles:
|
| 511 |
-
- 75–90% of repetitive build, test, packaging and release-candidate preparation may be automated.
|
| 512 |
```
|
| 513 |
|
| 514 |
-
|
| 515 |
-
|
| 516 |
-
### Agent-autonomous scope
|
| 517 |
-
|
| 518 |
-
Within isolated workstations or disposable Windows test VMs, scoped agents may autonomously:
|
| 519 |
-
|
| 520 |
-
- install approved build SDKs and packaging dependencies;
|
| 521 |
-
- inventory and pin Python packages, native libraries and model assets;
|
| 522 |
-
- build and debug PyInstaller onedir bundles;
|
| 523 |
-
- build the Tauri shell and local sidecar lifecycle;
|
| 524 |
-
- create setup.exe and MSI release candidates;
|
| 525 |
-
- implement loopback binding, health checks and shutdown cleanup;
|
| 526 |
-
- run offline, install, upgrade, uninstall, Defender and synthetic document tests;
|
| 527 |
-
- inspect network, process, filesystem, logging and temporary-file evidence;
|
| 528 |
-
- maintain CI, checksums, SBOMs, release notes and handovers;
|
| 529 |
-
- open pull requests and prepare unsigned release candidates.
|
| 530 |
-
|
| 531 |
-
All test documents and Scrub Keys must remain synthetic. Agents must not weaken review, Scrub Key, export or privacy controls to make packaging easier.
|
| 532 |
-
|
| 533 |
-
### Human-controlled gates
|
| 534 |
-
|
| 535 |
-
The following responsibilities must not be delegated as an unreviewed autonomous chain:
|
| 536 |
-
|
| 537 |
-
- legal publisher identity and long-lived code-signing authority;
|
| 538 |
-
- final production signing and public release;
|
| 539 |
-
- acceptance of privacy and security claims;
|
| 540 |
-
- approval of export, Scrub Key or document-processing semantic changes;
|
| 541 |
-
- acceptance testing by non-technical users on real managed Windows environments;
|
| 542 |
-
- independent security validation where a strong local-only product claim is made.
|
| 543 |
-
|
| 544 |
-
No single agent should simultaneously hold unrestricted repository-write, signing-identity and public-release authority. Use protected environments, least-privilege identities and an explicit human release gate.
|
| 545 |
-
|
| 546 |
-
### Indicative external and retained costs
|
| 547 |
-
|
| 548 |
-
Costs that remain materially external or human-controlled include:
|
| 549 |
-
|
| 550 |
-
```text
|
| 551 |
-
- managed Windows code signing: approximately EUR 120–300 per year;
|
| 552 |
-
- build/CI/test infrastructure: usually limited, depending on runner and artifact use;
|
| 553 |
-
- physical or managed Windows test devices: approximately EUR 0–1,500 initially;
|
| 554 |
-
- targeted independent desktop/privacy security review: approximately EUR 5,000–15,000;
|
| 555 |
-
- broader penetration testing and retest, if required: approximately EUR 15,000–30,000;
|
| 556 |
-
- human packaging/security/release oversight: retained even with extensive agent autonomy.
|
| 557 |
-
```
|
| 558 |
|
| 559 |
-
##
|
| 560 |
|
| 561 |
-
|
| 562 |
|
| 563 |
```text
|
| 564 |
-
|
| 565 |
-
|
| 566 |
-
|
| 567 |
-
|
| 568 |
-
|
| 569 |
-
6. Signed setup.exe and MSI release candidate
|
| 570 |
-
7. Managed Windows pilot, upgrade, rollback and uninstall validation
|
| 571 |
-
8. Independent security review and quality-gate closeout
|
| 572 |
```
|
| 573 |
|
| 574 |
-
|
| 575 |
|
| 576 |
-
|
| 577 |
|
| 578 |
-
|
| 579 |
-
|
| 580 |
-
The first product wedge already identifies Legal and Zorg as the most relevant Dutch professional domains. Zorg now receives an explicit profile line rather than remaining an incidental subset of general and legal recognition.
|
| 581 |
|
| 582 |
-
#
|
| 583 |
|
| 584 |
-
|
| 585 |
-
Geboortedatum: vervangen
|
| 586 |
-
Overige exacte zorgdata: controleren en standaard geselecteerd
|
| 587 |
-
Patiënt- en cliëntidentificatie: vervangen
|
| 588 |
-
Zorgverleneridentificatie: controleren en standaard geselecteerd
|
| 589 |
-
Diagnose, medicatie, dosering, labwaarden en observaties: behouden
|
| 590 |
-
Zeldzame-casus-herleidbaarheid: auditwaarschuwing, niet blind maskeren
|
| 591 |
-
```
|
| 592 |
|
| 593 |
-
The
|
| 594 |
|
| 595 |
-
|
| 596 |
-
Remove identity and patient-specific administrative references while preserving clinical meaning.
|
| 597 |
-
```
|
| 598 |
|
| 599 |
-
|
| 600 |
|
| 601 |
-
#
|
| 602 |
|
| 603 |
-
-
|
| 604 |
-
- care plans and evaluations;
|
| 605 |
-
- nursing transfers;
|
| 606 |
-
- medical specialist discharge letters;
|
| 607 |
-
- GP referrals and consultation letters;
|
| 608 |
-
- medication overviews or administration lists;
|
| 609 |
-
- laboratory reports;
|
| 610 |
-
- MIC/MIM/VIM care-incident reports.
|
| 611 |
|
| 612 |
-
|
| 613 |
|
| 614 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 615 |
|
| 616 |
-
|
| 617 |
|
| 618 |
```text
|
| 619 |
-
|
| 620 |
-
|
| 621 |
-
|
| 622 |
-
care_reference_taxonomy.py
|
| 623 |
-
dutch_care_recognizers.py
|
| 624 |
-
recognition_profiles.py
|
| 625 |
```
|
| 626 |
|
| 627 |
-
|
| 628 |
-
|
| 629 |
-
Current-engine baseline evidence is now recorded: 25 of 81 expected replace/review values were found as exact normalized spans, only 14 under the intended entity type, 11 were misclassified and 56 were missed. The bounded custom-rule baseline produced zero overlaps with the designated clinical preserve passages. Generic NER was excluded, so PERSON and e-mail results are not full-app measurements.
|
| 630 |
|
| 631 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 632 |
|
| 633 |
-
|
| 634 |
|
| 635 |
-
|
| 636 |
|
| 637 |
-
|
| 638 |
|
| 639 |
-
|
| 640 |
|
| 641 |
-
|
| 642 |
|
| 643 |
-
|
| 644 |
|
| 645 |
-
|
| 646 |
-
Zorgcontrole — streng
|
| 647 |
-
Juridische controle — streng
|
| 648 |
-
Algemene Nederlandse controle
|
| 649 |
-
Algemene internationale controle
|
| 650 |
-
```
|
| 651 |
|
| 652 |
-
|
| 653 |
|
| 654 |
```text
|
| 655 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 656 |
```
|
| 657 |
|
| 658 |
-
|
| 659 |
-
|
| 660 |
-
### Sequential workpackages
|
| 661 |
-
|
| 662 |
-
```text
|
| 663 |
-
1. SCRUB-WP_CARE_PROFILE_V1_POLICY_AND_CORPUS_FOUNDATION
|
| 664 |
-
2. SCRUB-WP_CARE_PROFILE_CURRENT_ENGINE_BASELINE
|
| 665 |
-
3. SCRUB-WP_CARE_PROFILE_GAP_TRIAGE
|
| 666 |
-
4. SCRUB-WP_CARE_PROFILE_RECOGNIZER_CONTRACT_TESTS
|
| 667 |
-
5. SCRUB-WP_CARE_PROFILE_RECOGNIZER_IMPLEMENTATION
|
| 668 |
-
6. SCRUB-WP_RECOGNITION_PROFILE_CONFIGURATION_REFACTOR
|
| 669 |
-
7. SCRUB-WP_CARE_PROFILE_CURRENT_UI_INTEGRATION
|
| 670 |
-
8. SCRUB-WP_CARE_PROFILE_CROSS_PROFILE_REGRESSION_MATRIX
|
| 671 |
-
9. SCRUB-WP_CARE_PROFILE_APP_VERIFY
|
| 672 |
-
10. SCRUB-WP_CARE_PROFILE_DESKTOP_UX_CONTRACT
|
| 673 |
-
```
|
| 674 |
|
| 675 |
-
|
| 676 |
|
| 677 |
-
|
| 678 |
-
- no blind masking of clinical meaning;
|
| 679 |
-
- no change to Scrub Key, export or reinsert semantics without a separate package;
|
| 680 |
-
- human review remains required;
|
| 681 |
-
- corpus or benchmark success does not prove production readiness;
|
| 682 |
-
- no cloud document processing is introduced.
|
|
|
|
| 1 |
+
# SolidPrivacy Scrub — Product & Development Roadmap
|
| 2 |
|
| 3 |
+
This document is the strategic roadmap for SolidPrivacy Scrub.
|
| 4 |
|
| 5 |
Use it together with:
|
| 6 |
|
| 7 |
+
- `WORKPACKAGES.md` — current executable queue, dependencies and verification gates;
|
| 8 |
+
- `RISK_REGISTER.md` — current privacy, security and product risks;
|
| 9 |
+
- `DECISION_LOG.md` — accepted architecture/product decisions;
|
| 10 |
+
- `CHANGELOG.md` — historical implementation record;
|
| 11 |
+
- `PROJECT_PROMPT.md` — worker/governance rules.
|
|
|
|
| 12 |
|
| 13 |
+
Last strategy reset: 2026-09-04 — Repository Convergence becomes the immediate execution priority before further normal feature development. After convergence, `main` becomes the active Scrub Private development line. Local/offline functionality remains recoverable and may become a separate active product only if a later business requirement justifies it.
|
| 14 |
|
| 15 |
---
|
| 16 |
|
| 17 |
+
## 1. Product objective
|
| 18 |
|
| 19 |
+
Scrub helps professionals create reviewed pseudonymised working copies of confidential documents while preserving useful legal and clinical meaning.
|
| 20 |
|
| 21 |
+
The product must:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
|
| 23 |
+
- detect and pseudonymise sensitive values;
|
| 24 |
+
- preserve professional context;
|
| 25 |
+
- keep meaningful human review mandatory where required;
|
| 26 |
+
- allow users to correct missed values;
|
| 27 |
+
- support controlled re-identification through a sensitive Scrub Key;
|
| 28 |
+
- preserve supported document structure and expose hygiene limitations;
|
| 29 |
+
- make residual risk visible rather than imply perfect automatic detection;
|
| 30 |
+
- ultimately run as a managed private service with a defensible content-handling boundary.
|
| 31 |
|
| 32 |
+
Core workflow:
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
|
| 34 |
```text
|
| 35 |
+
Scrub → Review → Scrub Key → AI / external use → Reinsert → Export → Audit
|
|
|
|
|
|
|
|
|
|
| 36 |
```
|
| 37 |
|
| 38 |
+
Scrub is primarily a pseudonymisation/privacy-processing product. A re-linkable Scrub Key means the output remains re-identifiable and must not be marketed as guaranteed irreversible anonymisation.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
---
|
| 41 |
|
| 42 |
+
## 2. Existing risk priorities remain binding
|
|
|
|
|
|
|
| 43 |
|
| 44 |
+
Repository Convergence does not demote the existing privacy/product risks.
|
| 45 |
|
| 46 |
+
Highest priorities remain:
|
|
|
|
|
|
|
| 47 |
|
| 48 |
+
1. false negatives / missed sensitive data;
|
| 49 |
+
2. Scrub Key leakage, mismatch or misuse;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
3. hidden document content and metadata leakage;
|
| 51 |
+
4. placeholder corruption during external/AI roundtrip;
|
| 52 |
+
5. review/state defects that can expose stale or insufficiently reviewed output;
|
| 53 |
+
6. domain-specific under-detection or harmful over-masking, especially Legal and Zorg;
|
| 54 |
+
7. service trust-boundary failures such as content persistence or unintended external processing.
|
| 55 |
|
| 56 |
+
Human review remains a required control. Synthetic benchmarks and tests do not by themselves establish production safety.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
|
| 58 |
---
|
| 59 |
|
| 60 |
+
## 3. Existing architecture to reuse
|
| 61 |
|
| 62 |
+
The current repository already contains substantial product architecture and validation capability. The default is reuse, not reinvention.
|
| 63 |
|
| 64 |
+
Preserve unless current evidence proves a defect:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 65 |
|
| 66 |
+
- layered Dutch/Legal/Zorg recognition and profile policy;
|
| 67 |
+
- clinical/legal context preservation;
|
| 68 |
+
- side-by-side human review and authoritative replacement decisions;
|
| 69 |
+
- direct/manual missed-value correction;
|
| 70 |
+
- generation-bound Standard/Expert workflow state and fail-closed export gating;
|
| 71 |
+
- bound Scrub Key, mapping digest and controlled reinsert;
|
| 72 |
+
- TXT/DOCX/text-PDF handling and DOCX hygiene/fidelity reporting;
|
| 73 |
+
- synthetic Legal/Zorg corpora, gold expectations, benchmark/gap-triage and E2E evidence.
|
| 74 |
|
| 75 |
+
No separate Evidence Framework, workflow engine, mapping service or replacement database is authorized merely by this roadmap.
|
| 76 |
|
| 77 |
---
|
| 78 |
|
| 79 |
+
# 4. Strategic stages
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 80 |
|
| 81 |
+
The strategic roadmap has five macro stages. Detailed implementation packages belong in `WORKPACKAGES.md`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 82 |
|
| 83 |
+
## Stage 1 — Repository Convergence — CURRENT
|
| 84 |
|
| 85 |
+
Goal:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 86 |
|
| 87 |
+
> Establish one accurate current repository truth before further normal feature development.
|
| 88 |
|
| 89 |
+
Sequence:
|
| 90 |
|
| 91 |
```text
|
| 92 |
+
Preserve exact state
|
| 93 |
+
→ reconstruct active/reachable behavior
|
| 94 |
+
→ classify capabilities and debt
|
| 95 |
+
→ reconcile known current issues
|
| 96 |
+
→ remove only proven obsolete/contradictory paths
|
| 97 |
+
→ identify canonical validation hierarchy
|
| 98 |
+
→ align canonical documentation
|
| 99 |
+
→ verify
|
| 100 |
+
→ freeze clean baseline
|
| 101 |
```
|
| 102 |
|
| 103 |
+
Rules:
|
| 104 |
|
| 105 |
+
- no source-tree clone (`app_v2`, `scrub-new`, duplicate main Streamlit implementation);
|
| 106 |
+
- exact Git SHA is the preservation authority;
|
| 107 |
+
- classification is capability-level: `CANONICAL | RECONCILE | RETIRE | VARIANT-SPECIFIC`;
|
| 108 |
+
- no refactor for aesthetics alone;
|
| 109 |
+
- no new product feature work by default;
|
| 110 |
+
- privacy/security defects discovered during convergence may be fixed as narrow root-cause packages;
|
| 111 |
+
- existing `implementation_operations` / blind `governance_release_assurance` separation remains mandatory for consequential changes;
|
| 112 |
+
- the temporary debt ledger is execution evidence, not a new permanent source of product truth.
|
| 113 |
|
| 114 |
+
Exit milestone:
|
| 115 |
|
| 116 |
```text
|
| 117 |
+
SCRUB_REPOSITORY_CONVERGED
|
|
|
|
| 118 |
```
|
| 119 |
|
| 120 |
+
Exit requires one exact clean-baseline SHA, aligned current docs/issues, green relevant regression/evidence, and no known material contradiction knowingly left active.
|
| 121 |
|
| 122 |
+
---
|
| 123 |
|
| 124 |
+
## Stage 2 — Scrub Private Application
|
| 125 |
|
| 126 |
+
Starts only after `SCRUB_REPOSITORY_CONVERGED`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 127 |
|
| 128 |
+
Goal:
|
| 129 |
|
| 130 |
+
> Reuse the existing Scrub product core while adapting only the application trust boundary required for managed private hosting.
|
| 131 |
|
| 132 |
+
### Content-plane target
|
| 133 |
|
| 134 |
+
The following are customer document content:
|
| 135 |
|
| 136 |
```text
|
| 137 |
+
uploaded document
|
| 138 |
+
extracted text
|
| 139 |
+
detected values
|
| 140 |
+
review/replacement mappings
|
| 141 |
+
processed document
|
| 142 |
+
Scrub Key
|
| 143 |
+
restored document
|
|
|
|
|
|
|
|
|
|
| 144 |
```
|
| 145 |
|
| 146 |
+
Private application policy:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 147 |
|
| 148 |
+
- no intentional persistent server-side storage of this content;
|
| 149 |
+
- no document-content logs;
|
| 150 |
+
- no document-content backup;
|
| 151 |
+
- no third-party document-processing egress;
|
| 152 |
+
- application-controlled session state is cleared when the processing interaction ends/reset occurs where the application controls that lifecycle.
|
| 153 |
|
| 154 |
+
### Variant-specific capability
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 155 |
|
| 156 |
+
Current functionality such as persistent remembered replacements or external Azure/OpenAI document processing may remain recoverable through Git/possible future Local product, but is excluded from Scrub Private unless explicitly re-approved.
|
| 157 |
|
| 158 |
+
### Hugging Face role
|
| 159 |
|
| 160 |
+
Hugging Face remains a synthetic/approved-test **application validation environment**, not the final confidential-production trust environment.
|
| 161 |
|
| 162 |
+
HF may prove application behavior under our control, such as:
|
| 163 |
|
| 164 |
+
- end-to-end Legal/Zorg flows;
|
| 165 |
+
- state integrity and stale-export blocking;
|
| 166 |
+
- no intentional application-level content persistence;
|
| 167 |
+
- no enabled third-party document-processing route;
|
| 168 |
+
- no content-bearing application logging.
|
| 169 |
|
| 170 |
+
HF does not prove provider-level snapshot, swap, backup, host-log or forensic-retention properties.
|
| 171 |
|
| 172 |
+
Exit milestone:
|
| 173 |
|
| 174 |
```text
|
| 175 |
+
SCRUB_HF_APPLICATION_COMPLETE
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 176 |
```
|
| 177 |
|
| 178 |
+
This is functional application evidence with synthetic data, not production-security certification.
|
|
|
|
|
|
|
| 179 |
|
| 180 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 181 |
|
| 182 |
+
## Stage 3 — Private Service
|
| 183 |
|
| 184 |
+
Starts only after `SCRUB_HF_APPLICATION_COMPLETE`.
|
| 185 |
|
| 186 |
+
Goal:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 187 |
|
| 188 |
+
> Add the smallest production service layer needed for a real managed private deployment.
|
| 189 |
|
| 190 |
+
Initial architecture:
|
| 191 |
|
| 192 |
```text
|
| 193 |
+
Browser
|
| 194 |
+
↓ HTTPS
|
| 195 |
+
Proven OIDC / customer identity
|
| 196 |
+
↓
|
| 197 |
+
Scrub service
|
| 198 |
+
├─ ephemeral content plane
|
| 199 |
+
└─ minimal persistent control plane
|
| 200 |
+
↓
|
| 201 |
+
User download
|
| 202 |
```
|
| 203 |
|
| 204 |
+
Control-plane data may include only necessary non-document operational information such as customer/service identity, IdP configuration, deployment version, security/login metadata, service configuration and health status.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 205 |
|
| 206 |
+
Principles:
|
| 207 |
|
| 208 |
+
- use proven OIDC/customer IdP rather than custom authentication;
|
| 209 |
+
- least privilege and non-root runtime;
|
| 210 |
+
- immutable production source/runtime behavior;
|
| 211 |
+
- no persistent customer-document store;
|
| 212 |
+
- no content backup/logging;
|
| 213 |
+
- deny unnecessary outbound network access;
|
| 214 |
+
- avoid request-time model/dependency downloads where practical;
|
| 215 |
+
- manual provisioning is acceptable until automation materially reduces burden;
|
| 216 |
+
- customer-dedicated isolation is the preferred starting hypothesis, but exact VPS/container granularity is decided from real threat/cost/assurance requirements.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 217 |
|
| 218 |
+
Avoid by default:
|
| 219 |
|
| 220 |
+
- Kubernetes;
|
| 221 |
+
- service mesh;
|
| 222 |
+
- Redis;
|
| 223 |
+
- queues;
|
| 224 |
+
- custom IAM;
|
| 225 |
+
- generalized multitenancy;
|
| 226 |
+
- billing/orchestration platforms;
|
| 227 |
+
- centralized document storage/processing services.
|
| 228 |
|
| 229 |
+
Exit milestone:
|
| 230 |
|
| 231 |
```text
|
| 232 |
+
SCRUB_PRIVATE_SERVICE_CANDIDATE
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 233 |
```
|
| 234 |
|
|
|
|
|
|
|
| 235 |
---
|
| 236 |
|
| 237 |
+
## Stage 4 — External Product & Service Assurance
|
| 238 |
|
| 239 |
+
This is separate from the internal release-assurance role used throughout development.
|
| 240 |
|
| 241 |
+
### Product-effectiveness evidence
|
|
|
|
|
|
|
|
|
|
| 242 |
|
| 243 |
+
Reuse existing Scrub evidence:
|
| 244 |
|
| 245 |
+
- synthetic Legal/Zorg corpora;
|
| 246 |
+
- canonical benchmark outputs;
|
| 247 |
+
- false-negative/false-positive and preserve/trap evidence;
|
| 248 |
+
- cross-profile regression;
|
| 249 |
+
- Scrub Key/reinsert safety;
|
| 250 |
+
- document hygiene/fidelity evidence;
|
| 251 |
+
- documented limitations and human-review controls.
|
| 252 |
|
| 253 |
+
### Service privacy/security evidence
|
| 254 |
|
| 255 |
+
Independently assess:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 256 |
|
| 257 |
+
- authentication/access control;
|
| 258 |
+
- runtime/infrastructure isolation;
|
| 259 |
+
- persistence/deletion behavior;
|
| 260 |
+
- logs and backups;
|
| 261 |
+
- outbound egress;
|
| 262 |
+
- secrets;
|
| 263 |
+
- update/recovery/incident controls;
|
| 264 |
+
- penetration-test findings;
|
| 265 |
+
- accuracy of customer-facing privacy/security claims.
|
| 266 |
|
| 267 |
+
Do not create per-document “privacy scores” or false precision.
|
| 268 |
|
| 269 |
+
Exit:
|
| 270 |
|
| 271 |
```text
|
| 272 |
+
material assurance findings closed or explicitly accepted
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 273 |
```
|
| 274 |
|
| 275 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 276 |
|
| 277 |
+
## Stage 5 — Pilot
|
| 278 |
|
| 279 |
+
Initial commercial bias:
|
| 280 |
|
| 281 |
```text
|
| 282 |
+
Legal first
|
| 283 |
+
→ observe real workflow/trust friction
|
| 284 |
+
→ fix evidence-backed issues
|
| 285 |
+
→ Zorg
|
| 286 |
+
→ scale only when justified
|
|
|
|
|
|
|
|
|
|
| 287 |
```
|
| 288 |
|
| 289 |
+
Pilot evidence may justify targeted recognizer/UX/identity/operations improvements.
|
| 290 |
|
| 291 |
+
Pilot evidence does not automatically justify a platform rewrite, broad multitenancy, batch/API architecture, dashboards or a new AI/model architecture.
|
| 292 |
|
| 293 |
+
---
|
|
|
|
|
|
|
| 294 |
|
| 295 |
+
# 5. Current execution direction
|
| 296 |
|
| 297 |
+
The only active strategic line is **Stage 1 — Repository Convergence**.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 298 |
|
| 299 |
+
The current executable queue is defined in `WORKPACKAGES.md`.
|
| 300 |
|
| 301 |
+
Do not resume historical Premium Input/Review/Export queues, installer work, pilot expansion or service infrastructure work simply because old plans/issues still mention them. Current source and evidence must justify any reopened package.
|
|
|
|
|
|
|
| 302 |
|
| 303 |
+
---
|
| 304 |
|
| 305 |
+
# 6. Validation strategy
|
| 306 |
|
| 307 |
+
Scrub already has multiple evidence generations. Convergence must identify the current release-validation hierarchy rather than create another framework.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 308 |
|
| 309 |
+
A validation path is a candidate for canonical release evidence when it:
|
| 310 |
|
| 311 |
+
1. executes against current supported product code or is intentionally a pure contract test;
|
| 312 |
+
2. uses current committed synthetic/gold material where applicable;
|
| 313 |
+
3. measures a currently meaningful risk;
|
| 314 |
+
4. is reproducibly automated;
|
| 315 |
+
5. is part of current CI/release validation or has a clear current role;
|
| 316 |
+
6. states its limitations and does not imply unsupported production safety;
|
| 317 |
+
7. is not superseded by a later path solving the same responsibility more completely.
|
| 318 |
|
| 319 |
+
Classifications:
|
| 320 |
|
| 321 |
```text
|
| 322 |
+
CANONICAL RELEASE VALIDATION
|
| 323 |
+
SUPPLEMENTAL DIAGNOSTIC
|
| 324 |
+
HISTORICAL / SUPERSEDED
|
|
|
|
|
|
|
|
|
|
| 325 |
```
|
| 326 |
|
| 327 |
+
Likely complementary current evidence includes:
|
|
|
|
|
|
|
| 328 |
|
| 329 |
+
- full Python regression suite;
|
| 330 |
+
- recognizer-backed corpus benchmark/report workflow;
|
| 331 |
+
- Zorg baseline/recognizer/cross-profile evidence;
|
| 332 |
+
- Phase-6 synthetic E2E workflow matrix;
|
| 333 |
+
- Scrub Key security/roundtrip suites;
|
| 334 |
+
- document hygiene/fidelity suites;
|
| 335 |
+
- Premium Streamlit/AppTest state suites.
|
| 336 |
|
| 337 |
+
No new production recall/precision threshold is introduced merely as part of repository cleanup.
|
| 338 |
|
| 339 |
+
---
|
| 340 |
|
| 341 |
+
# 7. Future Local product
|
| 342 |
|
| 343 |
+
Local/offline capability is preserved as a possible future product direction, not an active delivery line.
|
| 344 |
|
| 345 |
+
Do not pre-build a two-edition architecture.
|
| 346 |
|
| 347 |
+
If Local later becomes a real maintained product requirement, determine then the smallest shared-core/product-specific structure. Git history preserves currently variant-specific functionality until that requirement exists.
|
| 348 |
|
| 349 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 350 |
|
| 351 |
+
# 8. Governing principle
|
| 352 |
|
| 353 |
```text
|
| 354 |
+
PRESERVE
|
| 355 |
+
→ RECONSTRUCT
|
| 356 |
+
→ REMOVE ONLY PROVEN DEBT
|
| 357 |
+
→ RECONCILE CURRENT TRUTH
|
| 358 |
+
→ PROVE
|
| 359 |
+
→ FREEZE
|
| 360 |
+
→ ADAPT ONLY WHAT PRIVATE REQUIRES
|
| 361 |
+
→ HARDEN
|
| 362 |
+
→ ASSURE
|
| 363 |
+
→ PILOT
|
| 364 |
```
|
| 365 |
|
| 366 |
+
For every proposed change:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 367 |
|
| 368 |
+
> What concrete current problem disappears if we make this change, and is this the smallest complete way to remove it?
|
| 369 |
|
| 370 |
+
If there is no strong answer, do not add the change.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
WORKPACKAGES.md
CHANGED
|
The diff for this file is too large to render.
See raw diff
|
|
|
handover/workpackages/20260904_2315_repository_convergence_bootstrap.md
ADDED
|
@@ -0,0 +1,278 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Handover — SCRUB-WP_REPOSITORY_CONVERGENCE_BOOTSTRAP
|
| 2 |
+
|
| 3 |
+
Repository: `solidprivacy-nl/scrub`
|
| 4 |
+
Role: `implementation_operations`
|
| 5 |
+
Issue: #113
|
| 6 |
+
PR: #114
|
| 7 |
+
Branch: `wp/repository-convergence-bootstrap`
|
| 8 |
+
Starting main: `54c73e0ebf5a3a3ed7039a50596fb57694add3cd`
|
| 9 |
+
Status: `IMPLEMENTATION_REMEDIATED` — pre-final full regression green; final exact-head CI and fresh independent assurance still required.
|
| 10 |
+
|
| 11 |
+
## Summary
|
| 12 |
+
|
| 13 |
+
Started the approved Repository Convergence programme without changing Scrub runtime product behavior.
|
| 14 |
+
|
| 15 |
+
The package:
|
| 16 |
+
|
| 17 |
+
- preserved the exact pre-convergence baseline by SHA;
|
| 18 |
+
- confirmed exact-main `Tests` and GitHub→Hugging Face sync were green at package start;
|
| 19 |
+
- reconstructed current reachable architecture far enough to distinguish canonical capability, genuine debt candidates and variant-specific Private conflicts;
|
| 20 |
+
- replaced stale roadmap/workpackage routing with one current Repository Convergence line;
|
| 21 |
+
- aligned worker/governance documentation while preserving the two-role assurance model;
|
| 22 |
+
- converted DECISION_LOG and RISK_REGISTER from historical execution narratives into current-truth documents while preserving exact pre-convergence provenance;
|
| 23 |
+
- archived the complete pre-convergence CHANGELOG byte-for-byte and started a clean post-convergence changelog;
|
| 24 |
+
- added source-level contracts protecting the new current-truth model;
|
| 25 |
+
- received a fresh independent FAIL on the first candidate;
|
| 26 |
+
- remediated only the eight test/governance inconsistencies identified by that reviewer;
|
| 27 |
+
- restored full-suite green without restoring obsolete current queues or changing runtime behavior.
|
| 28 |
+
|
| 29 |
+
## Baseline preservation
|
| 30 |
+
|
| 31 |
+
Authoritative pre-convergence SHA:
|
| 32 |
+
|
| 33 |
+
```text
|
| 34 |
+
54c73e0ebf5a3a3ed7039a50596fb57694add3cd
|
| 35 |
+
```
|
| 36 |
+
|
| 37 |
+
At package start:
|
| 38 |
+
|
| 39 |
+
- GitHub main: exact SHA above;
|
| 40 |
+
- Tests workflow: success on exact main;
|
| 41 |
+
- Sync to Hugging Face Space: success on exact main.
|
| 42 |
+
|
| 43 |
+
No source clone (`app_v2`, `scrub-new`, duplicate Streamlit main) was created.
|
| 44 |
+
|
| 45 |
+
## Files added
|
| 46 |
+
|
| 47 |
+
- `REPOSITORY_CONVERGENCE_DEBT_LEDGER.md`
|
| 48 |
+
- `tests/test_repository_convergence_bootstrap_contracts.py`
|
| 49 |
+
- `workpackage_claims/SCRUB-WP_REPOSITORY_CONVERGENCE_BOOTSTRAP.md`
|
| 50 |
+
- `handover/workpackages/20260904_2315_repository_convergence_bootstrap.md`
|
| 51 |
+
- `history/CHANGELOG_PRE_CONVERGENCE_20260904.md` — exact pre-convergence CHANGELOG blob, historical/non-authoritative.
|
| 52 |
+
|
| 53 |
+
## Files changed
|
| 54 |
+
|
| 55 |
+
Canonical documentation/governance:
|
| 56 |
+
|
| 57 |
+
- `ROADMAP.md`
|
| 58 |
+
- `WORKPACKAGES.md`
|
| 59 |
+
- `PROJECT_PROMPT.md`
|
| 60 |
+
- `PROJECT_PROMPT_SHORT.md`
|
| 61 |
+
- `AGENTS.md`
|
| 62 |
+
- `DECISION_LOG.md`
|
| 63 |
+
- `RISK_REGISTER.md`
|
| 64 |
+
- `CHANGELOG.md`
|
| 65 |
+
|
| 66 |
+
Governance/contract tests remediated after independent FAIL:
|
| 67 |
+
|
| 68 |
+
- `tests/test_repository_convergence_bootstrap_contracts.py`
|
| 69 |
+
- `tests/test_mvp_document_fidelity_pr_final_contracts.py`
|
| 70 |
+
- `tests/test_premium_core_flow_ui_realignment_plan.py`
|
| 71 |
+
- `tests/test_premium_staged_workspace_decision.py`
|
| 72 |
+
- `tests/test_reinsert_auto_flow_app_verify_closeout.py`
|
| 73 |
+
|
| 74 |
+
No runtime/product source file is changed by PR #114.
|
| 75 |
+
|
| 76 |
+
## Current audit findings
|
| 77 |
+
|
| 78 |
+
### Canonical capability to preserve
|
| 79 |
+
|
| 80 |
+
- Premium staged workspace and generation-bound source/analysis/review/export state;
|
| 81 |
+
- mandatory review authority, review table and direct/manual correction;
|
| 82 |
+
- bound Scrub Key + mapping digest + fail-closed reinsert validation;
|
| 83 |
+
- Legal/Dutch and Zorg recognition/profile policy;
|
| 84 |
+
- TXT/DOCX/text-PDF handling and DOCX hygiene/fidelity boundaries;
|
| 85 |
+
- existing recognizer benchmark, domain evidence, Phase-6 E2E, Scrub Key/document suites and Premium AppTests.
|
| 86 |
+
|
| 87 |
+
### Variant-specific / Private-incompatible capability
|
| 88 |
+
|
| 89 |
+
- persistent remembered replacements (`replacement_memory.py` and Expert UI);
|
| 90 |
+
- Azure AI Language external document recognition;
|
| 91 |
+
- OpenAI/Azure synthesis;
|
| 92 |
+
- content-bearing synthesis prompt logging.
|
| 93 |
+
|
| 94 |
+
These are recorded but intentionally not changed in this Stage-1 bootstrap. Their Private disposition belongs after the clean baseline unless current evidence proves an immediate reference-environment safety defect.
|
| 95 |
+
|
| 96 |
+
### Retire candidates
|
| 97 |
+
|
| 98 |
+
- Docker still invokes `fix_streamlit_nested_expanders.py` and `fix_streamlit_pdf_text_reinsert.py` even though current Premium/direct-source markers make both exit without mutating source.
|
| 99 |
+
|
| 100 |
+
This requires an isolated proof/retirement package, not deletion in the bootstrap.
|
| 101 |
+
|
| 102 |
+
### Evidence-authority reconciliation
|
| 103 |
+
|
| 104 |
+
- current recognizer-backed recall workflow appears to be the primary automated recognition diagnostic path;
|
| 105 |
+
- older entity-scorecard/residual-risk helpers remain useful within explicit report-only limitations;
|
| 106 |
+
- Phase-6 E2E, Zorg, Scrub Key/document and Premium AppTest evidence are complementary;
|
| 107 |
+
- a narrow hierarchy package should classify existing paths as canonical release validation, supplemental diagnostic or historical/superseded without building a new framework or inventing thresholds.
|
| 108 |
+
|
| 109 |
+
### GitHub issue truth
|
| 110 |
+
|
| 111 |
+
- #112 and #109 contain independent PASS evidence for PR #111/#108, both of which are merged; these are stale as active candidate gates;
|
| 112 |
+
- #107/#106 correspond to merged child repairs and require stale-state reconciliation;
|
| 113 |
+
- #105/#96 additionally required a final consolidated deployed live retest after those repairs. No evidence found in this implementation session proves that final retest occurred. Do not falsely close it as verified.
|
| 114 |
+
|
| 115 |
+
## Independent assurance history
|
| 116 |
+
|
| 117 |
+
Fresh blind assurance reviewed exact prior head:
|
| 118 |
+
|
| 119 |
+
```text
|
| 120 |
+
ea203abb04f008a7e583387242a6f4917c72e591
|
| 121 |
+
```
|
| 122 |
+
|
| 123 |
+
against base:
|
| 124 |
+
|
| 125 |
+
```text
|
| 126 |
+
54c73e0ebf5a3a3ed7039a50596fb57694add3cd
|
| 127 |
+
```
|
| 128 |
+
|
| 129 |
+
and returned:
|
| 130 |
+
|
| 131 |
+
```text
|
| 132 |
+
FAIL
|
| 133 |
+
```
|
| 134 |
+
|
| 135 |
+
Reviewer findings:
|
| 136 |
+
|
| 137 |
+
1. exact-head `Tests` run `33920335521` was red: `8 failed, 1256 passed`;
|
| 138 |
+
2. two new convergence tests encoded brittle string/prose placement instead of semantic governance contracts;
|
| 139 |
+
3. six legacy tests still required historical Premium/fidelity/reinsert state to remain in current ROADMAP/WORKPACKAGES/CHANGELOG positions.
|
| 140 |
+
|
| 141 |
+
The reviewer also independently found:
|
| 142 |
+
|
| 143 |
+
- scope integrity good;
|
| 144 |
+
- five-stage roadmap materially coherent/minimal;
|
| 145 |
+
- still-binding product semantics materially preserved;
|
| 146 |
+
- archived pre-convergence changelog preservation appropriate;
|
| 147 |
+
- no runtime/HF/product activation occurred.
|
| 148 |
+
|
| 149 |
+
No prior verdict transfers to a changed head.
|
| 150 |
+
|
| 151 |
+
## Remediation performed
|
| 152 |
+
|
| 153 |
+
Only tests/governance contracts were changed.
|
| 154 |
+
|
| 155 |
+
### New convergence-contract repair
|
| 156 |
+
|
| 157 |
+
- stage validation now parses formal `## Stage N — ...` headings and compares the exact ordered five-stage set;
|
| 158 |
+
- Evidence Framework prohibition now checks a stable semantic prohibition once instead of requiring duplicate prose in multiple files.
|
| 159 |
+
|
| 160 |
+
### Historical fidelity contract repair
|
| 161 |
+
|
| 162 |
+
- historical completion/no-duplicate evidence is read from `history/CHANGELOG_PRE_CONVERGENCE_20260904.md` and the historical handover;
|
| 163 |
+
- current supported DOCX scope is bound to D030 in `DECISION_LOG.md`;
|
| 164 |
+
- current WORKPACKAGES no longer needs to carry the historical fidelity package.
|
| 165 |
+
|
| 166 |
+
### Premium UI contract repair
|
| 167 |
+
|
| 168 |
+
- plan/decision artifacts still protect the implemented one-document/staged-workspace semantics;
|
| 169 |
+
- current authority is bound to D041/D043 and the current shared-surface sequencing rule;
|
| 170 |
+
- obsolete Premium package names/order are explicitly not required in the current queue.
|
| 171 |
+
|
| 172 |
+
### Reinsert contract repair
|
| 173 |
+
|
| 174 |
+
- historical app-verification evidence is read from the archived changelog/claim/handover;
|
| 175 |
+
- current document-first flow is bound to D031;
|
| 176 |
+
- current fail-closed document/key binding is bound to D037;
|
| 177 |
+
- old roadmap/workpackage placement is no longer release authority.
|
| 178 |
+
|
| 179 |
+
## Validation evidence
|
| 180 |
+
|
| 181 |
+
### Initial exact-head candidate — FAIL evidence
|
| 182 |
+
|
| 183 |
+
Head:
|
| 184 |
+
|
| 185 |
+
```text
|
| 186 |
+
ea203abb04f008a7e583387242a6f4917c72e591
|
| 187 |
+
```
|
| 188 |
+
|
| 189 |
+
Tests run:
|
| 190 |
+
|
| 191 |
+
```text
|
| 192 |
+
33920335521
|
| 193 |
+
```
|
| 194 |
+
|
| 195 |
+
Result:
|
| 196 |
+
|
| 197 |
+
```text
|
| 198 |
+
8 failed, 1256 passed in 12.64s
|
| 199 |
+
```
|
| 200 |
+
|
| 201 |
+
### Remediated pre-final candidate — GREEN
|
| 202 |
+
|
| 203 |
+
Head:
|
| 204 |
+
|
| 205 |
+
```text
|
| 206 |
+
dcdfb2c84bbaafe0beca79191bee509b9607b461
|
| 207 |
+
```
|
| 208 |
+
|
| 209 |
+
PR merge candidate checked out by Actions:
|
| 210 |
+
|
| 211 |
+
```text
|
| 212 |
+
ca222802fe5a5c21213df1ed71045fee0826c4ff
|
| 213 |
+
```
|
| 214 |
+
|
| 215 |
+
Tests run:
|
| 216 |
+
|
| 217 |
+
```text
|
| 218 |
+
33922952965
|
| 219 |
+
```
|
| 220 |
+
|
| 221 |
+
Job:
|
| 222 |
+
|
| 223 |
+
```text
|
| 224 |
+
101185112554
|
| 225 |
+
```
|
| 226 |
+
|
| 227 |
+
Raw result:
|
| 228 |
+
|
| 229 |
+
```text
|
| 230 |
+
1264 passed in 12.34s
|
| 231 |
+
```
|
| 232 |
+
|
| 233 |
+
This proves the reviewer-requested test/governance remediation is green.
|
| 234 |
+
|
| 235 |
+
### Final exact-head status
|
| 236 |
+
|
| 237 |
+
Administrative claim/handover/changelog/workpackage updates follow this green run and therefore change the branch head. A final exact-head full regression is still mandatory. The final frozen SHA and run must be recorded in the PR conversation without mutating the head again.
|
| 238 |
+
|
| 239 |
+
## Hugging Face sync status
|
| 240 |
+
|
| 241 |
+
- Baseline exact-main sync before this workpackage: success.
|
| 242 |
+
- PR #114 contains no runtime product change.
|
| 243 |
+
- No candidate deployment/app verification is required before merge for this docs/governance/test-only package.
|
| 244 |
+
- After authorized merge, verify exact-main Actions and HF sync/path-ignore behavior according to the runbook.
|
| 245 |
+
|
| 246 |
+
## App verification status
|
| 247 |
+
|
| 248 |
+
Not applicable for PR #114: no product/UI/runtime behavior changes.
|
| 249 |
+
|
| 250 |
+
## Intentionally unchanged
|
| 251 |
+
|
| 252 |
+
- recognizers/profile/threshold semantics;
|
| 253 |
+
- review/include/replacement authority;
|
| 254 |
+
- export payloads, filenames and MIME types;
|
| 255 |
+
- Scrub Key schema/binding/lifecycle;
|
| 256 |
+
- reinsert behavior;
|
| 257 |
+
- document-processing behavior;
|
| 258 |
+
- runtime dependencies;
|
| 259 |
+
- Docker/Streamlit runtime product behavior;
|
| 260 |
+
- current replacement-memory/external-provider functionality;
|
| 261 |
+
- mandatory human review.
|
| 262 |
+
|
| 263 |
+
## Remaining risks / blockers
|
| 264 |
+
|
| 265 |
+
- Final administrative head must complete the full `Tests` workflow successfully.
|
| 266 |
+
- A completely fresh independent assurance worker must review that exact new frozen SHA and return PASS before merge.
|
| 267 |
+
- Prior FAIL is correctly non-transferable to the new head, and any future PASS is likewise exact-head only.
|
| 268 |
+
- The later technical cleanup queue must remain audit-derived rather than pre-invented.
|
| 269 |
+
- Parent Premium live-verification state (#105/#96) is not yet proven closed.
|
| 270 |
+
|
| 271 |
+
## Next recommended step
|
| 272 |
+
|
| 273 |
+
1. Complete final administration without runtime changes.
|
| 274 |
+
2. Run full exact-head CI.
|
| 275 |
+
3. Freeze the exact final PR head and post the raw run/job/pass count to PR #114 without changing source.
|
| 276 |
+
4. Dispatch fresh blind `governance_release_assurance` on that exact SHA.
|
| 277 |
+
5. Merge only on PASS; verify exact-main Actions/HF sync handling.
|
| 278 |
+
6. Continue with evidence-backed convergence packages derived from the ledger.
|
history/CHANGELOG_PRE_CONVERGENCE_20260904.md
ADDED
|
@@ -0,0 +1,2256 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
## 2026-08-09 23:51 Europe/Amsterdam — SCRUB-WP_REVIEW_MARKER_COMPACTION_LIVE_REGRESSION_REPAIR
|
| 2 |
+
|
| 3 |
+
Status: implementation candidate in PR #108; functional full-suite CI green; final exact-head CI and fresh blind assurance required.
|
| 4 |
+
|
| 5 |
+
Live verification after the independently PASSed/deployed Premium App Shell V2 found yellow processed-text markers shifted into neighboring ordinary text and full document-binding tokens visible instead of compact aliases. Source reconstruction confirmed one display root cause: `find_exact_highlight_spans()` normalized the entire processed document with `.strip()` before computing offsets while the renderer used the original untrimmed text. Leading DOCX/PDF whitespace therefore shifted all marker coordinates and could split a strict bound token so the compact-display segmenter could no longer recognize it.
|
| 6 |
+
|
| 7 |
+
Repair: document/processed text now retains exact outer whitespace for coordinate calculations; scalar review-cell normalization remains trimmed. Added synthetic regressions for leading whitespace, exact full-token spans, highlighted compact `[LABEL_NN]` display and marker-off compaction. No recognizer, review-authority, export, Scrub Key, reinsert or audit semantics changed.
|
| 8 |
+
|
| 9 |
+
Functional evidence: head `a9e977b2ad21ff4f5c86d405d56ebce60cafe3ec`; merge candidate `5b9af3a09519128d40b842b50fe1421b289ea2bd`; Tests #2279 / run `31337824222`, job `93306469803`; Streamlit 1.61.1; `1247 passed in 12.80s`.
|
| 10 |
+
|
| 11 |
+
Separate live finding: Dutch address spans around `Polderweg 8` absorb adjacent ordinary words and proliferate near-duplicate review rows. This is isolated in issue #107 and is intentionally not repaired in PR #108.
|
| 12 |
+
|
| 13 |
+
Governance transparency: two accidental empty `operator_triggers/.keep` adds on main were each immediately followed by clean-tree fast-forward commits restoring the exact prior repository tree; final tree delta is zero and the path is excluded from HF sync. Details are in the #106 handover.
|
| 14 |
+
|
| 15 |
+
---
|
| 16 |
+
|
| 17 |
+
## 2026-08-08 21:50 Europe/Amsterdam — SCRUB-WP_PREMIUM_APP_SHELL_POST_MERGE_STATE_REPAIR_V2 — executable fail-closed Expert export repair
|
| 18 |
+
|
| 19 |
+
Status: `IMPLEMENTATION_IN_PROGRESS` until the post-administration exact-head full-suite run is green; fresh independent assurance remains mandatory.
|
| 20 |
+
|
| 21 |
+
Reason:
|
| 22 |
+
- independent assurance issue #101 correctly returned `FAIL` on PR #99 because a real Expert review edit cleared review/export lineage but the same render still exposed document, Scrub Key and audit downloads;
|
| 23 |
+
- the same review also found that prior evidence lacked an executable Streamlit transition test.
|
| 24 |
+
|
| 25 |
+
Repair:
|
| 26 |
+
- all existing export/download controls now sit behind current `CoreFlowState.export_is_current` lineage;
|
| 27 |
+
- a real Expert review edit immediately blocks the entire downstream export surface;
|
| 28 |
+
- Expert exposes one explicit `Controle opnieuw afronden` action only when current processing is still reviewable;
|
| 29 |
+
- review is never silently auto-completed after edits;
|
| 30 |
+
- real source/processing-setting changes remain blocked until reprocessing and cannot use the review-recompletion shortcut;
|
| 31 |
+
- authoritative edited review rows remain generation-bound and survive Standard/Expert presentation switches;
|
| 32 |
+
- product export payload bytes, filenames and MIME semantics are unchanged once eligibility is current.
|
| 33 |
+
|
| 34 |
+
Executable regression:
|
| 35 |
+
- added `streamlit.testing.v1.AppTest` coverage for Standard Download → Expert → review edit → export blocked → explicit review completion → Standard with source/generation/analysis/review state preserved;
|
| 36 |
+
- added executable processing-setting invalidation coverage;
|
| 37 |
+
- added executable Expert-only `highlight` preservation coverage;
|
| 38 |
+
- CI now installs Streamlit for the test environment.
|
| 39 |
+
|
| 40 |
+
Pre-administration machine evidence:
|
| 41 |
+
- PR #104 head `d688cc3b980da63c3c3f0bfb98e990a00b746c4b`;
|
| 42 |
+
- base main `2623524c858216318d238213e37445193510fa73`;
|
| 43 |
+
- tested merge candidate `abeb56ee683fb4c97e0c2fef7786ed65c825b582`;
|
| 44 |
+
- Tests #2261 / run `31275276471`, job `93147717998`;
|
| 45 |
+
- raw checkout confirms the exact merge candidate;
|
| 46 |
+
- Streamlit `1.61.1` installed in CI;
|
| 47 |
+
- `python -m pytest -q tests` → `1244 passed in 13.89s`.
|
| 48 |
+
|
| 49 |
+
Protected semantics intentionally unchanged:
|
| 50 |
+
- recognizers/profile rules and threshold meaning;
|
| 51 |
+
- authoritative include/replacement decisions and direct masking;
|
| 52 |
+
- eligible export bytes, filenames and MIME types;
|
| 53 |
+
- Scrub Key schema/binding/lifecycle;
|
| 54 |
+
- reinsert and audit semantics;
|
| 55 |
+
- local/cloud processing boundary;
|
| 56 |
+
- mandatory human review.
|
| 57 |
+
|
| 58 |
+
Final exact-head CI is mandatory after this administration. Implementation does not self-certify or self-merge. Parent issue #96 and Premium Input Stage remain blocked pending fresh assurance, merge, deployment evidence and live app verification.
|
| 59 |
+
|
| 60 |
+
---
|
| 61 |
+
|
| 62 |
+
## 2026-08-08 20:35 Europe/Amsterdam — SCRUB-WP_PREMIUM_APP_SHELL_POST_MERGE_STATE_REPAIR
|
| 63 |
+
|
| 64 |
+
Status: `IMPLEMENTATION_IN_PROGRESS`; issue #98 / PR #99 repairs the reproduced issue #92 cross-mode source, analysis-cache and review-state defects. Standard and Expert now share authoritative source context, current-generation analysis and generation-bound review rows. Real source/processing/review changes continue to invalidate downstream state fail-closed.
|
| 65 |
+
|
| 66 |
+
Pre-administration evidence: Tests #2247 / run `31272188513`, job `93139919879`, head `69fde971783aa20d5cbcb2c13cd8a8538fa1ebe4`, merge candidate `65ca1ab73f03a2513a2004d49f190c2a233d261e`, `1240 passed in 9.65s`.
|
| 67 |
+
|
| 68 |
+
Fresh exact-head CI and independent assurance remain mandatory. Issue #96 and Premium Input Stage remain blocked.
|
| 69 |
+
|
| 70 |
+
---
|
| 71 |
+
|
| 72 |
+
## 2026-08-08 16:55 Europe/Amsterdam — SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION — repair after blind-assurance FAIL
|
| 73 |
+
|
| 74 |
+
Status: `RELEASE_CANDIDATE_READY` only after a fresh exact-head full-suite run; fresh independent assurance remains mandatory.
|
| 75 |
+
|
| 76 |
+
Independent assurance on prior head `2b04ca6260bddee07fbcf901239cee2955bd6dc7` returned `FAIL` because switching `Standaard → Expert` could silently reset an explicitly selected Zorg profile to the hard-coded Expert default Juridisch. The same repair cycle reviewed operator, threshold, entity selection, allow/deny lists and analyzer configuration as processing-affecting state.
|
| 77 |
+
|
| 78 |
+
Repair:
|
| 79 |
+
- Standard and Expert now hydrate the same persisted processing settings;
|
| 80 |
+
- entering Expert no longer uses a hard-coded recognition-profile or operator default when a valid current choice exists;
|
| 81 |
+
- returning to Standard rehydrates the Standard profile widget from the shared processing state;
|
| 82 |
+
- threshold, entity selection, allow/deny lists and analyzer/model settings are preserved across presentation-only switches;
|
| 83 |
+
- deterministic processing-generation synchronization now runs in both presentation modes;
|
| 84 |
+
- presentation-only switching with unchanged settings preserves valid downstream lineage;
|
| 85 |
+
- a real processing-affecting Expert change invalidates processed/review/export lineage fail-closed;
|
| 86 |
+
- Standard still refuses unsupported Expert-only operators without silently rewriting them.
|
| 87 |
+
|
| 88 |
+
Regression evidence before administrative finalization:
|
| 89 |
+
- GitHub Actions Tests run #2229 / ID `31263099583`, job `93116829430`;
|
| 90 |
+
- PR merge candidate `bd3c404f7075d86620272bf28c9a5192006e8209`;
|
| 91 |
+
- `python -m pytest -q tests` → `1235 passed in 14.48s`;
|
| 92 |
+
- new coverage explicitly includes Standard Zorg → Expert → Standard state preservation and real-change invalidation.
|
| 93 |
+
|
| 94 |
+
Protected semantics unchanged:
|
| 95 |
+
- recognizers/profile rules and threshold meaning;
|
| 96 |
+
- review-table/include authority and direct masking;
|
| 97 |
+
- export bytes, filenames and MIME types;
|
| 98 |
+
- Scrub Key, reinsert and audit semantics;
|
| 99 |
+
- dependencies and local/cloud processing boundary;
|
| 100 |
+
- mandatory human review.
|
| 101 |
+
|
| 102 |
+
A fresh exact-head regression is required after this administrative closeout, followed by a new blind `governance_release_assurance` review. The later Premium Input Stage package remains blocked.
|
| 103 |
+
|
| 104 |
+
---
|
| 105 |
+
|
| 106 |
+
## 2026-08-08 15:28 Europe/Amsterdam — SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION — release candidate prepared
|
| 107 |
+
|
| 108 |
+
Status: `RELEASE_CANDIDATE_READY`; final exact-head CI and fresh independent assurance required before merge.
|
| 109 |
+
|
| 110 |
+
Purpose:
|
| 111 |
+
- move the merged Premium staged-workspace architecture from helper/state primitives into the production Streamlit shell;
|
| 112 |
+
- make Standard behave as one persistent document workspace with `Toevoegen → Controleren → Downloaden` and exactly one dominant stage;
|
| 113 |
+
- preserve Expert and all existing processing/review/export/Scrub Key/reinsert/audit semantics.
|
| 114 |
+
|
| 115 |
+
Implemented:
|
| 116 |
+
- global `Anonimiseren | Terugzetten` and `Standaard | Expert` controls;
|
| 117 |
+
- persistent application-panel stage headers with active/completed/future states;
|
| 118 |
+
- compact completed summaries and passive future stages;
|
| 119 |
+
- primary `Document verwerken` and `Controle afronden` progression actions;
|
| 120 |
+
- explicit earlier-stage return;
|
| 121 |
+
- deterministic processing lineage and fail-closed downstream invalidation;
|
| 122 |
+
- current-generation analysis cache to prevent silent recognition reruns during stage navigation;
|
| 123 |
+
- cached review rows; reopening a completed Review invalidates Download until Review is explicitly completed again;
|
| 124 |
+
- Standard hides the permanent settings sidebar while Expert retains advanced controls;
|
| 125 |
+
- Standard does not silently modify Expert-only `highlight`/`synthesize` operator choices;
|
| 126 |
+
- legacy runtime source patch exits for the direct Premium shell, preventing the retired form/two-mode UI from being re-injected at startup.
|
| 127 |
+
|
| 128 |
+
Files added:
|
| 129 |
+
- `premium_streamlit_state.py`;
|
| 130 |
+
- `premium_streamlit_shell_ui.py`;
|
| 131 |
+
- `tests/test_premium_streamlit_state.py`;
|
| 132 |
+
- `tests/test_premium_app_shell_streamlit_integration.py`;
|
| 133 |
+
- implementation claim and handover.
|
| 134 |
+
|
| 135 |
+
Files changed:
|
| 136 |
+
- `premium_app_shell.py`;
|
| 137 |
+
- `presidio_streamlit.py`;
|
| 138 |
+
- `fix_streamlit_nested_expanders.py`;
|
| 139 |
+
- source-level UI contract tests updated where literal expectations described the retired shell;
|
| 140 |
+
- `WORKPACKAGES.md`, `CHANGELOG.md`, `RELEASE_NOTES.md` for closeout administration.
|
| 141 |
+
|
| 142 |
+
Validation:
|
| 143 |
+
- clean runtime/product head `0e1a5fbb3d6c3b8f8293779e598ececd6ea4aa1d`;
|
| 144 |
+
- GitHub Actions Tests #2200 / run `31259576962`, job `93108182555`;
|
| 145 |
+
- `python -m pytest -q tests` → `1225 passed in 12.55s`;
|
| 146 |
+
- final post-administration exact-head full regression still required before assurance;
|
| 147 |
+
- Hugging Face sync and live app verification are pending after PASS/merge because runtime UI changed.
|
| 148 |
+
|
| 149 |
+
Intentionally unchanged:
|
| 150 |
+
- recognizers/profile rules and threshold meaning;
|
| 151 |
+
- replacement/include authority and direct masking semantics;
|
| 152 |
+
- export bytes, filenames and MIME types;
|
| 153 |
+
- Scrub Key schema/binding/lifecycle;
|
| 154 |
+
- reinsert and audit semantics;
|
| 155 |
+
- dependencies and cloud/local-processing boundary;
|
| 156 |
+
- mandatory human review.
|
| 157 |
+
|
| 158 |
+
Next gate:
|
| 159 |
+
- freeze final exact head after this administrative closeout;
|
| 160 |
+
- full GitHub Actions regression;
|
| 161 |
+
- fresh blind `SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION_VERIFY`;
|
| 162 |
+
- no merge and no Input Stage work before independent PASS.
|
| 163 |
+
|
| 164 |
+
---
|
| 165 |
+
|
| 166 |
+
## 2026-08-08 Europe/Amsterdam — SCRUB-WP_PREMIUM_STAGED_WORKSPACE_DECISION_FREEZE — staged workspace architecture candidate
|
| 167 |
+
|
| 168 |
+
Status: `IMPLEMENTATION_IN_PROGRESS`; exact-head CI and independent assurance required before merge.
|
| 169 |
+
|
| 170 |
+
Purpose:
|
| 171 |
+
- convert the coordinator-approved first-principles comparison of three routed screens versus a one-page staged workspace into a binding Premium UI architecture decision;
|
| 172 |
+
- intervene before PR #85 enters production `presidio_streamlit.py` integration;
|
| 173 |
+
- make the Premium staged-workspace sequence the authoritative active execution queue.
|
| 174 |
+
|
| 175 |
+
Decision:
|
| 176 |
+
```text
|
| 177 |
+
One document. One workspace. Three stages. One active task.
|
| 178 |
+
|
| 179 |
+
Toevoegen → Controleren → Downloaden
|
| 180 |
+
```
|
| 181 |
+
|
| 182 |
+
Binding Standard behavior:
|
| 183 |
+
- all three stage headers remain in one persistent page/workspace;
|
| 184 |
+
- exactly one stage is expanded/dominant at a time;
|
| 185 |
+
- completed stages collapse to compact status summaries;
|
| 186 |
+
- future stages remain visible but passive;
|
| 187 |
+
- successful completion auto-advances to the next stage;
|
| 188 |
+
- deliberate return to an earlier stage remains possible;
|
| 189 |
+
- processing-affecting earlier changes invalidate processed/review/export lineage fail-closed;
|
| 190 |
+
- three separate routed pages are rejected for these core stages;
|
| 191 |
+
- a classic nested-expander form is also rejected as the default Standard pattern.
|
| 192 |
+
|
| 193 |
+
First-principles rationale:
|
| 194 |
+
- Scrub is an iterative document-review workflow, not a strictly linear checkout;
|
| 195 |
+
- the interface should represent document state rather than page navigation;
|
| 196 |
+
- the staged workspace preserves one-task-at-a-time focus while retaining document identity, progress, correction context and visible trust state;
|
| 197 |
+
- it aligns directly with the merged `premium_core_flow_state.py` generation/invalidation model and reduces routing/state-restoration complexity in Streamlit.
|
| 198 |
+
|
| 199 |
+
Current PR #85 consequence:
|
| 200 |
+
- PR #85 is amended, not discarded;
|
| 201 |
+
- current pure `premium_app_shell.py` helper work remains reusable;
|
| 202 |
+
- production Streamlit integration is gated until this decision is independently assured and incorporated;
|
| 203 |
+
- PR #85 must add/test completed/future/active panels, compact summaries, auto-progression hooks, prior-stage return/edit and no-three-page semantics before production integration.
|
| 204 |
+
|
| 205 |
+
Files added/changed in the candidate:
|
| 206 |
+
- added `PREMIUM_STAGED_WORKSPACE_DECISION.md`;
|
| 207 |
+
- updated `ROADMAP.md`;
|
| 208 |
+
- updated `WORKPACKAGES.md` with an authoritative current Premium queue while retaining older content as historical records;
|
| 209 |
+
- updated `DECISION_LOG.md` with D043;
|
| 210 |
+
- updated `CHANGELOG.md`;
|
| 211 |
+
- added `tests/test_premium_staged_workspace_decision.py`;
|
| 212 |
+
- added implementation claim and handover administration.
|
| 213 |
+
|
| 214 |
+
Validation boundary:
|
| 215 |
+
- documentation/architecture/contract-only package;
|
| 216 |
+
- no production Streamlit or runtime product behavior changes;
|
| 217 |
+
- no recognizer, threshold, replacement, review authority, export bytes/names/MIME, Scrub Key, reinsert, audit, dependency or Hugging Face behavior changes;
|
| 218 |
+
- Hugging Face sync: not applicable;
|
| 219 |
+
- app verification: not applicable;
|
| 220 |
+
- exact-head GitHub Actions and independent `governance_release_assurance` remain required before merge.
|
| 221 |
+
|
| 222 |
+
---
|
| 223 |
+
|
| 224 |
+
## 2026-08-07 Europe/Amsterdam — Issue #70 Actions evidence recovery repair
|
| 225 |
+
|
| 226 |
+
Status: `RELEASE_CANDIDATE_READY`; independent exact-head assurance pending.
|
| 227 |
+
|
| 228 |
+
- Replaced PR #73's expired 2026-06-17 carrier dependency with a purpose-built read-only no-op carrier workflow.
|
| 229 |
+
- Demonstrated current connector executability: carrier run `31216068355` was rerun successfully as attempt 2; latest rerun job `92989859101` concluded `success`.
|
| 230 |
+
- Full PR regression run `31216068325` / #2115 passed all `1170` tests in `11.00s` using the unchanged `python -m pytest -q tests` command.
|
| 231 |
+
- Strengthened the workflow contract tests to freeze carrier safety, rerun gating, default-branch checkout semantics and unchanged full-suite behavior.
|
| 232 |
+
- No recognizer, review, replacement, export, Scrub Key, reinsert, UI, document-processing, dependency or Hugging Face product behavior changed.
|
| 233 |
+
- Fresh blind `governance_release_assurance` remains mandatory for the final exact PR head before merge; issue #70 stays open until independent exact-main post-merge evidence is confirmed.
|
| 234 |
+
|
| 235 |
+
## 2026-08-06 21:30 Europe/Amsterdam — SCRUB-WP_TWO_ROLE_GOVERNANCE_ADOPTION_VERIFY / SCRUB-WP_PROCESSED_TEXT_SELECTION_CROSS_FLOW_REGRESSION_VERIFY
|
| 236 |
+
|
| 237 |
+
Status: both initial assurance decisions `PASS`; PR #69 merged unchanged; post-action closeout remains `ACTION_EXECUTED_UNVERIFIED` because no distinct Actions push run on the actual merged SHA was observable.
|
| 238 |
+
|
| 239 |
+
Purpose:
|
| 240 |
+
- independently reconstruct and decide the two verification packages under issue #70;
|
| 241 |
+
- preserve the blind-review boundary before reading implementation handovers, claims or conclusions;
|
| 242 |
+
- merge the passing candidate without assurance-side repair;
|
| 243 |
+
- record separate verification claims and handovers.
|
| 244 |
+
|
| 245 |
+
Assurance decisions:
|
| 246 |
+
- `SCRUB-WP_TWO_ROLE_GOVERNANCE_ADOPTION_VERIFY`: `PASS`;
|
| 247 |
+
- `SCRUB-WP_PROCESSED_TEXT_SELECTION_CROSS_FLOW_REGRESSION_VERIFY`: `PASS`.
|
| 248 |
+
|
| 249 |
+
Action and evidence:
|
| 250 |
+
- candidate head: `41bf09abe3966ae40a51c526d162c57a824557e8`;
|
| 251 |
+
- tested merge candidate: `13d55b6d74ad6f31446e16bcad0794abea32f9e7`;
|
| 252 |
+
- raw run #2105 / ID `31091265208`: `1165 passed in 12.41s`;
|
| 253 |
+
- actual merge commit: `07cf12d0cfa57dd81bd2c964cb081e5f8f8c4f71`;
|
| 254 |
+
- actual and tested merge candidates have identical parents and identical tree `4c993cbed86eade252cec6799f7dae5919b84085`;
|
| 255 |
+
- candidate repairs: none.
|
| 256 |
+
|
| 257 |
+
Files added:
|
| 258 |
+
- `workpackage_claims/scrub_wp_two_role_governance_adoption_verify.md`;
|
| 259 |
+
- `workpackage_claims/scrub_wp_processed_text_selection_cross_flow_regression_verify.md`;
|
| 260 |
+
- `handover/workpackages/20260806_2130_two_role_governance_adoption_verify.md`;
|
| 261 |
+
- `handover/workpackages/20260806_2130_processed_text_selection_cross_flow_regression_verify.md`.
|
| 262 |
+
|
| 263 |
+
Files changed:
|
| 264 |
+
- `WORKPACKAGES.md`;
|
| 265 |
+
- `CHANGELOG.md`.
|
| 266 |
+
|
| 267 |
+
Validation:
|
| 268 |
+
- implementation handovers and claims were opened only after the two initial decisions and were administratively complete;
|
| 269 |
+
- no distinct GitHub Actions push run on merged SHA `07cf12d0cfa57dd81bd2c964cb081e5f8f8c4f71` was observable at closeout-writing time, so `OUTCOME_CONFIRMED` is not claimed;
|
| 270 |
+
- Hugging Face sync: not applicable because no runtime file changed and all changed paths are ignored by the sync workflow;
|
| 271 |
+
- app verification: not applicable because no UI behavior changed.
|
| 272 |
+
|
| 273 |
+
Intentionally not changed:
|
| 274 |
+
- candidate source, tests or candidate head;
|
| 275 |
+
- production Python, Streamlit, frontend, runtime, dependency or deployment behavior;
|
| 276 |
+
- recognizers, review semantics, exports, Scrub Key, reinsert or audit semantics;
|
| 277 |
+
- human-review and local-processing boundaries.
|
| 278 |
+
|
| 279 |
+
## 2026-08-06 11:37 Europe/Amsterdam — SCRUB-WP_TWO_ROLE_GOVERNANCE_ADOPTION / SCRUB-WP_PROCESSED_TEXT_SELECTION_CROSS_FLOW_REGRESSION
|
| 280 |
+
|
| 281 |
+
Status: implementation `RELEASE_CANDIDATE_READY`; GitHub Actions PR #69 run #2097 green (`1165 passed in 9.62s`); independent governance assurance pending.
|
| 282 |
+
|
| 283 |
+
Purpose:
|
| 284 |
+
- adopt the Weekly ETF donor's canonical implementation-versus-release-assurance model for Scrub;
|
| 285 |
+
- strengthen it with a blind-review boundary so the assurance worker does not read implementation conclusions before its initial decision;
|
| 286 |
+
- add end-to-end synthetic regression evidence from processed-text selection commit through exports, bound Scrub Key, TXT/DOCX reinsert and audit outputs.
|
| 287 |
+
|
| 288 |
+
Files added:
|
| 289 |
+
- `control/PROJECT_GOVERNANCE_BOOTSTRAP.md`;
|
| 290 |
+
- `control/SCRUB_RELEASE_ASSURANCE_CONTRACT_V1.md`;
|
| 291 |
+
- `PROCESSED_TEXT_SELECTION_CROSS_FLOW_REGRESSION.md`;
|
| 292 |
+
- `tests/test_processed_text_selection_cross_flow_regression.py`;
|
| 293 |
+
- implementation claim and handover files.
|
| 294 |
+
|
| 295 |
+
Files changed:
|
| 296 |
+
- `PROJECT_PROMPT.md`;
|
| 297 |
+
- `PROJECT_PROMPT_SHORT.md`;
|
| 298 |
+
- `ROADMAP.md`;
|
| 299 |
+
- `WORKPACKAGES.md`;
|
| 300 |
+
- `DECISION_LOG.md`;
|
| 301 |
+
- `CHANGELOG.md`.
|
| 302 |
+
|
| 303 |
+
Regression coverage:
|
| 304 |
+
- selection-created row provenance, binding, all-exact occurrence count and authoritative include state;
|
| 305 |
+
- processed text/TXT export;
|
| 306 |
+
- original-DOCX replacement path;
|
| 307 |
+
- schema-1.1 bound Scrub Key generation and fail-closed custom-text behavior;
|
| 308 |
+
- verified TXT and DOCX reinsert;
|
| 309 |
+
- replacement CSV and scrub-report audit evidence;
|
| 310 |
+
- local-only/no-AI/no-cloud metadata.
|
| 311 |
+
|
| 312 |
+
Validation:
|
| 313 |
+
- local execution unavailable in this connector-only session;
|
| 314 |
+
- GitHub Actions PR #69 run #2097: `1165 passed in 9.62s`;
|
| 315 |
+
- independent governance decisions deliberately not issued by implementation.
|
| 316 |
+
|
| 317 |
+
Intentionally not changed:
|
| 318 |
+
- production Python/Streamlit behavior;
|
| 319 |
+
- recognizers, profiles, thresholds or review decisions;
|
| 320 |
+
- export bytes, filenames or MIME types;
|
| 321 |
+
- Scrub Key schema/binding or reinsert semantics;
|
| 322 |
+
- audit semantics, dependencies, runtime or Hugging Face app behavior.
|
| 323 |
+
|
| 324 |
+
## 2026-08-05 10:49 Europe/Amsterdam — SCRUB-WP_PREMIUM_CORE_FLOW_UI_REALIGNMENT_PLAN
|
| 325 |
+
|
| 326 |
+
Status: completed planning/design-only.
|
| 327 |
+
|
| 328 |
+
Purpose:
|
| 329 |
+
- Convert new live-app UX evidence into a structural interface strategy rather than another isolated decluttering patch.
|
| 330 |
+
|
| 331 |
+
Result:
|
| 332 |
+
- added `PREMIUM_CORE_FLOW_UI_REALIGNMENT_PLAN.md`;
|
| 333 |
+
- documented the gap between the implemented review-local Basic/Expert split and an application-wide presentation model;
|
| 334 |
+
- defined top-level Anonymize/Reinsert workflows;
|
| 335 |
+
- defined global Standard/Expert visibility and grouping;
|
| 336 |
+
- defined one-active-stage `Toevoegen → Controleren → Downloaden` behavior;
|
| 337 |
+
- defined no permanent settings sidebar in Standard;
|
| 338 |
+
- defined one recommended document download with other formats, Scrub Key and audit evidence secondary;
|
| 339 |
+
- sequenced contract, pure state model, app shell, input, review, export, Expert parity and app verification packages;
|
| 340 |
+
- added planning contract tests;
|
| 341 |
+
- changed no runtime product behavior.
|
| 342 |
+
|
| 343 |
+
Intentionally not changed:
|
| 344 |
+
- Streamlit product code;
|
| 345 |
+
- recognizers, profiles, replacement logic or review decisions;
|
| 346 |
+
- export payloads, filenames or MIME types;
|
| 347 |
+
- Scrub Key or reinsert semantics;
|
| 348 |
+
- runtime, dependencies or cloud-processing boundaries.
|
| 349 |
+
|
| 350 |
+
## 2026-08-05 10:49 Europe/Amsterdam — SCRUB-WP_BOUND_PLACEHOLDER_DISPLAY_COMPACTION_APP_VERIFY_CLOSEOUT
|
| 351 |
+
|
| 352 |
+
Status: completed; deployment and live app verification green.
|
| 353 |
+
|
| 354 |
+
Validation:
|
| 355 |
+
- PR #66 merged as `74b7a15ee74f6330f7fc37892b65246c1a61afaf`;
|
| 356 |
+
- final run #2080: 1155 tests passed in 12.44s;
|
| 357 |
+
- deployment run #2082: 4/4 files exact, Space health `ok`, root HTTP 200, frontend tests passed and 1155 tests passed in 11.49s;
|
| 358 |
+
- coordinator/user confirmed shorter replacement codes are visible and working.
|
| 359 |
+
|
| 360 |
+
Intentionally not changed:
|
| 361 |
+
- full bound tokens in exports;
|
| 362 |
+
- 80-bit binding entropy;
|
| 363 |
+
- Scrub Key, reinsert or export semantics;
|
| 364 |
+
- human-review and production-readiness boundaries.
|
| 365 |
+
|
| 366 |
+
## 2026-08-04 22:22 Europe/Amsterdam — SCRUB-WP_PROCESSED_TEXT_SELECTION_APP_VERIFY_CLOSEOUT
|
| 367 |
+
|
| 368 |
+
Status: completed; deployment and live app verification green.
|
| 369 |
+
|
| 370 |
+
Purpose:
|
| 371 |
+
- Close the processed-text selection integration after independent deployment checks and coordinator/user browser verification.
|
| 372 |
+
|
| 373 |
+
Validation:
|
| 374 |
+
- PR #63 merged as `53fad202ae88a97b1ea476a9c3ba787932cd62ae`;
|
| 375 |
+
- final merge-candidate run #2051: 1146 tests passed in 11.59s;
|
| 376 |
+
- independent deployment run #2064: 11/11 runtime/component files matched Hugging Face, Space health was `ok`, root returned HTTP 200, frontend tests passed, and 1146 Python tests passed in 11.47s;
|
| 377 |
+
- coordinator/user confirmed `Het werkt.` with a deployed-app screenshot.
|
| 378 |
+
|
| 379 |
+
Observed follow-up:
|
| 380 |
+
- bound placeholders remain safe but visually noisy because the same 80-bit binding ID is repeated;
|
| 381 |
+
- four-character binding IDs are rejected as an unsafe weakening;
|
| 382 |
+
- display-only compaction is routed to `SCRUB-WP_BOUND_PLACEHOLDER_DISPLAY_COMPACTION`.
|
| 383 |
+
|
| 384 |
+
Intentionally not changed:
|
| 385 |
+
- product code, placeholder grammar and binding entropy;
|
| 386 |
+
- export, Scrub Key, reinsert, recognizers, profiles or dependencies;
|
| 387 |
+
- production-readiness and human-review boundaries.
|
| 388 |
+
|
| 389 |
+
## 2026-08-04 01:34 Europe/Amsterdam — SCRUB-WP_PROCESSED_TEXT_SELECTION_TABLE_INTEGRATION
|
| 390 |
+
|
| 391 |
+
Status: completed in GitHub; deployment and app verification pending.
|
| 392 |
+
|
| 393 |
+
Purpose:
|
| 394 |
+
- Make direct selection in `Verwerkte tekst` a safe input route into the existing manual replacement-table path.
|
| 395 |
+
|
| 396 |
+
Files added:
|
| 397 |
+
- `processed_text_selection_integration.py`
|
| 398 |
+
- `tests/test_processed_text_selection_integration.py`
|
| 399 |
+
- `tests/test_processed_text_selection_table_integration_contract.py`
|
| 400 |
+
- `PROCESSED_TEXT_SELECTION_TABLE_INTEGRATION.md`
|
| 401 |
+
- `workpackage_claims/scrub_wp_processed_text_selection_table_integration.md`
|
| 402 |
+
- `handover/workpackages/20260804_0134_processed_text_selection_table_integration.md`
|
| 403 |
+
|
| 404 |
+
Files changed:
|
| 405 |
+
- `processed_text_selection_component.py`
|
| 406 |
+
- `side_by_side_review_panel_ui.py`
|
| 407 |
+
- `presidio_streamlit.py`
|
| 408 |
+
- `PROCESSED_TEXT_SELECTION_MASKING_CONTRACT.md`
|
| 409 |
+
- `ROADMAP.md`
|
| 410 |
+
- `WORKPACKAGES.md`
|
| 411 |
+
- `RELEASE_NOTES.md`
|
| 412 |
+
- `RISK_REGISTER.md`
|
| 413 |
+
|
| 414 |
+
Implementation:
|
| 415 |
+
- promoted the local component wrapper while retaining the isolated spike alias;
|
| 416 |
+
- added interactive review with environment-controlled and exception-safe static fallback;
|
| 417 |
+
- added server-protected spans independent of marker visibility;
|
| 418 |
+
- processed inspect/commit only after the editable table and before serial review/export;
|
| 419 |
+
- appended one bound normal manual row through document-scoped `manual_mask_rows`;
|
| 420 |
+
- added immediate reruns, feedback and safe one-step undo;
|
| 421 |
+
- blocked undo when the visible table row was subsequently edited;
|
| 422 |
+
- retained the full review table and existing manual form.
|
| 423 |
+
|
| 424 |
+
Validation:
|
| 425 |
+
- cleanup run #2044: frontend component tests passed; two obsolete phase-status assertions failed in Python only;
|
| 426 |
+
- final clean PR run #2047: 1146 Python tests passed in 11.97s;
|
| 427 |
+
- Hugging Face sync and live app verification pending.
|
| 428 |
+
|
| 429 |
+
Intentionally not changed:
|
| 430 |
+
- all-exact version-one scope;
|
| 431 |
+
- export formats, filenames, MIME or download behavior;
|
| 432 |
+
- Scrub Key schema, binding, digest, warning or lifecycle;
|
| 433 |
+
- reinsert semantics;
|
| 434 |
+
- recognizers, profiles, thresholds or dependencies;
|
| 435 |
+
- external network, telemetry, browser persistence or cloud processing.
|
| 436 |
+
|
| 437 |
+
## 2026-08-04 01:00 Europe/Amsterdam — SCRUB-WP_PROCESSED_TEXT_SELECTION_COMPONENT_SPIKE
|
| 438 |
+
|
| 439 |
+
Status: completed; technical validation green.
|
| 440 |
+
|
| 441 |
+
Purpose:
|
| 442 |
+
- Prove the browser/component layer before connecting selection events to the production replacement table.
|
| 443 |
+
|
| 444 |
+
Files added:
|
| 445 |
+
- `processed_text_selection_component.py`
|
| 446 |
+
- `processed_text_selection_component_spike_demo.py`
|
| 447 |
+
- `frontend/processed_text_selection_component/index.html`
|
| 448 |
+
- `frontend/processed_text_selection_component/styles.css`
|
| 449 |
+
- `frontend/processed_text_selection_component/streamlit_bridge.js`
|
| 450 |
+
- `frontend/processed_text_selection_component/component_core.js`
|
| 451 |
+
- `frontend/processed_text_selection_component/component.js`
|
| 452 |
+
- `frontend/processed_text_selection_component/NOTICE.md`
|
| 453 |
+
- `tests/test_processed_text_selection_component_spike.py`
|
| 454 |
+
- `tests/frontend/processed_text_selection_component_core.test.js`
|
| 455 |
+
- `PROCESSED_TEXT_SELECTION_COMPONENT_SPIKE.md`
|
| 456 |
+
- `workpackage_claims/scrub_wp_processed_text_selection_component_spike.md`
|
| 457 |
+
- `handover/workpackages/20260804_0100_processed_text_selection_component_spike.md`
|
| 458 |
+
|
| 459 |
+
Files changed:
|
| 460 |
+
- `PROCESSED_TEXT_SELECTION_MASKING_CONTRACT.md`
|
| 461 |
+
- `ROADMAP.md`
|
| 462 |
+
- `WORKPACKAGES.md`
|
| 463 |
+
- `RISK_REGISTER.md`
|
| 464 |
+
|
| 465 |
+
Implementation:
|
| 466 |
+
- local lazy Streamlit Components v1 wrapper on pinned Streamlit 1.39;
|
| 467 |
+
- Python code-point highlight validation and conversion to UTF-16 browser offsets;
|
| 468 |
+
- safe text-node rendering and marked segments;
|
| 469 |
+
- DOM selection offsets across plain and marked text nodes;
|
| 470 |
+
- synchronized scrolling and scroll restoration;
|
| 471 |
+
- right-click, keyboard and visible selection entry points;
|
| 472 |
+
- accessible type/confirmation menu;
|
| 473 |
+
- inspect event, server result and commit-intent transport;
|
| 474 |
+
- standalone synthetic inspection-only demo;
|
| 475 |
+
- no runtime build step, external assets, network, storage or telemetry.
|
| 476 |
+
|
| 477 |
+
Validation:
|
| 478 |
+
- run #1977: 1126 tests passed in 13.83s;
|
| 479 |
+
- Streamlit 1.39 smoke run #1979: 1126 tests passed in 13.79s;
|
| 480 |
+
- AppTest completed without script exceptions;
|
| 481 |
+
- local server health `ok`, root HTML and startup log checks passed;
|
| 482 |
+
- clean post-governance standard run #1989: 1126 tests passed in 10.87s.
|
| 483 |
+
|
| 484 |
+
Intentionally not changed:
|
| 485 |
+
- `presidio_streamlit.py` and production side-by-side renderer;
|
| 486 |
+
- replacement table, exports, Scrub Key or reinsert;
|
| 487 |
+
- recognizers, profiles, dependencies or cloud processing;
|
| 488 |
+
- occurrence-specific masking.
|
| 489 |
+
|
| 490 |
+
## 2026-08-04 00:30 Europe/Amsterdam — SCRUB-WP_PROCESSED_TEXT_SELECTION_MASKING_ACTION_MODEL
|
| 491 |
+
|
| 492 |
+
Status: completed; validation green.
|
| 493 |
+
|
| 494 |
+
Purpose:
|
| 495 |
+
- Implement the server-authoritative selection action model before any browser or Streamlit integration.
|
| 496 |
+
|
| 497 |
+
Files added:
|
| 498 |
+
- `selection_mask_action.py`
|
| 499 |
+
- `tests/test_selection_mask_action.py`
|
| 500 |
+
- `PROCESSED_TEXT_SELECTION_MASKING_ACTION_MODEL.md`
|
| 501 |
+
- `workpackage_claims/scrub_wp_processed_text_selection_masking_action_model.md`
|
| 502 |
+
- `handover/workpackages/20260804_0030_processed_text_selection_masking_action_model.md`
|
| 503 |
+
|
| 504 |
+
Files changed:
|
| 505 |
+
- `manual_mask_entry.py`
|
| 506 |
+
- `PROCESSED_TEXT_SELECTION_MASKING_CONTRACT.md`
|
| 507 |
+
- `ROADMAP.md`
|
| 508 |
+
- `WORKPACKAGES.md`
|
| 509 |
+
- `RISK_REGISTER.md`
|
| 510 |
+
|
| 511 |
+
Implementation:
|
| 512 |
+
- strict schema/action/event/scope/hash/payload parsing;
|
| 513 |
+
- exact UTF-16-to-Python index conversion and surrogate-pair rejection;
|
| 514 |
+
- quick-selection length, line, control, placeholder and marked-range checks;
|
| 515 |
+
- exact occurrence and Unicode-aware embedded-token analysis;
|
| 516 |
+
- duplicate and nested included-replacement collision guards;
|
| 517 |
+
- ready, confirmation-required and blocked inspection results;
|
| 518 |
+
- bounded replay state and opaque single-use inspections;
|
| 519 |
+
- commit-time revalidation of document, binding, source, processed text, replacement table and impact;
|
| 520 |
+
- existing document-bound manual-row construction for eight internal quick types;
|
| 521 |
+
- stable manual action records and fail-closed undo.
|
| 522 |
+
|
| 523 |
+
Validation:
|
| 524 |
+
- initial run #1957 failed from one local event-ID assignment defect and one overly literal test; no safety boundary was weakened;
|
| 525 |
+
- corrected run #1961: 1106 tests passed in 10.66s;
|
| 526 |
+
- clean standard run #1970: 1106 tests passed in 10.71s.
|
| 527 |
+
|
| 528 |
+
Intentionally not changed:
|
| 529 |
+
- visible manual form options;
|
| 530 |
+
- Streamlit, browser component or session state;
|
| 531 |
+
- review-table and export/download flows;
|
| 532 |
+
- Scrub Key, reinsert, recognizers, profiles or cloud processing;
|
| 533 |
+
- occurrence-specific masking or dependencies.
|
| 534 |
+
|
| 535 |
+
## 2026-08-04 00:09 Europe/Amsterdam — SCRUB-WP_PROCESSED_TEXT_SELECTION_MASKING_CONTRACT
|
| 536 |
+
|
| 537 |
+
Status: completed; validation green.
|
| 538 |
+
|
| 539 |
+
Purpose:
|
| 540 |
+
- Convert the approved processed-text selection direction into an exact implementation contract before product code changes.
|
| 541 |
+
|
| 542 |
+
Files added:
|
| 543 |
+
- `PROCESSED_TEXT_SELECTION_MASKING_CONTRACT.md`
|
| 544 |
+
- `test_cases/processed_text_selection_masking/contract.json`
|
| 545 |
+
- `tests/test_processed_text_selection_masking_contract.py`
|
| 546 |
+
- `workpackage_claims/scrub_wp_processed_text_selection_masking_contract.md`
|
| 547 |
+
- `handover/workpackages/20260804_0009_processed_text_selection_masking_contract.md`
|
| 548 |
+
|
| 549 |
+
Files changed:
|
| 550 |
+
- `PROCESSED_TEXT_SELECTION_MASKING_PLAN.md`
|
| 551 |
+
- `ROADMAP.md`
|
| 552 |
+
- `WORKPACKAGES.md`
|
| 553 |
+
- `RISK_REGISTER.md`
|
| 554 |
+
- `DECISION_LOG.md`
|
| 555 |
+
|
| 556 |
+
Contract result:
|
| 557 |
+
- froze a two-stage `inspect_selection` / `commit_manual_mask` protocol so occurrence impact is server-derived before commitment;
|
| 558 |
+
- froze all-exact scope, UTF-16 offset semantics and strict stale/replay handling;
|
| 559 |
+
- froze 1–5 ready, 6–20 confirmation-required and >20 blocked impact bands;
|
| 560 |
+
- froze 160-code-point single-line selections and an 8192-byte payload cap;
|
| 561 |
+
- froze eight broad quick types with server-owned entity and placeholder mappings;
|
| 562 |
+
- froze embedded-substring, nested replacement and marked-range collision blocking;
|
| 563 |
+
- froze visible, right-click and keyboard access plus one-step undo behavior;
|
| 564 |
+
- froze no-network, no-browser-persistence, escaped-rendering and fail-closed boundaries;
|
| 565 |
+
- authorized only the pure action-model package next.
|
| 566 |
+
|
| 567 |
+
Validation:
|
| 568 |
+
- machine-readable contract fixture and contract tests added;
|
| 569 |
+
- clean contract run #1954: 1027 tests passed in 11.48s;
|
| 570 |
+
- Hugging Face sync not functionally relevant;
|
| 571 |
+
- app verification not applicable because no runtime behavior changed.
|
| 572 |
+
|
| 573 |
+
Intentionally not changed:
|
| 574 |
+
- `manual_mask_entry.py`, `presidio_streamlit.py` or side-by-side renderer;
|
| 575 |
+
- Streamlit/dependency versions or Docker runtime;
|
| 576 |
+
- review table, export, Scrub Key or reinsert semantics;
|
| 577 |
+
- recognizers, profiles, cloud processing or product claims.
|
| 578 |
+
|
| 579 |
+
## 2026-08-03 23:42 Europe/Amsterdam — SCRUB-WP_PROCESSED_TEXT_SELECTION_MASKING_PLAN
|
| 580 |
+
|
| 581 |
+
Status: completed as planning; validation green.
|
| 582 |
+
|
| 583 |
+
Purpose:
|
| 584 |
+
- Assess a document-centric path for correcting missed sensitive values directly from the processed-text pane.
|
| 585 |
+
|
| 586 |
+
Files added:
|
| 587 |
+
- `PROCESSED_TEXT_SELECTION_MASKING_PLAN.md`
|
| 588 |
+
- `tests/test_processed_text_selection_masking_plan.py`
|
| 589 |
+
- `workpackage_claims/scrub_wp_processed_text_selection_masking_plan.md`
|
| 590 |
+
- `handover/workpackages/20260803_2342_processed_text_selection_masking_plan.md`
|
| 591 |
+
|
| 592 |
+
Files changed:
|
| 593 |
+
- `ROADMAP.md`
|
| 594 |
+
- `WORKPACKAGES.md`
|
| 595 |
+
- `RISK_REGISTER.md`
|
| 596 |
+
- `SIDE_BY_SIDE_REVIEW_UX_DIRECTION.md`
|
| 597 |
+
|
| 598 |
+
Planning result:
|
| 599 |
+
- confirmed that the current static `components.html` pane cannot return a supported mutation event;
|
| 600 |
+
- recommended a bounded bidirectional Streamlit v1 custom component on the current Streamlit 1.39 stack;
|
| 601 |
+
- routed accepted actions through server-side validation and the existing `build_manual_mask_row` path;
|
| 602 |
+
- kept the review table authoritative and the manual form available as fallback;
|
| 603 |
+
- limited version one to all exact occurrences and deferred occurrence-specific masking to a separate span-aware architecture line;
|
| 604 |
+
- defined quick type choices, occurrence-impact warnings, collision blocking, event replay/stale-view protection, scroll restoration, undo, accessibility, XSS and no-external-network requirements;
|
| 605 |
+
- defined six sequential test-first implementation workpackages.
|
| 606 |
+
|
| 607 |
+
Validation:
|
| 608 |
+
- documentation/architecture contract tests added;
|
| 609 |
+
- PR #59 run #1937: 1015 tests passed in 11.64s;
|
| 610 |
+
- one final clean regression follows the status-only update;
|
| 611 |
+
- Hugging Face sync not functionally relevant;
|
| 612 |
+
- app verification not applicable because no product behavior changed.
|
| 613 |
+
|
| 614 |
+
Intentionally not changed:
|
| 615 |
+
- `presidio_streamlit.py`, side-by-side renderer or manual mask helper;
|
| 616 |
+
- Streamlit/dependency versions or Docker runtime;
|
| 617 |
+
- replacement table, export, Scrub Key or reinsert semantics;
|
| 618 |
+
- recognizers, profiles, cloud processing or product claims.
|
| 619 |
+
|
| 620 |
+
## 2026-08-03 23:26 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_LONG_FORM_SYNTHETIC_CORPUS_APP_VERIFY_CLOSEOUT
|
| 621 |
+
|
| 622 |
+
Status: completed; final documentation-only regression pending.
|
| 623 |
+
|
| 624 |
+
Purpose:
|
| 625 |
+
- Close the long-form synthetic Zorgfilter corpus after technical deployment verification and coordinator/user app confirmation.
|
| 626 |
+
|
| 627 |
+
Files added:
|
| 628 |
+
- `workpackage_claims/scrub_wp_care_profile_long_form_synthetic_corpus_app_verify_closeout.md`
|
| 629 |
+
- `handover/workpackages/20260803_2326_care_profile_long_form_synthetic_corpus_app_verify_closeout.md`
|
| 630 |
+
|
| 631 |
+
Files changed:
|
| 632 |
+
- `workpackage_claims/scrub_wp_care_profile_long_form_synthetic_corpus.md`
|
| 633 |
+
- `handover/workpackages/20260803_2217_care_profile_long_form_synthetic_corpus.md`
|
| 634 |
+
- `WORKPACKAGES.md`
|
| 635 |
+
- `CHANGELOG.md`
|
| 636 |
+
|
| 637 |
+
Validation evidence:
|
| 638 |
+
- final clean PR #56 run #1926: 1003 tests passed in 11.51s;
|
| 639 |
+
- deployment verification run #1931: exact GitHub/Hugging Face matches for both runtime files, health HTTP 200 / `ok`, and 1003 tests passed in 11.35s;
|
| 640 |
+
- coordinator/user confirmation at 2026-08-03 23:26 Europe/Amsterdam: `Alles werkt.`
|
| 641 |
+
|
| 642 |
+
Intentionally not changed:
|
| 643 |
+
- product code, synthetic corpus content, UI, recognizers or profile behavior;
|
| 644 |
+
- replacement table, export, Scrub Key or reinsert semantics;
|
| 645 |
+
- dependencies, cloud processing or product claims.
|
| 646 |
+
|
| 647 |
+
## 2026-08-03 22:17 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_LONG_FORM_SYNTHETIC_CORPUS
|
| 648 |
+
|
| 649 |
+
Status: completed, synchronized and app-verified.
|
| 650 |
+
|
| 651 |
+
Purpose:
|
| 652 |
+
- Give Zorgfilter testers realistic long-form care documents instead of examples that stop after one short clinical paragraph.
|
| 653 |
+
|
| 654 |
+
Files added:
|
| 655 |
+
- `care_test_example_expansions.py`
|
| 656 |
+
- `tests/test_care_profile_long_form_corpus.py`
|
| 657 |
+
- `CARE_PROFILE_LONG_FORM_SYNTHETIC_CORPUS.md`
|
| 658 |
+
- `workpackage_claims/scrub_wp_care_profile_long_form_synthetic_corpus.md`
|
| 659 |
+
- `handover/workpackages/20260803_2217_care_profile_long_form_synthetic_corpus.md`
|
| 660 |
+
|
| 661 |
+
Files changed:
|
| 662 |
+
- `care_test_examples.py`
|
| 663 |
+
- `WORKPACKAGES.md`
|
| 664 |
+
- `RELEASE_NOTES.md`
|
| 665 |
+
- `RISK_REGISTER.md`
|
| 666 |
+
|
| 667 |
+
Implementation result:
|
| 668 |
+
- retained all eight stable care-document IDs, names, sectors and document types;
|
| 669 |
+
- retained every replace, review-selected, preserve, audit-only and ambiguity-trap contract;
|
| 670 |
+
- appended five document-specific sections and at least two hundred words of synthetic care context to every example;
|
| 671 |
+
- kept the additions free of digits and new names, identifiers, dates, addresses, contact details, organizations and locations;
|
| 672 |
+
- exposed the expanded texts through the existing Zorgfilter example selector without changing the selector or Streamlit flow;
|
| 673 |
+
- added tests for length, structure, non-mutation, identity-marker absence, exact expected-value occurrence and UI-adapter parity.
|
| 674 |
+
|
| 675 |
+
Validation:
|
| 676 |
+
- final clean PR run #1926: 1003 tests passed in 11.51s;
|
| 677 |
+
- deployment verification run #1931: both runtime files matched Hugging Face byte-for-byte, Space health returned HTTP 200 / `ok`, and 1003 tests passed in 11.35s;
|
| 678 |
+
- coordinator/user app verification confirmed at 2026-08-03 23:26 Europe/Amsterdam: `Alles werkt.`
|
| 679 |
+
|
| 680 |
+
Intentionally not changed:
|
| 681 |
+
- recognizers, thresholds, profile composition or collision precedence;
|
| 682 |
+
- review selection or replacement-table behavior;
|
| 683 |
+
- export filenames, MIME types or formats;
|
| 684 |
+
- Scrub Key schema, binding, warnings or lifecycle;
|
| 685 |
+
- reinsert behavior, dependencies, cloud processing or production claims.
|
| 686 |
+
|
| 687 |
+
## 2026-08-03 19:12 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_APP_VERIFY
|
| 688 |
+
|
| 689 |
+
Status: completed and app-verified after technical deployment verification.
|
| 690 |
+
|
| 691 |
+
Purpose:
|
| 692 |
+
- Independently verify that the merged Zorgfilter integration reached Hugging Face before requesting human app verification.
|
| 693 |
+
|
| 694 |
+
Files added:
|
| 695 |
+
- `CARE_PROFILE_APP_VERIFICATION.md`
|
| 696 |
+
- `output/validation/care_profile_hf_sync_verification.json`
|
| 697 |
+
- `tests/test_care_profile_hf_sync_verification.py`
|
| 698 |
+
- `workpackage_claims/scrub_wp_care_profile_app_verify.md`
|
| 699 |
+
- `handover/workpackages/20260803_1912_care_profile_app_verify.md`
|
| 700 |
+
|
| 701 |
+
Verification result:
|
| 702 |
+
- compared twelve relevant GitHub and Hugging Face files byte-for-byte;
|
| 703 |
+
- verified equal SHA-256 values for all twelve files;
|
| 704 |
+
- verified correctly scoped Zorgfilter markers;
|
| 705 |
+
- verified Streamlit health `HTTP 200 / ok` and root HTTP 200;
|
| 706 |
+
- corrected an initial verification false negative caused by two marker groups being checked in the wrong modules;
|
| 707 |
+
- recorded coordinator/user confirmation `alles groen` at 2026-08-03 20:35 Europe/Amsterdam;
|
| 708 |
+
- confirmed all nine visible verification checks and retained the non-production/human-review boundary.
|
| 709 |
+
|
| 710 |
+
Validation context:
|
| 711 |
+
- UI integration final run #1885 passed: 986 tests;
|
| 712 |
+
- cross-profile matrix final run #1906 passed: 995 tests;
|
| 713 |
+
- verification-only run #1908 passed 998 tests; final closeout run #1909 passed 998 tests in 10.45s.
|
| 714 |
+
|
| 715 |
+
Intentionally not changed:
|
| 716 |
+
- product code, recognizers, profile behavior or UI;
|
| 717 |
+
- review, export, Scrub Key or reinsert semantics;
|
| 718 |
+
- runtime dependencies, cloud processing or production claims.
|
| 719 |
+
|
| 720 |
+
## 2026-08-03 18:58 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_CROSS_PROFILE_REGRESSION_MATRIX
|
| 721 |
+
|
| 722 |
+
Status: completed and regression-green.
|
| 723 |
+
|
| 724 |
+
Purpose:
|
| 725 |
+
- Add deterministic evidence that the new Care profile remains isolated from Legal and General profiles and preserves clinical meaning.
|
| 726 |
+
|
| 727 |
+
Files added:
|
| 728 |
+
- `care_profile_cross_profile_matrix.py`
|
| 729 |
+
- `CARE_PROFILE_CROSS_PROFILE_REGRESSION_MATRIX.md`
|
| 730 |
+
- `tests/test_care_profile_cross_profile_matrix.py`
|
| 731 |
+
- `output/validation/care_profile_cross_profile_matrix.json`
|
| 732 |
+
- `workpackage_claims/scrub_wp_care_profile_cross_profile_regression_matrix.md`
|
| 733 |
+
- `handover/workpackages/20260803_1858_care_profile_cross_profile_regression_matrix.md`
|
| 734 |
+
|
| 735 |
+
Implementation result:
|
| 736 |
+
- executed real deterministic Dutch custom recognizers across four configured profiles;
|
| 737 |
+
- covered eight care-document families and twelve legal examples;
|
| 738 |
+
- verified 108/108 dedicated Care expectations across Care and International;
|
| 739 |
+
- verified Care/International and Legal/International dedicated-type parity;
|
| 740 |
+
- verified no dedicated Care/Legal leakage into the wrong profiles;
|
| 741 |
+
- verified zero overlap with protected clinical phrases;
|
| 742 |
+
- separated hard profile gates from historical legal metadata observations;
|
| 743 |
+
- preserved sixteen historical legal metadata gaps and four negative observations in the evidence snapshot;
|
| 744 |
+
- added snapshot reproducibility checks for all twenty observations.
|
| 745 |
+
|
| 746 |
+
Validation:
|
| 747 |
+
- run #1887 exposed an invalid hard-contract assumption for historical legal metadata;
|
| 748 |
+
- run #1888 made all twenty observations explicit;
|
| 749 |
+
- run #1890 passed after correcting the methodology: 994 tests;
|
| 750 |
+
- run #1897 exposed only an incorrect legal-example count in the snapshot;
|
| 751 |
+
- run #1899 passed: 995 tests in 9.56s;
|
| 752 |
+
- final clean run #1906 passed: 995 tests in 9.96s.
|
| 753 |
+
|
| 754 |
+
Intentionally not changed:
|
| 755 |
+
- current Streamlit UI or profile selector;
|
| 756 |
+
- review table, export, Scrub Key or reinsert semantics;
|
| 757 |
+
- runtime dependencies, cloud processing or production claims;
|
| 758 |
+
- generic NER behavior.
|
| 759 |
+
|
| 760 |
+
## 2026-08-03 18:28 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_CURRENT_UI_INTEGRATION
|
| 761 |
+
|
| 762 |
+
Status: completed, deployed and app-verified.
|
| 763 |
+
|
| 764 |
+
Purpose:
|
| 765 |
+
- Promote the approved Zorgfilter profile into the current Streamlit/analyzer flow after policy, corpus, recognizer and profile-configuration gates passed.
|
| 766 |
+
|
| 767 |
+
Files added:
|
| 768 |
+
- `care_candidate_scanner.py`
|
| 769 |
+
- `profile_ui_support.py`
|
| 770 |
+
- `CARE_PROFILE_CURRENT_UI_INTEGRATION.md`
|
| 771 |
+
- `output/validation/care_profile_current_ui_integration.json`
|
| 772 |
+
- `tests/test_care_candidate_scanner.py`
|
| 773 |
+
- `tests/test_profile_ui_support.py`
|
| 774 |
+
- `tests/test_presidio_helpers_care_registration.py`
|
| 775 |
+
- `tests/test_care_profile_current_ui_integration_snapshot.py`
|
| 776 |
+
- `workpackage_claims/scrub_wp_care_profile_current_ui_integration.md`
|
| 777 |
+
- `handover/workpackages/20260803_1828_care_profile_current_ui_integration.md`
|
| 778 |
+
|
| 779 |
+
Files changed:
|
| 780 |
+
- `presidio_helpers.py`
|
| 781 |
+
- `presidio_streamlit.py`
|
| 782 |
+
- `document_tools.py`
|
| 783 |
+
- `display_labels_nl.py`
|
| 784 |
+
- `ui_texts_nl.py`
|
| 785 |
+
- `ROADMAP.md`
|
| 786 |
+
- `WORKPACKAGES.md`
|
| 787 |
+
- `CHANGELOG.md`
|
| 788 |
+
- `RELEASE_NOTES.md`
|
| 789 |
+
- `RISK_REGISTER.md`
|
| 790 |
+
|
| 791 |
+
Implementation result:
|
| 792 |
+
- registered sixteen dedicated care recognizers;
|
| 793 |
+
- added `Zorgcontrole — streng` while preserving the existing three labels and Legal default;
|
| 794 |
+
- centralized thresholds, profile descriptions and default entity composition;
|
| 795 |
+
- applied profile-level exact-span collision resolution;
|
| 796 |
+
- added eight synthetic care examples and a conservative unchecked candidate layer;
|
| 797 |
+
- aligned review-selected care detections to `Controle nodig` while keeping them selected;
|
| 798 |
+
- added care display labels and stable placeholders;
|
| 799 |
+
- generalized app copy from Legal-only to professional Legal/Zorg use;
|
| 800 |
+
- preserved review table, export, Scrub Key and reinsert semantics.
|
| 801 |
+
|
| 802 |
+
Validation:
|
| 803 |
+
- run #1876 failed only because the new registration test imported optional Streamlit dependencies absent from lean CI;
|
| 804 |
+
- test isolation was corrected without changing runtime code or dependencies;
|
| 805 |
+
- run #1877 passed: 983 tests in 9.69s;
|
| 806 |
+
- final clean integration run #1885 passed: 986 tests;
|
| 807 |
+
- Hugging Face synchronization verified byte-for-byte for 12/12 relevant files;
|
| 808 |
+
- deployed app verification confirmed `alles groen` by the coordinator/user.
|
| 809 |
+
|
| 810 |
+
Intentionally not changed:
|
| 811 |
+
- export filenames, MIME types and document formats;
|
| 812 |
+
- Scrub Key schema, binding, warnings or lifecycle;
|
| 813 |
+
- TXT/DOCX/PDF-to-TXT reinsert semantics;
|
| 814 |
+
- cloud processing, runtime dependencies or production claims;
|
| 815 |
+
- broad free-text medical scanning.
|
| 816 |
+
|
| 817 |
+
## 2026-08-03 17:12 Europe/Amsterdam — SCRUB-WP_RECOGNITION_PROFILE_CONFIGURATION_REFACTOR
|
| 818 |
+
|
| 819 |
+
Status: completed; pure recognition-profile configuration implemented.
|
| 820 |
+
|
| 821 |
+
Purpose:
|
| 822 |
+
- Replace scattered future profile branching with one tested source of truth before exposing Zorg in the current UI.
|
| 823 |
+
|
| 824 |
+
Files added:
|
| 825 |
+
- `recognition_profiles.py`
|
| 826 |
+
- `recognition_profile_validation.py`
|
| 827 |
+
- `RECOGNITION_PROFILE_CONFIGURATION.md`
|
| 828 |
+
- `tests/test_recognition_profiles.py`
|
| 829 |
+
- `tests/test_recognition_profile_validation.py`
|
| 830 |
+
- `output/validation/recognition_profile_configuration.json`
|
| 831 |
+
- `workpackage_claims/scrub_wp_recognition_profile_configuration_refactor.md`
|
| 832 |
+
- `handover/workpackages/20260803_1712_recognition_profile_configuration_refactor.md`
|
| 833 |
+
|
| 834 |
+
Implementation result:
|
| 835 |
+
- preserved the exact current three-profile order and labels;
|
| 836 |
+
- defined the future Streamlit and desktop four-profile orders;
|
| 837 |
+
- centralized thresholds, entity groups, candidate/example direction and Care policy actions;
|
| 838 |
+
- added deterministic exact-span precedence for fifteen care-specific winners;
|
| 839 |
+
- kept partial overlaps and all non-Care profile results unchanged;
|
| 840 |
+
- kept live UI and care recognizer registration explicitly false.
|
| 841 |
+
|
| 842 |
+
Validation:
|
| 843 |
+
- initial GitHub Actions run #1865 passed: 965 tests;
|
| 844 |
+
- final clean validation pending after governance finalization;
|
| 845 |
+
- Hugging Face sync not functionally relevant;
|
| 846 |
+
- app verification not applicable.
|
| 847 |
+
|
| 848 |
+
Intentionally not changed:
|
| 849 |
+
- `presidio_streamlit.py`, `presidio_helpers.py` or current analyzer registration;
|
| 850 |
+
- visible profile selector, thresholds or entity defaults;
|
| 851 |
+
- review, export, Scrub Key or reinsert semantics;
|
| 852 |
+
- runtime, dependencies or cloud processing.
|
| 853 |
+
|
| 854 |
+
## 2026-08-03 16:52 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_RECOGNIZER_IMPLEMENTATION
|
| 855 |
+
|
| 856 |
+
Status: completed; pure recognizer implementation validated.
|
| 857 |
+
|
| 858 |
+
Purpose:
|
| 859 |
+
- Implement the frozen Zorgfilter recognizer contract without registering it in the current app.
|
| 860 |
+
|
| 861 |
+
Files added:
|
| 862 |
+
- `dutch_care_recognizers.py`
|
| 863 |
+
- `care_recognizer_validation.py`
|
| 864 |
+
- `CARE_RECOGNIZER_IMPLEMENTATION_V1.md`
|
| 865 |
+
- `scripts/generate_care_recognizer_validation.py`
|
| 866 |
+
- `tests/test_dutch_care_recognizers.py`
|
| 867 |
+
- `tests/test_care_recognizer_validation.py`
|
| 868 |
+
- `output/validation/care_recognizer_implementation_validation.json`
|
| 869 |
+
- `workpackage_claims/scrub_wp_care_profile_recognizer_implementation.md`
|
| 870 |
+
- `handover/workpackages/20260803_1652_care_profile_recognizer_implementation.md`
|
| 871 |
+
|
| 872 |
+
Implementation result:
|
| 873 |
+
- implemented sixteen dedicated, context-bound care entities;
|
| 874 |
+
- returned exact value-only Presidio spans with explanations and metadata;
|
| 875 |
+
- corrected bounded variants for Dutch client labels, prescription numbers, generic incident labels and lowercase residential-care organization labels;
|
| 876 |
+
- passed all frozen positive and negative contracts;
|
| 877 |
+
- covered all 54 dedicated corpus expectations with zero protected-clinical overlaps;
|
| 878 |
+
- kept app registration explicitly false.
|
| 879 |
+
|
| 880 |
+
Validation:
|
| 881 |
+
- initial run #1850 exposed five bounded missing context variants and no clinical-overmasking failure;
|
| 882 |
+
- corrected GitHub Actions run #1854 passed: 953 tests;
|
| 883 |
+
- final clean validation pending after governance finalization;
|
| 884 |
+
- Hugging Face sync not functionally relevant;
|
| 885 |
+
- app verification not applicable.
|
| 886 |
+
|
| 887 |
+
Intentionally not changed:
|
| 888 |
+
- `presidio_helpers.py`, `presidio_streamlit.py` or current profile behavior;
|
| 889 |
+
- thresholds, entity defaults or generic NER;
|
| 890 |
+
- review, export, Scrub Key or reinsert semantics;
|
| 891 |
+
- runtime, dependencies or cloud processing.
|
| 892 |
+
|
| 893 |
+
## 2026-08-03 16:34 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_RECOGNIZER_CONTRACT_TESTS
|
| 894 |
+
|
| 895 |
+
Status: completed; test/specification contract frozen.
|
| 896 |
+
|
| 897 |
+
Purpose:
|
| 898 |
+
- Define dedicated Zorgfilter recognizer behavior before writing implementation code.
|
| 899 |
+
|
| 900 |
+
Files added:
|
| 901 |
+
- `CARE_RECOGNIZER_CONTRACT_V1.md`
|
| 902 |
+
- `care_recognizer_contracts.py`
|
| 903 |
+
- `care_recognizer_contract_summary.py`
|
| 904 |
+
- `tests/test_care_recognizer_contracts.py`
|
| 905 |
+
- `scripts/generate_care_recognizer_contract_summary.py`
|
| 906 |
+
- `output/validation/care_recognizer_contract_v1_summary.json`
|
| 907 |
+
- `workpackage_claims/scrub_wp_care_profile_recognizer_contract_tests.md`
|
| 908 |
+
- `handover/workpackages/20260803_1634_care_profile_recognizer_contract_tests.md`
|
| 909 |
+
|
| 910 |
+
Implementation result:
|
| 911 |
+
- froze sixteen dedicated care entities and the future pure helper API;
|
| 912 |
+
- added 37 positive exact-span fixtures;
|
| 913 |
+
- added 16 negative/collision/clinical-preservation fixtures;
|
| 914 |
+
- froze replace/review-selected policy alignment;
|
| 915 |
+
- froze care-event/date-of-birth and AGB/BSN precedence;
|
| 916 |
+
- kept generic PERSON/e-mail outside the dedicated care module.
|
| 917 |
+
|
| 918 |
+
Validation:
|
| 919 |
+
- GitHub Actions pending final PR validation;
|
| 920 |
+
- Hugging Face sync not functionally relevant;
|
| 921 |
+
- app verification not applicable.
|
| 922 |
+
|
| 923 |
+
Intentionally not changed:
|
| 924 |
+
- recognizer implementation or registration;
|
| 925 |
+
- current profile selector, thresholds or entity defaults;
|
| 926 |
+
- review, export, Scrub Key and reinsert semantics;
|
| 927 |
+
- runtime, dependencies or cloud processing.
|
| 928 |
+
|
| 929 |
+
## 2026-08-03 16:25 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_GAP_TRIAGE
|
| 930 |
+
|
| 931 |
+
Status: completed; evidence-driven gap routing implemented.
|
| 932 |
+
|
| 933 |
+
Purpose:
|
| 934 |
+
- Convert the current-engine care baseline into explicit recognizer, profile and collision-prevention contract routes.
|
| 935 |
+
|
| 936 |
+
Files added:
|
| 937 |
+
- `CARE_PROFILE_GAP_TRIAGE.md`
|
| 938 |
+
- `care_profile_gap_triage.py`
|
| 939 |
+
- `care_profile_gap_triage_summary.py`
|
| 940 |
+
- `tests/test_care_profile_gap_triage.py`
|
| 941 |
+
- `scripts/generate_care_profile_gap_triage.py`
|
| 942 |
+
- `output/validation/care_profile_v1_gap_triage.json`
|
| 943 |
+
- `workpackage_claims/scrub_wp_care_profile_gap_triage.md`
|
| 944 |
+
- `handover/workpackages/20260803_1625_care_profile_gap_triage.md`
|
| 945 |
+
|
| 946 |
+
Implementation result:
|
| 947 |
+
- classified all 81 expectations with zero unclassified values;
|
| 948 |
+
- froze six implementation routes and five contract families;
|
| 949 |
+
- separated generic NER dependencies from care-specific rule work;
|
| 950 |
+
- routed ten broad-entity matches to care-specific reclassification;
|
| 951 |
+
- routed 36 context-dependent values to review-selected recognition;
|
| 952 |
+
- made AGB/BSN and medical-number collision guards mandatory;
|
| 953 |
+
- preserved clinical-content negative contracts as a first-class requirement.
|
| 954 |
+
|
| 955 |
+
Validation:
|
| 956 |
+
- GitHub Actions pending final PR validation;
|
| 957 |
+
- Hugging Face sync not functionally relevant;
|
| 958 |
+
- app verification not applicable.
|
| 959 |
+
|
| 960 |
+
Intentionally not changed:
|
| 961 |
+
- recognizer implementation or registration;
|
| 962 |
+
- profile selector, thresholds or entity defaults;
|
| 963 |
+
- review, export, Scrub Key or reinsert semantics;
|
| 964 |
+
- runtime, dependencies or cloud processing.
|
| 965 |
+
|
| 966 |
+
## 2026-08-03 16:10 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_CURRENT_ENGINE_BASELINE
|
| 967 |
+
|
| 968 |
+
Status: completed; evidence baseline implemented and validated.
|
| 969 |
+
|
| 970 |
+
Purpose:
|
| 971 |
+
- Establish a corrected, reproducible pre-Zorgfilter measurement for the current deterministic Dutch custom recognizers.
|
| 972 |
+
|
| 973 |
+
Files added:
|
| 974 |
+
- `CARE_PROFILE_CURRENT_ENGINE_BASELINE.md`
|
| 975 |
+
- `care_profile_baseline_summary.py`
|
| 976 |
+
- `tests/test_care_profile_baseline_summary.py`
|
| 977 |
+
- `output/validation/care_profile_v1_current_engine_baseline.json`
|
| 978 |
+
- `workpackage_claims/scrub_wp_care_profile_current_engine_baseline.md`
|
| 979 |
+
- `handover/workpackages/20260803_1610_care_profile_current_engine_baseline.md`
|
| 980 |
+
|
| 981 |
+
Files changed:
|
| 982 |
+
- `care_profile_baseline.py`
|
| 983 |
+
- `scripts/generate_care_profile_baseline.py`
|
| 984 |
+
- `tests/test_care_profile_current_engine_baseline.py`
|
| 985 |
+
- `ROADMAP.md`
|
| 986 |
+
- `WORKPACKAGES.md`
|
| 987 |
+
- `RISK_REGISTER.md`
|
| 988 |
+
|
| 989 |
+
Implementation result:
|
| 990 |
+
- changed baseline matching from substring containment to exact normalized span equality;
|
| 991 |
+
- prevented false AGB coverage from an eight-digit prefix inside a longer BIG number;
|
| 992 |
+
- added compact policy/entity/case summaries;
|
| 993 |
+
- committed a reproducible JSON evidence artifact;
|
| 994 |
+
- documented 25/81 span recall, 14/81 correct-entity recall, 11 misclassifications, 56 misses and zero protected-clinical overlaps.
|
| 995 |
+
|
| 996 |
+
Validation:
|
| 997 |
+
- diagnostic run #1825 produced the evidence while 918 non-diagnostic tests passed;
|
| 998 |
+
- final clean GitHub Actions validation required after temporary diagnostics were removed;
|
| 999 |
+
- Hugging Face sync not functionally relevant;
|
| 1000 |
+
- app verification not applicable.
|
| 1001 |
+
|
| 1002 |
+
Intentionally not changed:
|
| 1003 |
+
- recognizer registration or behavior;
|
| 1004 |
+
- current profile selector or UI;
|
| 1005 |
+
- thresholds or entity defaults;
|
| 1006 |
+
- review, export, Scrub Key and reinsert semantics;
|
| 1007 |
+
- dependencies, runtime or cloud processing.
|
| 1008 |
+
|
| 1009 |
+
## 2026-08-03 15:31 Europe/Amsterdam — SCRUB-WP_CARE_PROFILE_V1_POLICY_AND_CORPUS_FOUNDATION
|
| 1010 |
+
|
| 1011 |
+
Status: completed; policy/corpus foundation implemented and validated.
|
| 1012 |
+
|
| 1013 |
+
Purpose:
|
| 1014 |
+
- Create the evidence base for an explicit Dutch Zorg profile without prematurely changing recognizers or the UI.
|
| 1015 |
+
|
| 1016 |
+
Files added:
|
| 1017 |
+
- `CARE_PROFILE_V1_PLAN.md`
|
| 1018 |
+
- `care_profile_policy.py`
|
| 1019 |
+
- `care_test_examples.py`
|
| 1020 |
+
- `care_profile_baseline.py`
|
| 1021 |
+
- `scripts/generate_care_profile_baseline.py`
|
| 1022 |
+
- `tests/test_care_profile_policy_contract.py`
|
| 1023 |
+
- `tests/test_care_profile_corpus_contracts.py`
|
| 1024 |
+
- `tests/test_care_profile_current_engine_baseline.py`
|
| 1025 |
+
- `workpackage_claims/scrub_wp_care_profile_v1_policy_and_corpus_foundation.md`
|
| 1026 |
+
|
| 1027 |
+
Main changes:
|
| 1028 |
+
- froze replace/review/preserve/audit-only care policy actions;
|
| 1029 |
+
- added eight fully synthetic document families;
|
| 1030 |
+
- separated patient identifiers from provider/location review and clinical content preservation;
|
| 1031 |
+
- added a deterministic baseline helper for current Dutch custom recognizers;
|
| 1032 |
+
- made no recognizer or product-UI behavior change.
|
| 1033 |
+
|
| 1034 |
+
Validation:
|
| 1035 |
+
- full GitHub Actions run #1818 passed: 918 tests;
|
| 1036 |
+
- Hugging Face sync not functionally relevant because no runtime/UI file changed;
|
| 1037 |
+
- app verification not applicable.
|
| 1038 |
+
|
| 1039 |
+
Intentionally not changed:
|
| 1040 |
+
- current three profile choices;
|
| 1041 |
+
- recognizer registration or thresholds;
|
| 1042 |
+
- review table, export, Scrub Key and reinsert semantics;
|
| 1043 |
+
- runtime, dependencies or cloud processing.
|
| 1044 |
+
|
| 1045 |
+
## 2026-08-03 14:47 Europe/Amsterdam — SCRUB-WP_AI_FIRST_DESKTOP_PACKAGING_ROADMAP_ALIGNMENT
|
| 1046 |
+
|
| 1047 |
+
Status: completed; roadmap/decision documentation only.
|
| 1048 |
+
|
| 1049 |
+
Purpose:
|
| 1050 |
+
- Incorporate an AI-first cost and authority model into the final local EXE/MSI roadmap while preserving security and release accountability.
|
| 1051 |
+
|
| 1052 |
+
Files added:
|
| 1053 |
+
- `AI_FIRST_DESKTOP_PACKAGING_EXECUTION_MODEL.md`
|
| 1054 |
+
- `handover/workpackages/20260803_1447_ai_first_desktop_packaging_roadmap_alignment.md`
|
| 1055 |
+
- `workpackage_claims/scrub_wp_ai_first_desktop_packaging_roadmap_alignment.md`
|
| 1056 |
+
|
| 1057 |
+
Files changed:
|
| 1058 |
+
- `ROADMAP.md`
|
| 1059 |
+
- `DESKTOP_PACKAGING_DECISION.md`
|
| 1060 |
+
- `DECISION_LOG.md`
|
| 1061 |
+
- `WORKPACKAGES.md`
|
| 1062 |
+
- `CHANGELOG.md`
|
| 1063 |
+
|
| 1064 |
+
Main changes:
|
| 1065 |
+
- Phase 9 target refined to signed setup.exe/MSI distribution around a Tauri shell and bundled PyInstaller onedir local engine.
|
| 1066 |
+
- AI-agent planning assumption recorded: 60–70% first-cycle labor substitution and 75–90% later repetitive release automation.
|
| 1067 |
+
- Indicative post-agent development/integration budget recorded as EUR 8,000–24,000, excluding retained independent security review and authority costs.
|
| 1068 |
+
- Human gates retained for signing, release, security claims, UX acceptance and safety-critical semantics.
|
| 1069 |
+
|
| 1070 |
+
Validation:
|
| 1071 |
+
- Documentation markers and phase gates checked by the branch governance operator.
|
| 1072 |
+
- No product tests required because no product code or runtime changed.
|
| 1073 |
+
- GitHub Actions: pending PR validation.
|
| 1074 |
+
- Hugging Face sync: not functionally relevant.
|
| 1075 |
+
- App verification: not applicable.
|
| 1076 |
+
|
| 1077 |
+
Intentionally not changed:
|
| 1078 |
+
- active Phase 6 execution order;
|
| 1079 |
+
- installer implementation authorization;
|
| 1080 |
+
- runtime, UI, dependencies, recognizers, export, Scrub Key or reinsert behavior;
|
| 1081 |
+
- cloud processing or telemetry behavior.
|
| 1082 |
+
|
| 1083 |
+
## 2026-07-28 00:52 Europe/Amsterdam — SCRUB-WP_MVP_SCRUB_KEY_BINDING_REINSERT_INTEGRATION
|
| 1084 |
+
|
| 1085 |
+
Status: implemented; full GitHub Actions passed; final PR validation pending.
|
| 1086 |
+
|
| 1087 |
+
Purpose:
|
| 1088 |
+
- Prevent a wrong, mixed or accidentally corrupted Scrub Key from restoring values before document/key binding is verified.
|
| 1089 |
+
|
| 1090 |
+
Files added:
|
| 1091 |
+
- `scrub_key_binding_reinsert_status.py`
|
| 1092 |
+
- `tests/test_scrub_key_binding_reinsert_integration.py`
|
| 1093 |
+
- `tests/test_scrub_key_binding_reinsert_status.py`
|
| 1094 |
+
- `tests/test_scrub_key_binding_reinsert_ui.py`
|
| 1095 |
+
- `output/validation/mvp_scrub_key_binding_reinsert_validation.json`
|
| 1096 |
+
- `handover/workpackages/20260728_0052_mvp_scrub_key_binding_reinsert_integration.md`
|
| 1097 |
+
- `workpackage_claims/scrub_wp_mvp_scrub_key_binding_reinsert_integration.md`
|
| 1098 |
+
|
| 1099 |
+
Files changed:
|
| 1100 |
+
- `scrub_key_import.py`
|
| 1101 |
+
- `scrub_key_reinsert.py`
|
| 1102 |
+
- `scrub_key_document_reinsert.py`
|
| 1103 |
+
- `reinsert_mode_ui.py`
|
| 1104 |
+
- `tests/test_scrub_key_binding_model.py`
|
| 1105 |
+
- `WORKPACKAGES.md`
|
| 1106 |
+
- `RELEASE_NOTES.md`
|
| 1107 |
+
- `ROADMAP.md`
|
| 1108 |
+
- `MVP_PHASE6_EXECUTION_PLAN.md`
|
| 1109 |
+
- `RISK_REGISTER.md`
|
| 1110 |
+
- `DECISION_LOG.md`
|
| 1111 |
+
|
| 1112 |
+
Implementation result:
|
| 1113 |
+
- Dual-read import supports structurally valid legacy v1.0 and bound v1.1 keys.
|
| 1114 |
+
- Binding validation runs before any local deterministic replacement.
|
| 1115 |
+
- `bound_match` is verified; `legacy_unbound` remains compatible but explicitly unverified.
|
| 1116 |
+
- Six frozen mismatch/corruption states fail closed with zero replacements.
|
| 1117 |
+
- DOCX mismatch returns exact original bytes; no partial package is produced.
|
| 1118 |
+
- Binding status, IDs, digest state and warnings are shown in existing feedback/report surfaces.
|
| 1119 |
+
- No new source/key execution button or acknowledgement checkbox was added.
|
| 1120 |
+
|
| 1121 |
+
Validation:
|
| 1122 |
+
- Normal full GitHub Actions run #1789 passed.
|
| 1123 |
+
- Synthetic adversarial coverage spans text, TXT, DOCX body/header/footer and PDF-to-TXT.
|
| 1124 |
+
- Human review remains required; production readiness remains false.
|
| 1125 |
+
|
| 1126 |
+
Intentionally not changed:
|
| 1127 |
+
- Output filenames or MIME types.
|
| 1128 |
+
- Supported TXT, DOCX and PDF-to-TXT boundaries.
|
| 1129 |
+
- Legacy key migration.
|
| 1130 |
+
- Signing/HMAC, secret storage, cloud, AI, OCR or restored-PDF behavior.
|
| 1131 |
+
|
| 1132 |
+
Next recommended step:
|
| 1133 |
+
- Complete final PR validation and merge, verify sync, then run `SCRUB-WP_MVP_SCRUB_KEY_BINDING_APP_VERIFY`.
|
| 1134 |
+
|
| 1135 |
+
---
|
| 1136 |
+
|
| 1137 |
+
## 2026-07-27 20:05 Europe/Amsterdam — SCRUB-WP_MVP_SCRUB_KEY_BINDING_MODEL_IMPLEMENTATION
|
| 1138 |
+
|
| 1139 |
+
Status: implemented; targeted validation passed; PR verification pending.
|
| 1140 |
+
|
| 1141 |
+
Purpose:
|
| 1142 |
+
- Implement the frozen document/Scrub-Key binding contract as pure helpers before changing shared placeholder generation, export or reinsert behavior.
|
| 1143 |
+
|
| 1144 |
+
Files added:
|
| 1145 |
+
- `scrub_key_binding.py`
|
| 1146 |
+
- `tests/test_scrub_key_binding_model.py`
|
| 1147 |
+
- `tests/test_scrub_key_binding_model_validation.py`
|
| 1148 |
+
- `output/validation/mvp_scrub_key_binding_model_validation.json`
|
| 1149 |
+
- `handover/workpackages/20260727_2005_mvp_scrub_key_binding_model_implementation.md`
|
| 1150 |
+
- `workpackage_claims/scrub_wp_mvp_scrub_key_binding_model_implementation.md`
|
| 1151 |
+
|
| 1152 |
+
Files changed:
|
| 1153 |
+
- `WORKPACKAGES.md`
|
| 1154 |
+
- `CHANGELOG.md`
|
| 1155 |
+
- `ROADMAP.md`
|
| 1156 |
+
- `MVP_PHASE6_EXECUTION_PLAN.md`
|
| 1157 |
+
- `RISK_REGISTER.md`
|
| 1158 |
+
- `DECISION_LOG.md`
|
| 1159 |
+
|
| 1160 |
+
Implementation result:
|
| 1161 |
+
- Local document binding ID generation supports injected deterministic bytes and normal `secrets` randomness.
|
| 1162 |
+
- Bound automatic/manual placeholder creation and parsing match the frozen grammar.
|
| 1163 |
+
- Canonical digest output matches the fixed synthetic SHA-256 fixture.
|
| 1164 |
+
- Bound key validation checks schema, policies, items, item bindings, duplicates and digest.
|
| 1165 |
+
- Document/key validation implements `bound_match`, `legacy_unbound`, `binding_mismatch`, `mixed_document_bindings`, `missing_document_binding`, `invalid_mapping_digest`, `invalid_bound_key` and `legacy_key_for_bound_document`.
|
| 1166 |
+
- Bound failures prohibit replacement; legacy compatibility remains explicitly unverified.
|
| 1167 |
+
- Inputs remain unmodified.
|
| 1168 |
+
|
| 1169 |
+
Intentionally not changed:
|
| 1170 |
+
- current generic automatic/manual placeholder generation;
|
| 1171 |
+
- current Scrub Key builder, serializer, export or import integration;
|
| 1172 |
+
- current reinsert execution helpers or UI;
|
| 1173 |
+
- filenames or MIME types;
|
| 1174 |
+
- legacy migration;
|
| 1175 |
+
- signing/HMAC or secret storage;
|
| 1176 |
+
- cloud, AI, OCR or restored-PDF behavior.
|
| 1177 |
+
|
| 1178 |
+
Next recommended step:
|
| 1179 |
+
- Start `SCRUB-WP_MVP_SCRUB_KEY_BINDING_EXPORT_INTEGRATION` after PR validation and merge.
|
| 1180 |
+
|
| 1181 |
+
---
|
| 1182 |
+
|
| 1183 |
+
## 2026-07-27 19:38 Europe/Amsterdam — SCRUB-WP_MVP_SCRUB_KEY_BINDING_CONTRACT_TESTS
|
| 1184 |
+
|
| 1185 |
+
Status: completed; contract frozen; PR validation pending.
|
| 1186 |
+
|
| 1187 |
+
Purpose:
|
| 1188 |
+
- Freeze the bound-placeholder, Scrub Key metadata, mapping-digest, legacy compatibility and fail-closed validation contracts before implementing the model.
|
| 1189 |
+
|
| 1190 |
+
Files added:
|
| 1191 |
+
- `SCRUB_KEY_BINDING_CONTRACT.md`
|
| 1192 |
+
- `test_cases/mvp_phase6/scrub_key_binding_contract.json`
|
| 1193 |
+
- `tests/test_mvp_scrub_key_binding_contracts.py`
|
| 1194 |
+
- `tests/test_mvp_scrub_key_binding_contract_validation.py`
|
| 1195 |
+
- `output/validation/mvp_scrub_key_binding_contract_validation.json`
|
| 1196 |
+
- `handover/workpackages/20260727_1938_mvp_scrub_key_binding_contract_tests.md`
|
| 1197 |
+
- `workpackage_claims/scrub_wp_mvp_scrub_key_binding_contract_tests.md`
|
| 1198 |
+
|
| 1199 |
+
Files changed:
|
| 1200 |
+
- `WORKPACKAGES.md`
|
| 1201 |
+
- `CHANGELOG.md`
|
| 1202 |
+
- `ROADMAP.md`
|
| 1203 |
+
- `MVP_PHASE6_EXECUTION_PLAN.md`
|
| 1204 |
+
- `RISK_REGISTER.md`
|
| 1205 |
+
- `DECISION_LOG.md`
|
| 1206 |
+
|
| 1207 |
+
Contract result:
|
| 1208 |
+
- Binding ID grammar: `B[A-Z2-7]{16}`.
|
| 1209 |
+
- Automatic placeholder grammar: `[LABEL_BINDINGID_INDEX]`.
|
| 1210 |
+
- Manual placeholder grammar: `[LABEL_BINDINGID_HANDMATIG_INDEX]`.
|
| 1211 |
+
- Proposed bound key schema: `1.1`; binding version: `1`.
|
| 1212 |
+
- Canonical digest algorithm: SHA-256.
|
| 1213 |
+
- Eight binding statuses; six are fail-closed with zero replacements.
|
| 1214 |
+
- Legacy v1.0 remains explicit unbound compatibility and cannot be used for bound documents.
|
| 1215 |
+
- Current three-step reinsert UX remains unchanged; no new source/key buttons or checkboxes.
|
| 1216 |
+
- Product code changed: false.
|
| 1217 |
+
- Production ready: false; human review required: true.
|
| 1218 |
+
|
| 1219 |
+
Intentionally not changed:
|
| 1220 |
+
- existing product helpers;
|
| 1221 |
+
- automatic/manual placeholder generation;
|
| 1222 |
+
- Scrub Key build, validation, serialization or import implementation;
|
| 1223 |
+
- export or reinsert semantics;
|
| 1224 |
+
- UI or download behavior;
|
| 1225 |
+
- signing/HMAC or secret storage;
|
| 1226 |
+
- cloud, AI, OCR or restored-PDF behavior.
|
| 1227 |
+
|
| 1228 |
+
Next recommended step:
|
| 1229 |
+
- Start `SCRUB-WP_MVP_SCRUB_KEY_BINDING_MODEL_IMPLEMENTATION` after PR validation and merge.
|
| 1230 |
+
|
| 1231 |
+
---
|
| 1232 |
+
|
| 1233 |
+
## 2026-07-27 19:18 Europe/Amsterdam — SCRUB-WP_MVP_SCRUB_KEY_DOCUMENT_BINDING_GAP_TRIAGE
|
| 1234 |
+
|
| 1235 |
+
Status: completed; targeted validation passed; PR verification pending.
|
| 1236 |
+
|
| 1237 |
+
Purpose:
|
| 1238 |
+
- Determine the smallest safe cross-format mitigation for the critical document/Scrub-Key binding gap before changing schema, placeholders, export or reinsert behavior.
|
| 1239 |
+
|
| 1240 |
+
Files added:
|
| 1241 |
+
- `MVP_SCRUB_KEY_DOCUMENT_BINDING_GAP_TRIAGE.md`
|
| 1242 |
+
- `output/validation/mvp_scrub_key_document_binding_gap_triage.json`
|
| 1243 |
+
- `output/validation/mvp_scrub_key_document_binding_gap_triage_validation.json`
|
| 1244 |
+
- `tests/test_mvp_scrub_key_document_binding_gap_triage.py`
|
| 1245 |
+
- `tests/test_mvp_scrub_key_document_binding_gap_triage_validation.py`
|
| 1246 |
+
- `handover/workpackages/20260727_1918_mvp_scrub_key_document_binding_gap_triage.md`
|
| 1247 |
+
- `workpackage_claims/scrub_wp_mvp_scrub_key_document_binding_gap_triage.md`
|
| 1248 |
+
|
| 1249 |
+
Files changed:
|
| 1250 |
+
- `WORKPACKAGES.md`
|
| 1251 |
+
- `CHANGELOG.md`
|
| 1252 |
+
- `ROADMAP.md`
|
| 1253 |
+
- `MVP_PHASE6_EXECUTION_PLAN.md`
|
| 1254 |
+
- `RISK_REGISTER.md`
|
| 1255 |
+
- `DECISION_LOG.md`
|
| 1256 |
+
|
| 1257 |
+
Triage result:
|
| 1258 |
+
- Primary MVP threat: accidental wrong-document/key pairing.
|
| 1259 |
+
- Secondary MVP threat: accidental key corruption.
|
| 1260 |
+
- Deferred threat: malicious tampering requiring protected signing-key infrastructure.
|
| 1261 |
+
- Recommended primary control: document-specific non-sensitive binding ID in all placeholders and the key.
|
| 1262 |
+
- Recommended complementary control: canonical SHA-256 mapping digest.
|
| 1263 |
+
- Explicitly not sufficient: document labels, filenames, content hashes, placeholder-list hashes or metadata-only binding.
|
| 1264 |
+
- Legacy v1.0 keys require explicit unbound status and warning; they must not be silently treated as bound.
|
| 1265 |
+
- Bound-key mismatch, mixed IDs and digest mismatch must fail closed with zero replacements.
|
| 1266 |
+
- Human review remains required; production readiness remains false.
|
| 1267 |
+
|
| 1268 |
+
Intentionally not changed:
|
| 1269 |
+
- product code or UI;
|
| 1270 |
+
- Scrub Key schema/version or serialization;
|
| 1271 |
+
- placeholder generation or grammar;
|
| 1272 |
+
- export/download or reinsert semantics;
|
| 1273 |
+
- document processing;
|
| 1274 |
+
- cloud, AI, OCR or secret storage.
|
| 1275 |
+
|
| 1276 |
+
Next recommended step:
|
| 1277 |
+
- Start `SCRUB-WP_MVP_SCRUB_KEY_BINDING_CONTRACT_TESTS` before implementation.
|
| 1278 |
+
|
| 1279 |
+
---
|
| 1280 |
+
|
| 1281 |
+
## 2026-07-27 18:55 Europe/Amsterdam — SCRUB-WP_MVP_SCRUB_KEY_ROUNDTRIP_VALIDATION
|
| 1282 |
+
|
| 1283 |
+
Status: completed; deterministic validation passed; PR verification pending.
|
| 1284 |
+
|
| 1285 |
+
Purpose:
|
| 1286 |
+
- Validate Scrub Key import/reinsert and placeholder roundtrip behavior against adversarial synthetic mutations.
|
| 1287 |
+
- Record evidence before authorizing any schema, export or reinsert changes.
|
| 1288 |
+
|
| 1289 |
+
Files added:
|
| 1290 |
+
- `test_cases/mvp_phase6/scrub_key_roundtrip_manifest.json`
|
| 1291 |
+
- `mvp_scrub_key_roundtrip_validation.py`
|
| 1292 |
+
- `scripts/run_mvp_scrub_key_roundtrip_validation.py`
|
| 1293 |
+
- `tests/test_mvp_scrub_key_roundtrip_validation.py`
|
| 1294 |
+
- `tests/test_mvp_scrub_key_roundtrip_report_contract.py`
|
| 1295 |
+
- `output/validation/mvp_scrub_key_roundtrip_validation_report.json`
|
| 1296 |
+
- `handover/workpackages/20260727_1855_mvp_scrub_key_roundtrip_validation.md`
|
| 1297 |
+
|
| 1298 |
+
Files changed:
|
| 1299 |
+
- `WORKPACKAGES.md`
|
| 1300 |
+
- `CHANGELOG.md`
|
| 1301 |
+
- `ROADMAP.md`
|
| 1302 |
+
- `MVP_PHASE6_EXECUTION_PLAN.md`
|
| 1303 |
+
- `RISK_REGISTER.md`
|
| 1304 |
+
- `DECISION_LOG.md`
|
| 1305 |
+
- `workpackage_claims/scrub_wp_mvp_scrub_key_roundtrip_validation.md`
|
| 1306 |
+
|
| 1307 |
+
Validation result:
|
| 1308 |
+
- 15 synthetic cases; 0 failed cases.
|
| 1309 |
+
- 1 critical finding: no reliable document/key binding when a wrong valid key reuses the same placeholder namespace.
|
| 1310 |
+
- 1 medium finding: malformed placeholder mutations outside the grammar are signalled indirectly.
|
| 1311 |
+
- Existing duplicate, incomplete, invalid, unknown and translated cases fail closed or remain visibly auditable as expected.
|
| 1312 |
+
- Local-only: true; external AI: false; cloud processing: false.
|
| 1313 |
+
- Production ready: false; human review required: true.
|
| 1314 |
+
|
| 1315 |
+
Intentionally not changed:
|
| 1316 |
+
- product code or UI;
|
| 1317 |
+
- Scrub Key schema, mappings, export, storage or lifecycle;
|
| 1318 |
+
- placeholder grammar or automatic repair;
|
| 1319 |
+
- reinsert helper semantics;
|
| 1320 |
+
- filenames, MIME types or audit fields;
|
| 1321 |
+
- cloud, AI, OCR or restored-PDF behavior.
|
| 1322 |
+
|
| 1323 |
+
Next recommended step:
|
| 1324 |
+
- Start `SCRUB-WP_MVP_SCRUB_KEY_DOCUMENT_BINDING_GAP_TRIAGE` before implementing a fix.
|
| 1325 |
+
|
| 1326 |
+
---
|
| 1327 |
+
|
| 1328 |
+
## 2026-07-27 18:28 Europe/Amsterdam — SCRUB-WP_MVP_REINSERT_AUTO_FLOW_SIMPLIFICATION_APP_VERIFY_CLOSEOUT
|
| 1329 |
+
|
| 1330 |
+
Status: completed and app-verified.
|
| 1331 |
+
|
| 1332 |
+
Purpose:
|
| 1333 |
+
- Record successful live verification of the document-first automatic reinsert workflow.
|
| 1334 |
+
- Close the evidence-driven UI blocker before continuing Scrub Key roundtrip validation.
|
| 1335 |
+
|
| 1336 |
+
Validation result:
|
| 1337 |
+
- PR #38 merge commit: `390f381c1464883f220716655c5067dadd0bb4c9`.
|
| 1338 |
+
- Final clean PR GitHub Actions run #1678: passed.
|
| 1339 |
+
- Full repository suite before merge: 797 passed.
|
| 1340 |
+
- Hugging Face deployment: confirmed by live testing of the merged three-step workflow.
|
| 1341 |
+
- App verification: passed; coordinator reported `getest en werkend`.
|
| 1342 |
+
|
| 1343 |
+
Verified product boundaries:
|
| 1344 |
+
- Document/text remains step 1, Scrub Key step 2 and restored download step 3.
|
| 1345 |
+
- Automatic source recognition, key validation and local deterministic reinsert work as intended.
|
| 1346 |
+
- One final confidential-output acknowledgement remains at download.
|
| 1347 |
+
- No Scrub Key schema, helper semantics, export filenames/MIME types, cloud, AI, OCR or restored-PDF behavior changed.
|
| 1348 |
+
- Human review remains required; no production-readiness claim is made.
|
| 1349 |
+
|
| 1350 |
+
Files added:
|
| 1351 |
+
- `workpackage_claims/scrub_wp_mvp_reinsert_auto_flow_simplification_app_verify_closeout.md`
|
| 1352 |
+
- `handover/workpackages/20260727_1828_mvp_reinsert_auto_flow_simplification_app_verify_closeout.md`
|
| 1353 |
+
|
| 1354 |
+
Files changed:
|
| 1355 |
+
- `WORKPACKAGES.md`
|
| 1356 |
+
- `CHANGELOG.md`
|
| 1357 |
+
- `ROADMAP.md`
|
| 1358 |
+
- `workpackage_claims/scrub_wp_mvp_reinsert_auto_flow_simplification_implementation.md`
|
| 1359 |
+
- `handover/workpackages/20260727_1706_mvp_reinsert_auto_flow_simplification_implementation.md`
|
| 1360 |
+
|
| 1361 |
+
Next recommended step:
|
| 1362 |
+
- Start `SCRUB-WP_MVP_SCRUB_KEY_ROUNDTRIP_VALIDATION`.
|
| 1363 |
+
|
| 1364 |
+
---
|
| 1365 |
+
|
| 1366 |
+
## 2026-07-27 — SCRUB-WP_MVP_REINSERT_AUTO_FLOW_SIMPLIFICATION_IMPLEMENTATION
|
| 1367 |
+
|
| 1368 |
+
Status: implemented; full suite passed; final PR validation pending.
|
| 1369 |
+
|
| 1370 |
+
Purpose:
|
| 1371 |
+
|
| 1372 |
+
- Remove redundant source- and Scrub-Key confirmation steps from the local reinsert workflow after live Phase 6 evidence showed that uploaded inputs looked complete while hidden action gates remained.
|
| 1373 |
+
- Present the workflow in the user’s natural order: source document/text, corresponding Scrub Key, restored result.
|
| 1374 |
+
- Preserve a clear confidentiality decision at the final restored-output download boundary.
|
| 1375 |
+
|
| 1376 |
+
Files added:
|
| 1377 |
+
|
| 1378 |
+
- `reinsert_auto_flow.py`
|
| 1379 |
+
- `tests/test_reinsert_auto_flow.py`
|
| 1380 |
+
- `tests/test_reinsert_auto_flow_ui.py`
|
| 1381 |
+
- `handover/workpackages/20260727_1706_mvp_reinsert_auto_flow_simplification_implementation.md`
|
| 1382 |
+
- `output/validation/mvp_reinsert_auto_flow_validation.json`
|
| 1383 |
+
|
| 1384 |
+
Files changed:
|
| 1385 |
+
|
| 1386 |
+
- `reinsert_mode_ui.py`
|
| 1387 |
+
- `tests/test_reinsert_interface_simplification_ui.py`
|
| 1388 |
+
- `tests/test_mvp_document_fidelity_ui_copy.py`
|
| 1389 |
+
- `tests/test_mvp_document_fidelity_pr_final_contracts.py`
|
| 1390 |
+
- `CHANGELOG.md`
|
| 1391 |
+
- `WORKPACKAGES.md`
|
| 1392 |
+
- `ROADMAP.md`
|
| 1393 |
+
- `RELEASE_NOTES.md`
|
| 1394 |
+
- `DECISION_LOG.md`
|
| 1395 |
+
- `RISK_REGISTER.md`
|
| 1396 |
+
- `workpackage_claims/scrub_wp_mvp_document_hygiene_fidelity_hardening.md`
|
| 1397 |
+
- `handover/workpackages/20260717_2230_mvp_document_hygiene_fidelity_hardening.md`
|
| 1398 |
+
- `workpackage_claims/scrub_wp_mvp_reinsert_auto_flow_simplification_implementation.md`
|
| 1399 |
+
|
| 1400 |
+
Implementation result:
|
| 1401 |
+
|
| 1402 |
+
- Step 1 is now the source document or pasted text.
|
| 1403 |
+
- One uploader recognises TXT, DOCX and text-based PDF by extension.
|
| 1404 |
+
- Step 2 automatically parses and validates the uploaded or pasted Scrub Key.
|
| 1405 |
+
- Local deterministic reinsert runs automatically once one valid source and one valid key are present.
|
| 1406 |
+
- Separate source/key acknowledgement checkboxes and execution buttons were removed.
|
| 1407 |
+
- One final confidentiality acknowledgement remains directly before restored-output download.
|
| 1408 |
+
- Existing output filenames, MIME types, reinsert helpers, audit fields and explicit DOCX/PDF boundaries are preserved.
|
| 1409 |
+
|
| 1410 |
+
Validation:
|
| 1411 |
+
|
| 1412 |
+
- Full repository suite: 797 passed.
|
| 1413 |
+
- Helper dispatch, deterministic request signatures and input precedence are covered.
|
| 1414 |
+
- Source-level UI contracts verify document-first order, automatic key validation, automatic local reinsert and removal of redundant gates.
|
| 1415 |
+
- Prior DOCX live verification passed for body, table, header and footer restoration.
|
| 1416 |
+
- Final GitHub Actions, merge, Hugging Face sync and live app verification remain pending.
|
| 1417 |
+
|
| 1418 |
+
Intentionally not changed:
|
| 1419 |
+
|
| 1420 |
+
- Scrub Key schema, mappings, lifecycle or storage;
|
| 1421 |
+
- document replacement or reinsert helper semantics;
|
| 1422 |
+
- recognizers or thresholds;
|
| 1423 |
+
- export filenames, MIME types or audit semantics;
|
| 1424 |
+
- cloud, AI or OCR processing;
|
| 1425 |
+
- restored-PDF support;
|
| 1426 |
+
- unsupported DOCX-part boundaries;
|
| 1427 |
+
- the requirement for human review and a final confidential-output warning.
|
| 1428 |
+
|
| 1429 |
+
Next recommended step:
|
| 1430 |
+
|
| 1431 |
+
- Complete final PR validation, merge and sync, then live-verify the three-step automatic flow.
|
| 1432 |
+
- Continue with `SCRUB-WP_MVP_SCRUB_KEY_ROUNDTRIP_VALIDATION` after app verification.
|
| 1433 |
+
|
| 1434 |
+
---
|
| 1435 |
+
|
| 1436 |
+
## 2026-07-17 — SCRUB-WP_MVP_DOCUMENT_HYGIENE_FIDELITY_HARDENING
|
| 1437 |
+
|
| 1438 |
+
Status: completed / ready for PR verification.
|
| 1439 |
+
|
| 1440 |
+
Purpose:
|
| 1441 |
+
|
| 1442 |
+
- Resolve the reproducible DOCX header/footer reinsert fidelity gap from the Phase 6 matrix.
|
| 1443 |
+
- Preserve DOCX hygiene visibility and explicit unsupported-part boundaries.
|
| 1444 |
+
- Keep the PDF restored-TXT-only/no-OCR boundary unchanged.
|
| 1445 |
+
|
| 1446 |
+
Files added:
|
| 1447 |
+
|
| 1448 |
+
- `mvp_document_fidelity_report.py`
|
| 1449 |
+
- `scripts/run_mvp_document_hygiene_fidelity_report.py`
|
| 1450 |
+
- `tests/test_mvp_document_hygiene_fidelity_hardening.py`
|
| 1451 |
+
- `tests/test_mvp_document_fidelity_report.py`
|
| 1452 |
+
- `tests/test_mvp_document_fidelity_ui_copy.py`
|
| 1453 |
+
- `tests/test_mvp_document_fidelity_pr_final_contracts.py`
|
| 1454 |
+
- `output/validation/mvp_phase6_document_hygiene_fidelity_hardening_report.json`
|
| 1455 |
+
- `output/validation/mvp_document_fidelity_pr_validation.json`
|
| 1456 |
+
- `output/validation/mvp_document_fidelity_pr_validation.log`
|
| 1457 |
+
- `handover/workpackages/20260717_2230_mvp_document_hygiene_fidelity_hardening.md`
|
| 1458 |
+
|
| 1459 |
+
Files changed:
|
| 1460 |
+
|
| 1461 |
+
- `scrub_key_document_reinsert.py`
|
| 1462 |
+
- `reinsert_mode_ui.py`
|
| 1463 |
+
- `mvp_phase6_document_cases.py`
|
| 1464 |
+
- `tests/test_mvp_phase6_e2e_synthetic_validation_matrix.py`
|
| 1465 |
+
- `CHANGELOG.md`
|
| 1466 |
+
- `WORKPACKAGES.md`
|
| 1467 |
+
- `RELEASE_NOTES.md`
|
| 1468 |
+
- `DECISION_LOG.md`
|
| 1469 |
+
- `RISK_REGISTER.md`
|
| 1470 |
+
- `workpackage_claims/scrub_wp_mvp_document_hygiene_fidelity_hardening.md`
|
| 1471 |
+
|
| 1472 |
+
Implementation result:
|
| 1473 |
+
|
| 1474 |
+
- DOCX body paragraphs and tables remain supported.
|
| 1475 |
+
- `word/header*.xml` and `word/footer*.xml` text nodes are restored deterministically.
|
| 1476 |
+
- The DOCX reinsert capability copy matches the supported body/table/header/footer scope.
|
| 1477 |
+
- The synthetic header/footer residual-placeholder finding is resolved: `true`.
|
| 1478 |
+
- Resolved findings: 1.
|
| 1479 |
+
- Remaining findings: 1.
|
| 1480 |
+
- The remaining finding is the explicit PDF restored-TXT-only/no-OCR product boundary.
|
| 1481 |
+
|
| 1482 |
+
Intentionally not changed:
|
| 1483 |
+
|
| 1484 |
+
- recognizers, thresholds or replacement semantics;
|
| 1485 |
+
- Scrub Key schema or lifecycle;
|
| 1486 |
+
- DOCX comments, tracked-change-only parts, footnotes/endnotes, text boxes or metadata;
|
| 1487 |
+
- split-placeholder support across Word text nodes;
|
| 1488 |
+
- export filenames or MIME types;
|
| 1489 |
+
- restored PDF or OCR support;
|
| 1490 |
+
- Streamlit controls/flow, runtime or dependencies; only capability copy was aligned.
|
| 1491 |
+
|
| 1492 |
+
Next recommended step:
|
| 1493 |
+
|
| 1494 |
+
- After Actions, sync and app verification, start `SCRUB-WP_MVP_SCRUB_KEY_ROUNDTRIP_VALIDATION`.
|
| 1495 |
+
|
| 1496 |
+
---
|
| 1497 |
+
|
| 1498 |
+
## 2026-07-17 — Hugging Face Space runtime incident recovery and sync-churn guard
|
| 1499 |
+
|
| 1500 |
+
Status: completed and app-verified.
|
| 1501 |
+
|
| 1502 |
+
Purpose:
|
| 1503 |
+
|
| 1504 |
+
- Restore the Hugging Face Space after it entered an error/rebuild state.
|
| 1505 |
+
- Diagnose the incident without exposing secrets or changing product behavior.
|
| 1506 |
+
- Prevent clearly non-runtime-only commits from repeatedly rebuilding the live Space.
|
| 1507 |
+
|
| 1508 |
+
Result:
|
| 1509 |
+
|
| 1510 |
+
- Sanitized runtime evidence observed the Space first at `BUILDING` and subsequently at `RUNNING`.
|
| 1511 |
+
- Streamlit started on port 7860 and the Flair model loaded.
|
| 1512 |
+
- The coordinator confirmed that the application opens again.
|
| 1513 |
+
- PR #35 added a conservative deployment `paths-ignore` guard while preserving runtime-relevant deployments and manual dispatch.
|
| 1514 |
+
- Temporary incident recovery/probe workflows and triggers were removed after verification.
|
| 1515 |
+
|
| 1516 |
+
Intentionally not changed:
|
| 1517 |
+
|
| 1518 |
+
- product code or UI;
|
| 1519 |
+
- recognizers, thresholds or replacement semantics;
|
| 1520 |
+
- export, Scrub Key or reinsert semantics;
|
| 1521 |
+
- dependencies, Dockerfile, hardware or Hugging Face configuration;
|
| 1522 |
+
- privacy and human-review controls.
|
| 1523 |
+
|
| 1524 |
+
Next recommended step:
|
| 1525 |
+
|
| 1526 |
+
- Resume PR #33 and the Phase 6 document-fidelity sequence.
|
| 1527 |
+
|
| 1528 |
+
---
|
| 1529 |
+
|
| 1530 |
+
## 2026-07-17 — SCRUB-WP_MVP_FALSE_NEGATIVE_GAP_TRIAGE
|
| 1531 |
+
|
| 1532 |
+
Status: completed / ready for PR verification.
|
| 1533 |
+
|
| 1534 |
+
Purpose:
|
| 1535 |
+
|
| 1536 |
+
- Classify every evidence gap from the first Phase 6 synthetic validation report.
|
| 1537 |
+
- Decide whether the evidence justifies recognizer or threshold changes.
|
| 1538 |
+
- Route document-fidelity and product-boundary findings to the correct next package.
|
| 1539 |
+
|
| 1540 |
+
Files added:
|
| 1541 |
+
|
| 1542 |
+
- `MVP_PHASE6_FALSE_NEGATIVE_GAP_TRIAGE.md`
|
| 1543 |
+
- `output/validation/mvp_phase6_false_negative_gap_triage.json`
|
| 1544 |
+
- `tests/test_mvp_phase6_false_negative_gap_triage.py`
|
| 1545 |
+
- `handover/workpackages/20260717_2208_mvp_false_negative_gap_triage.md`
|
| 1546 |
+
|
| 1547 |
+
Files changed:
|
| 1548 |
+
|
| 1549 |
+
- `CHANGELOG.md`
|
| 1550 |
+
- `WORKPACKAGES.md`
|
| 1551 |
+
- `DECISION_LOG.md`
|
| 1552 |
+
- `RISK_REGISTER.md`
|
| 1553 |
+
- `workpackage_claims/scrub_wp_mvp_false_negative_gap_triage.md`
|
| 1554 |
+
|
| 1555 |
+
Triage result:
|
| 1556 |
+
|
| 1557 |
+
- Input evidence gaps: 2.
|
| 1558 |
+
- Reproducible detection false negatives: 0.
|
| 1559 |
+
- Misclassifications: 0.
|
| 1560 |
+
- Legal-role over-masking findings: 0.
|
| 1561 |
+
- Recognizer fix required: `false`.
|
| 1562 |
+
- Next package: `SCRUB-WP_MVP_DOCUMENT_HYGIENE_FIDELITY_HARDENING`.
|
| 1563 |
+
|
| 1564 |
+
Decision:
|
| 1565 |
+
|
| 1566 |
+
- The DOCX finding is a header/footer reinsert and fidelity-scope issue, not a detection failure.
|
| 1567 |
+
- The PDF finding is the approved restored-TXT-only/no-OCR product boundary, not a detection failure.
|
| 1568 |
+
- No recognizer implementation package is opened from this evidence.
|
| 1569 |
+
|
| 1570 |
+
Intentionally not changed:
|
| 1571 |
+
|
| 1572 |
+
- product recognizers or thresholds;
|
| 1573 |
+
- replacement semantics;
|
| 1574 |
+
- document processing or reinsert behavior;
|
| 1575 |
+
- export, Scrub Key or audit semantics;
|
| 1576 |
+
- UI, runtime or dependencies.
|
| 1577 |
+
|
| 1578 |
+
Next recommended step:
|
| 1579 |
+
|
| 1580 |
+
- Start `SCRUB-WP_MVP_DOCUMENT_HYGIENE_FIDELITY_HARDENING`.
|
| 1581 |
+
|
| 1582 |
+
---
|
| 1583 |
+
|
| 1584 |
+
## 2026-07-17 — SCRUB-WP_MVP_E2E_SYNTHETIC_VALIDATION_MATRIX
|
| 1585 |
+
|
| 1586 |
+
Status: completed / ready for PR verification.
|
| 1587 |
+
|
| 1588 |
+
Purpose:
|
| 1589 |
+
|
| 1590 |
+
- Establish a repeatable synthetic evidence baseline for the supported MVP workflow.
|
| 1591 |
+
- Exercise TXT, DOCX and text-based PDF paths across import, review-row replacement, manual addition, Scrub Key creation/validation, deterministic reinsert, export representations and audit evidence.
|
| 1592 |
+
- Record known limitations and reproducible gaps without weakening tests or making production-readiness claims.
|
| 1593 |
+
|
| 1594 |
+
Files added:
|
| 1595 |
+
|
| 1596 |
+
- `test_cases/mvp_phase6/validation_manifest.json`
|
| 1597 |
+
- `mvp_phase6_validation_manifest.py`
|
| 1598 |
+
- `mvp_phase6_detection_matrix.py`
|
| 1599 |
+
- `mvp_phase6_workflow_core.py`
|
| 1600 |
+
- `mvp_phase6_document_cases.py`
|
| 1601 |
+
- `mvp_phase6_validation_report.py`
|
| 1602 |
+
- `scripts/run_mvp_phase6_validation_matrix.py`
|
| 1603 |
+
- `tests/test_mvp_phase6_e2e_synthetic_validation_matrix.py`
|
| 1604 |
+
- `output/validation/mvp_phase6_synthetic_validation_report.json`
|
| 1605 |
+
- `handover/workpackages/20260717_2020_mvp_e2e_synthetic_validation_matrix.md`
|
| 1606 |
+
|
| 1607 |
+
Files changed:
|
| 1608 |
+
|
| 1609 |
+
- `CHANGELOG.md`
|
| 1610 |
+
- `WORKPACKAGES.md`
|
| 1611 |
+
- `RISK_REGISTER.md`
|
| 1612 |
+
- `workpackage_claims/scrub_wp_mvp_e2e_synthetic_validation_matrix.md`
|
| 1613 |
+
|
| 1614 |
+
Validation result:
|
| 1615 |
+
|
| 1616 |
+
- Synthetic cases: 3.
|
| 1617 |
+
- Failing cases: 0.
|
| 1618 |
+
- Evidence gaps/known limitations: 2.
|
| 1619 |
+
- Gap categories: known_docx_reinsert_limitation, known_pdf_reinsert_limitation.
|
| 1620 |
+
- Human review required: `true`.
|
| 1621 |
+
- Production ready: `false`.
|
| 1622 |
+
- Local-only validation: `true`.
|
| 1623 |
+
- External AI/cloud/OCR processing: none.
|
| 1624 |
+
|
| 1625 |
+
Methodology correction:
|
| 1626 |
+
|
| 1627 |
+
- Standard deterministic Presidio email recognition is included alongside the Dutch recognizer pack, preventing a standard e-mail value from being misclassified as a Dutch-pack false negative.
|
| 1628 |
+
|
| 1629 |
+
Intentionally not changed:
|
| 1630 |
+
|
| 1631 |
+
- Streamlit UI or review controls;
|
| 1632 |
+
- recognizers or detection thresholds in product code;
|
| 1633 |
+
- replacement semantics;
|
| 1634 |
+
- export payload, filename or MIME semantics;
|
| 1635 |
+
- Scrub Key schema or lifecycle behavior;
|
| 1636 |
+
- reinsert semantics;
|
| 1637 |
+
- document-processing implementation;
|
| 1638 |
+
- runtime/startup or dependencies.
|
| 1639 |
+
|
| 1640 |
+
Next recommended step:
|
| 1641 |
+
|
| 1642 |
+
- Start `SCRUB-WP_MVP_FALSE_NEGATIVE_GAP_TRIAGE` and classify the report evidence before implementing any fix.
|
| 1643 |
+
|
| 1644 |
+
---
|
| 1645 |
+
|
| 1646 |
+
## 2026-07-17 — SCRUB-WP_MVP_PHASE6_ROADMAP_REALIGNMENT
|
| 1647 |
+
|
| 1648 |
+
Status: completed / ready for PR verification.
|
| 1649 |
+
|
| 1650 |
+
Purpose:
|
| 1651 |
+
|
| 1652 |
+
- Realign the central roadmap after completion and live verification of the MVP UI simplification line.
|
| 1653 |
+
- Make Phase 6 end-to-end workflow validation and trust hardening the active development line.
|
| 1654 |
+
- Define an ordered evidence-driven workpackage queue before pilot or packaging work resumes.
|
| 1655 |
+
|
| 1656 |
+
Files changed:
|
| 1657 |
+
|
| 1658 |
+
- `ROADMAP.md`
|
| 1659 |
+
- `WORKPACKAGES.md`
|
| 1660 |
+
- `CHANGELOG.md`
|
| 1661 |
+
- `DECISION_LOG.md`
|
| 1662 |
+
- `RISK_REGISTER.md`
|
| 1663 |
+
- `workpackage_claims/scrub_wp_mvp_phase6_roadmap_realignment.md`
|
| 1664 |
+
|
| 1665 |
+
Files added:
|
| 1666 |
+
|
| 1667 |
+
- `MVP_PHASE6_EXECUTION_PLAN.md`
|
| 1668 |
+
- `handover/workpackages/20260717_2012_mvp_phase6_roadmap_realignment.md`
|
| 1669 |
+
|
| 1670 |
+
Main changes:
|
| 1671 |
+
|
| 1672 |
+
- The verified UI baseline is no longer the active development focus.
|
| 1673 |
+
- Phase 6 starts with a synthetic end-to-end validation matrix.
|
| 1674 |
+
- False-negative, document-hygiene, Scrub Key/roundtrip and audit work must be driven by reproducible evidence.
|
| 1675 |
+
- Phase 7 pilots and local packaging remain gated.
|
| 1676 |
+
|
| 1677 |
+
Validation status:
|
| 1678 |
+
|
| 1679 |
+
- Documentation-only package.
|
| 1680 |
+
- GitHub Actions pending after PR.
|
| 1681 |
+
- Hugging Face functional sync not applicable.
|
| 1682 |
+
- App verification not applicable.
|
| 1683 |
+
|
| 1684 |
+
Intentionally not changed:
|
| 1685 |
+
|
| 1686 |
+
- product code or tests;
|
| 1687 |
+
- UI behavior;
|
| 1688 |
+
- recognizers or replacement semantics;
|
| 1689 |
+
- export payloads, filenames or MIME types;
|
| 1690 |
+
- Scrub Key JSON or reinsert behavior;
|
| 1691 |
+
- document processing, runtime/startup or dependencies.
|
| 1692 |
+
|
| 1693 |
+
Next recommended step:
|
| 1694 |
+
|
| 1695 |
+
- Start `SCRUB-WP_MVP_E2E_SYNTHETIC_VALIDATION_MATRIX` after this realignment is merged.
|
| 1696 |
+
|
| 1697 |
+
---
|
| 1698 |
+
|
| 1699 |
+
## 2026-07-16 — SCRUB-WP_MANUAL_CORRECTION_PANEL_DENSITY_SIMPLIFICATION_APP_VERIFY_CLOSEOUT
|
| 1700 |
+
|
| 1701 |
+
Status: completed and app-verified.
|
| 1702 |
+
|
| 1703 |
+
Purpose:
|
| 1704 |
+
|
| 1705 |
+
- Record live Hugging Face app verification after PR #28 merged.
|
| 1706 |
+
- Close the manual correction panel density simplification line.
|
| 1707 |
+
- Confirm the compact layout preserves the existing manual correction workflow.
|
| 1708 |
+
|
| 1709 |
+
Verification evidence:
|
| 1710 |
+
|
| 1711 |
+
- Coordinator live app screenshot reviewed at 2026-07-16 23:43 Europe/Amsterdam.
|
| 1712 |
+
- `Gemiste waarde toevoegen` remains collapsed by default and opens without a duplicate internal heading.
|
| 1713 |
+
- The value, type and replacement controls appear in one compact row.
|
| 1714 |
+
- The full-width `Toevoegen aan vervangtabel` action remains visible.
|
| 1715 |
+
- Synthetic value `lantaarnbloem` was added successfully.
|
| 1716 |
+
- The replacement table shows `lantaarnbloem` with `[WAARDE_HANDMATIG_01]` and status `Handmatig toegevoegd`.
|
| 1717 |
+
- No Script execution error is visible.
|
| 1718 |
+
- The live screenshot confirms GitHub-to-Hugging-Face deployment of the merged UI.
|
| 1719 |
+
|
| 1720 |
+
Files changed:
|
| 1721 |
+
|
| 1722 |
+
- `CHANGELOG.md`
|
| 1723 |
+
- `WORKPACKAGES.md`
|
| 1724 |
+
- `workpackage_claims/scrub_wp_manual_correction_panel_density_simplification_implementation.md`
|
| 1725 |
+
- `handover/workpackages/20260716_2040_manual_correction_panel_density_implementation.md`
|
| 1726 |
+
|
| 1727 |
+
Files added:
|
| 1728 |
+
|
| 1729 |
+
- `workpackage_claims/scrub_wp_manual_correction_panel_density_simplification_app_verify_closeout.md`
|
| 1730 |
+
- `handover/workpackages/20260716_2343_manual_correction_panel_density_app_verify_closeout.md`
|
| 1731 |
+
|
| 1732 |
+
Intentionally not changed:
|
| 1733 |
+
|
| 1734 |
+
- product code or tests;
|
| 1735 |
+
- recognizer or replacement semantics;
|
| 1736 |
+
- validation or session-state behavior;
|
| 1737 |
+
- export payloads, filenames or MIME types;
|
| 1738 |
+
- Scrub Key JSON or reinsert behavior;
|
| 1739 |
+
- document processing, startup/runtime or dependencies.
|
| 1740 |
+
|
| 1741 |
+
Next recommended step:
|
| 1742 |
+
|
| 1743 |
+
- Use the simplified MVP UI with representative synthetic legal documents before approving another UI package.
|
| 1744 |
+
|
| 1745 |
+
---
|
| 1746 |
+
|
| 1747 |
+
## 2026-07-16 — SCRUB-WP_MANUAL_CORRECTION_PANEL_DENSITY_SIMPLIFICATION_IMPLEMENTATION
|
| 1748 |
+
|
| 1749 |
+
Status: implemented; local validation passed.
|
| 1750 |
+
|
| 1751 |
+
Purpose:
|
| 1752 |
+
|
| 1753 |
+
- Make the existing `Gemiste waarde toevoegen` panel materially shorter and less form-like.
|
| 1754 |
+
- Remove the duplicate internal heading and group value, type and replacement controls in one compact row.
|
| 1755 |
+
- Preserve the existing validation, session-state and replacement-table workflow.
|
| 1756 |
+
|
| 1757 |
+
Files changed:
|
| 1758 |
+
|
| 1759 |
+
- `presidio_streamlit.py`
|
| 1760 |
+
- `CHANGELOG.md`
|
| 1761 |
+
- `WORKPACKAGES.md`
|
| 1762 |
+
- `RELEASE_NOTES.md`
|
| 1763 |
+
- `workpackage_claims/scrub_wp_manual_correction_panel_density_simplification_implementation.md`
|
| 1764 |
+
|
| 1765 |
+
Files added:
|
| 1766 |
+
|
| 1767 |
+
- `tests/test_manual_correction_panel_density_implementation.py`
|
| 1768 |
+
- `handover/workpackages/20260716_2040_manual_correction_panel_density_implementation.md`
|
| 1769 |
+
|
| 1770 |
+
Validation status:
|
| 1771 |
+
|
| 1772 |
+
- Required worker validation passed.
|
| 1773 |
+
- GitHub Actions pending after PR update.
|
| 1774 |
+
- Hugging Face sync pending after merge.
|
| 1775 |
+
- Live app verification required because visible UI behavior changed.
|
| 1776 |
+
|
| 1777 |
+
Intentionally not changed:
|
| 1778 |
+
|
| 1779 |
+
- validation rules or duplicate detection;
|
| 1780 |
+
- placeholder generation or entity types;
|
| 1781 |
+
- replacement-row structure or replacement semantics;
|
| 1782 |
+
- export payloads, filenames or MIME types;
|
| 1783 |
+
- Scrub Key JSON or warning behavior;
|
| 1784 |
+
- reinsert behavior;
|
| 1785 |
+
- recognizers, thresholds, document processing, runtime/startup or dependencies.
|
| 1786 |
+
|
| 1787 |
+
Next recommended step:
|
| 1788 |
+
|
| 1789 |
+
- Verify PR Actions, merge when green, verify Hugging Face sync and request live app verification.
|
| 1790 |
+
|
| 1791 |
+
---
|
| 1792 |
+
|
| 1793 |
+
## 2026-07-05 — SCRUB-WP_REVIEW_EXPORT_VERTICAL_DENSITY_SIMPLIFICATION_APP_VERIFY_CLOSEOUT
|
| 1794 |
+
|
| 1795 |
+
Status: completed and app-verified.
|
| 1796 |
+
|
| 1797 |
+
Purpose:
|
| 1798 |
+
|
| 1799 |
+
- Record live app verification for `SCRUB-WP_REVIEW_EXPORT_VERTICAL_DENSITY_SIMPLIFICATION_IMPLEMENTATION`.
|
| 1800 |
+
- Close the Review/Export vertical-density implementation line after PR #26 merge and live Hugging Face verification.
|
| 1801 |
+
- Add concise closeout notes for the related plan and contract-test packages that were previously missing from `CHANGELOG.md`.
|
| 1802 |
+
|
| 1803 |
+
Verification evidence:
|
| 1804 |
+
|
| 1805 |
+
- Coordinator live Hugging Face screenshot reviewed.
|
| 1806 |
+
- The app starts without Script execution error.
|
| 1807 |
+
- The input section remains coherent.
|
| 1808 |
+
- `2. Controleer resultaat` remains visible.
|
| 1809 |
+
- Basiscontrole / Expertcontrole remain visible.
|
| 1810 |
+
- `Markeringen tonen` and side-by-side review remain visible.
|
| 1811 |
+
- `Gemiste waarde toevoegen` and the vervangtabel remain accessible.
|
| 1812 |
+
- `3. Exporteer resultaat` remains visible.
|
| 1813 |
+
- TXT/DOCX/PDF downloads are visible in a compact row.
|
| 1814 |
+
- Scrub Key, audit/technical files and DOCX hygiene audit remain separate and accessible.
|
| 1815 |
+
|
| 1816 |
+
Files changed:
|
| 1817 |
+
|
| 1818 |
+
- `CHANGELOG.md`
|
| 1819 |
+
- `WORKPACKAGES.md`
|
| 1820 |
+
- `workpackage_claims/scrub_wp_review_export_vertical_density_simplification_implementation.md`
|
| 1821 |
+
- `workpackage_claims/scrub_wp_review_export_vertical_density_simplification_app_verify_closeout.md`
|
| 1822 |
+
- `handover/workpackages/20260705_2213_review_export_vertical_density_implementation.md`
|
| 1823 |
+
- `handover/workpackages/20260705_2242_review_export_vertical_density_app_verify_closeout.md`
|
| 1824 |
+
|
| 1825 |
+
Intentionally not changed:
|
| 1826 |
+
|
| 1827 |
+
- product code;
|
| 1828 |
+
- tests;
|
| 1829 |
+
- recognizer logic;
|
| 1830 |
+
- replacement logic;
|
| 1831 |
+
- export payloads;
|
| 1832 |
+
- download filenames;
|
| 1833 |
+
- MIME types;
|
| 1834 |
+
- Scrub Key JSON;
|
| 1835 |
+
- reinsert behavior;
|
| 1836 |
+
- startup/runtime behavior.
|
| 1837 |
+
|
| 1838 |
+
Related package closeout:
|
| 1839 |
+
|
| 1840 |
+
- `SCRUB-WP_REVIEW_EXPORT_VERTICAL_DENSITY_SIMPLIFICATION_PLAN` — completed and merged; planning-only.
|
| 1841 |
+
- `SCRUB-WP_REVIEW_EXPORT_VERTICAL_DENSITY_SIMPLIFICATION_CONTRACT_TESTS` — completed and merged; source-level guardrails only.
|
| 1842 |
+
|
| 1843 |
+
Next recommended step:
|
| 1844 |
+
|
| 1845 |
+
- Decide whether the current MVP UI is good enough for this pass, or start a new separately approved small UI package.
|
| 1846 |
+
|
| 1847 |
+
---
|
| 1848 |
+
|
| 1849 |
+
## 2026-07-05 — SCRUB-WP_REVIEW_EXPORT_VERTICAL_DENSITY_SIMPLIFICATION_IMPLEMENTATION
|
| 1850 |
+
|
| 1851 |
+
Status: implemented / local validation passed.
|
| 1852 |
+
|
| 1853 |
+
Purpose:
|
| 1854 |
+
|
| 1855 |
+
- Reduce vertical density in the Review and Export areas after plan and contract-test packages were merged.
|
| 1856 |
+
- Compress repeated Review helper copy while keeping side-by-side review, marker toggle, manual missed-value entry and replacement table accessible.
|
| 1857 |
+
- Put the three primary document downloads in a compact three-column layout.
|
| 1858 |
+
- Keep Scrub Key, audit/technical files and DOCX hygiene audit separate and accessible.
|
| 1859 |
+
|
| 1860 |
+
Files changed:
|
| 1861 |
+
|
| 1862 |
+
- `presidio_streamlit.py`
|
| 1863 |
+
- `side_by_side_review_panel_ui.py`
|
| 1864 |
+
- `CHANGELOG.md`
|
| 1865 |
+
- `WORKPACKAGES.md`
|
| 1866 |
+
- `workpackage_claims/scrub_wp_review_export_vertical_density_simplification_implementation.md`
|
| 1867 |
+
|
| 1868 |
+
Files added:
|
| 1869 |
+
|
| 1870 |
+
- `handover/workpackages/20260705_2213_review_export_vertical_density_implementation.md`
|
| 1871 |
+
|
| 1872 |
+
Validation status:
|
| 1873 |
+
|
| 1874 |
+
- Local validation passed.
|
| 1875 |
+
- GitHub Actions pending after PR.
|
| 1876 |
+
- Hugging Face sync pending after merge.
|
| 1877 |
+
- App verification required after sync because visible UI behavior changed.
|
| 1878 |
+
|
| 1879 |
+
Intentionally not changed:
|
| 1880 |
+
|
| 1881 |
+
- recognizer logic;
|
| 1882 |
+
- replacement logic;
|
| 1883 |
+
- review table semantics;
|
| 1884 |
+
- export payloads;
|
| 1885 |
+
- download filenames;
|
| 1886 |
+
- MIME types;
|
| 1887 |
+
- Scrub Key JSON semantics;
|
| 1888 |
+
- Scrub Key warning meaning;
|
| 1889 |
+
- reinsert behavior;
|
| 1890 |
+
- DOCX/PDF parsing;
|
| 1891 |
+
- startup/runtime behavior;
|
| 1892 |
+
- dependencies;
|
| 1893 |
+
- benchmark or recall logic.
|
| 1894 |
+
|
| 1895 |
+
Next recommended step:
|
| 1896 |
+
|
| 1897 |
+
- Open PR, verify GitHub Actions, merge when green, verify Hugging Face sync, then request live app verification.
|
| 1898 |
+
|
| 1899 |
+
---
|
| 1900 |
+
|
| 1901 |
+
## 2026-07-05 — SCRUB-WP_DUPLICATE_INPUT_SURFACE_SIMPLIFICATION_IMPLEMENTATION
|
| 1902 |
+
|
| 1903 |
+
Status: implemented / local validation passed.
|
| 1904 |
+
|
| 1905 |
+
Purpose:
|
| 1906 |
+
|
| 1907 |
+
- Resume the duplicate input surface implementation after the connector-only worker was blocked.
|
| 1908 |
+
- Keep one visible `1. Voeg document of tekst toe` step and group upload, synthetic example selection and pasted/extracted text into one input surface.
|
| 1909 |
+
- Preserve TXT/DOCX/PDF upload support, synthetic legal examples, pasted/extracted text handling and input precedence.
|
| 1910 |
+
- Preserve review, export/download, Scrub Key, reinsert, audit and DOCX hygiene behavior.
|
| 1911 |
+
|
| 1912 |
+
Files changed:
|
| 1913 |
+
|
| 1914 |
+
- `presidio_streamlit.py`
|
| 1915 |
+
- `tests/test_duplicate_input_surface_simplification_contracts.py`
|
| 1916 |
+
- `WORKPACKAGES.md`
|
| 1917 |
+
- `CHANGELOG.md`
|
| 1918 |
+
- `workpackage_claims/scrub_wp_duplicate_input_surface_simplification_implementation.md`
|
| 1919 |
+
|
| 1920 |
+
Files added:
|
| 1921 |
+
|
| 1922 |
+
- `handover/workpackages/20260705_0113_duplicate_input_surface_implementation.md`
|
| 1923 |
+
|
| 1924 |
+
Validation status:
|
| 1925 |
+
|
| 1926 |
+
- Local validation passed:
|
| 1927 |
+
- `python -m pytest -q tests/test_duplicate_input_surface_simplification_contracts.py`
|
| 1928 |
+
- related UI/export guardrail tests
|
| 1929 |
+
- `git diff --check`
|
| 1930 |
+
- GitHub Actions pending after PR.
|
| 1931 |
+
- Hugging Face sync pending after merge.
|
| 1932 |
+
- App verification required after merge/sync because visible UI grouping changed.
|
| 1933 |
+
|
| 1934 |
+
Intentionally not changed:
|
| 1935 |
+
|
| 1936 |
+
- document parsing behavior;
|
| 1937 |
+
- upload backend;
|
| 1938 |
+
- recognizer logic;
|
| 1939 |
+
- replacement logic;
|
| 1940 |
+
- export content;
|
| 1941 |
+
- download filenames;
|
| 1942 |
+
- download MIME types;
|
| 1943 |
+
- Scrub Key JSON semantics;
|
| 1944 |
+
- reinsert behavior;
|
| 1945 |
+
- runtime/startup behavior;
|
| 1946 |
+
- dependencies.
|
| 1947 |
+
|
| 1948 |
+
Next recommended step:
|
| 1949 |
+
|
| 1950 |
+
- Open PR, verify GitHub Actions, merge when green, verify Hugging Face sync, and request live app verification.
|
| 1951 |
+
|
| 1952 |
+
---
|
| 1953 |
+
|
| 1954 |
+
## 2026-07-04 — SCRUB-WP_DUPLICATE_INPUT_SURFACE_SIMPLIFICATION_CONTRACT_TESTS
|
| 1955 |
+
|
| 1956 |
+
Status: completed / PR validation pending.
|
| 1957 |
+
|
| 1958 |
+
Purpose:
|
| 1959 |
+
|
| 1960 |
+
- Lock the single-input-surface contract before the duplicate input implementation touches `presidio_streamlit.py`.
|
| 1961 |
+
- Preserve existing TXT/DOCX/PDF upload support, synthetic legal example support, pasted/extracted text handling and input precedence.
|
| 1962 |
+
- Preserve review, replacement table, Scrub Key, export/download, audit and DOCX hygiene surfaces.
|
| 1963 |
+
- Block duplicate-input runtime/startup patching and prohibited scope expansion.
|
| 1964 |
+
|
| 1965 |
+
Files added:
|
| 1966 |
+
|
| 1967 |
+
- `tests/test_duplicate_input_surface_simplification_contracts.py`
|
| 1968 |
+
- `workpackage_claims/scrub_wp_duplicate_input_surface_simplification_contract_tests.md`
|
| 1969 |
+
- `handover/workpackages/20260704_2318_duplicate_input_surface_contract_tests.md`
|
| 1970 |
+
|
| 1971 |
+
Files changed:
|
| 1972 |
+
|
| 1973 |
+
- `WORKPACKAGES.md`
|
| 1974 |
+
- `CHANGELOG.md`
|
| 1975 |
+
|
| 1976 |
+
Validation status:
|
| 1977 |
+
|
| 1978 |
+
- Source-level contract tests added.
|
| 1979 |
+
- GitHub Actions pending after PR.
|
| 1980 |
+
- Hugging Face sync not applicable until merge.
|
| 1981 |
+
- App verification not applicable because this package changes no UI behavior.
|
| 1982 |
+
|
| 1983 |
+
Intentionally not changed:
|
| 1984 |
+
|
| 1985 |
+
- product implementation code;
|
| 1986 |
+
- Streamlit UI behavior;
|
| 1987 |
+
- document ingestion behavior;
|
| 1988 |
+
- export content;
|
| 1989 |
+
- download filenames;
|
| 1990 |
+
- download MIME types;
|
| 1991 |
+
- Scrub Key JSON semantics;
|
| 1992 |
+
- reinsert behavior;
|
| 1993 |
+
- recognizer logic;
|
| 1994 |
+
- benchmark logic;
|
| 1995 |
+
- DOCX/PDF parsing behavior;
|
| 1996 |
+
- runtime/startup behavior;
|
| 1997 |
+
- dependencies.
|
| 1998 |
+
|
| 1999 |
+
Next recommended step:
|
| 2000 |
+
|
| 2001 |
+
- Review PR validation.
|
| 2002 |
+
- If green, merge this contract-test package.
|
| 2003 |
+
- Then start `SCRUB-WP_DUPLICATE_INPUT_SURFACE_SIMPLIFICATION_IMPLEMENTATION` as the next narrow implementation package.
|
| 2004 |
+
|
| 2005 |
+
---
|
| 2006 |
+
|
| 2007 |
+
## 2026-07-03 — SCRUB-WP_SECONDARY_CONTROL_GROUPING_POLISH_IMPLEMENTATION
|
| 2008 |
+
|
| 2009 |
+
Status: implemented / PR validation pending.
|
| 2010 |
+
|
| 2011 |
+
Purpose:
|
| 2012 |
+
|
| 2013 |
+
- Make the secondary review controls under `2. Controleer resultaat` calmer and easier to understand.
|
| 2014 |
+
- Add a clear `Meer controleopties` grouping cue below the side-by-side review without introducing nested Streamlit expanders.
|
| 2015 |
+
- Preserve side-by-side review, manual missed-value entry, replacement table, serial review, Scrub Key, export/download, audit and DOCX hygiene controls.
|
| 2016 |
+
|
| 2017 |
+
Files changed:
|
| 2018 |
+
|
| 2019 |
+
- `side_by_side_review_panel_ui.py`
|
| 2020 |
+
- `RELEASE_NOTES.md`
|
| 2021 |
+
- `CHANGELOG.md`
|
| 2022 |
+
- `workpackage_claims/scrub_wp_secondary_control_grouping_polish_implementation.md`
|
| 2023 |
+
|
| 2024 |
+
Files added:
|
| 2025 |
+
|
| 2026 |
+
- `tests/test_secondary_control_grouping_polish_implementation.py`
|
| 2027 |
+
- `handover/workpackages/20260703_0000_secondary_control_grouping_polish_implementation.md`
|
| 2028 |
+
|
| 2029 |
+
Validation status:
|
| 2030 |
+
|
| 2031 |
+
- Source-level implementation tests added.
|
| 2032 |
+
- GitHub Actions pending after PR.
|
| 2033 |
+
- Hugging Face sync pending after merge.
|
| 2034 |
+
- App verification required after merge/sync because visible UI behavior changed.
|
| 2035 |
+
|
| 2036 |
+
Intentionally not changed:
|
| 2037 |
+
|
| 2038 |
+
- replacement logic;
|
| 2039 |
+
- review table data semantics;
|
| 2040 |
+
- export content;
|
| 2041 |
+
- download filenames;
|
| 2042 |
+
- download MIME types;
|
| 2043 |
+
- Scrub Key JSON semantics;
|
| 2044 |
+
- Scrub Key warning meaning;
|
| 2045 |
+
- reinsert behavior;
|
| 2046 |
+
- recognizer logic;
|
| 2047 |
+
- benchmark logic;
|
| 2048 |
+
- DOCX/PDF parsing behavior;
|
| 2049 |
+
- runtime/startup behavior;
|
| 2050 |
+
- dependencies.
|
| 2051 |
+
|
| 2052 |
+
Next recommended step:
|
| 2053 |
+
|
| 2054 |
+
- Review PR validation.
|
| 2055 |
+
- If green, merge and verify Hugging Face sync.
|
| 2056 |
+
- Then request coordinator live app verification for the new `Meer controleopties` grouping cue.
|
| 2057 |
+
|
| 2058 |
+
---
|
| 2059 |
+
|
| 2060 |
+
## 2026-07-03 — SCRUB-WP_SECONDARY_CONTROL_GROUPING_POLISH
|
| 2061 |
+
|
| 2062 |
+
Status: implemented planning/contract-tests-only; PR validation pending.
|
| 2063 |
+
|
| 2064 |
+
Purpose:
|
| 2065 |
+
|
| 2066 |
+
- Prepare the final small UI-polish step for calmer grouping of secondary review controls.
|
| 2067 |
+
- Avoid nested Streamlit expanders before touching `presidio_streamlit.py`.
|
| 2068 |
+
- Preserve side-by-side review, manual missed-value entry, replacement table, serial review, Scrub Key, export/download, audit and DOCX hygiene controls.
|
| 2069 |
+
|
| 2070 |
+
Files added:
|
| 2071 |
+
|
| 2072 |
+
- `SECONDARY_CONTROL_GROUPING_POLISH_PLAN.md`
|
| 2073 |
+
- `tests/test_secondary_control_grouping_polish_contracts.py`
|
| 2074 |
+
- `workpackage_claims/scrub_wp_secondary_control_grouping_polish.md`
|
| 2075 |
+
- `handover/workpackages/20260703_0000_secondary_control_grouping_polish.md`
|
| 2076 |
+
|
| 2077 |
+
Validation status:
|
| 2078 |
+
|
| 2079 |
+
- Source-level contract tests added.
|
| 2080 |
+
- GitHub Actions pending after PR.
|
| 2081 |
+
- Hugging Face sync not applicable until merge.
|
| 2082 |
+
- App verification not applicable because no UI behavior changed in this planning/contract step.
|
| 2083 |
+
|
| 2084 |
+
Intentionally not changed:
|
| 2085 |
+
|
| 2086 |
+
- product code;
|
| 2087 |
+
- Streamlit UI;
|
| 2088 |
+
- replacement logic;
|
| 2089 |
+
- review table data semantics;
|
| 2090 |
+
- export content;
|
| 2091 |
+
- download filenames;
|
| 2092 |
+
- download MIME types;
|
| 2093 |
+
- Scrub Key JSON semantics;
|
| 2094 |
+
- Scrub Key warning meaning;
|
| 2095 |
+
- reinsert behavior;
|
| 2096 |
+
- recognizer logic;
|
| 2097 |
+
- benchmark logic;
|
| 2098 |
+
- DOCX/PDF parsing behavior;
|
| 2099 |
+
- runtime/startup behavior;
|
| 2100 |
+
- dependencies.
|
| 2101 |
+
|
| 2102 |
+
Next recommended step:
|
| 2103 |
+
|
| 2104 |
+
- Run PR validation for the contract tests.
|
| 2105 |
+
- If green, merge this plan/contract package.
|
| 2106 |
+
- Then start `SCRUB-WP_SECONDARY_CONTROL_GROUPING_POLISH_IMPLEMENTATION` as the actual narrow UI implementation.
|
| 2107 |
+
|
| 2108 |
+
---
|
| 2109 |
+
|
| 2110 |
+
## 2026-07-03 — SCRUB-WP_REVIEW_SURFACE_SIMPLIFICATION_IMPLEMENTATION
|
| 2111 |
+
|
| 2112 |
+
Status: completed and app-verified.
|
| 2113 |
+
|
| 2114 |
+
Purpose:
|
| 2115 |
+
|
| 2116 |
+
- Make the side-by-side review surface calmer and less form-like.
|
| 2117 |
+
- Keep the source-vs-processed comparison central while pointing users toward the safe download step.
|
| 2118 |
+
- Preserve review table, manual missed-value entry, serial review, Scrub Key, export/download and audit controls.
|
| 2119 |
+
|
| 2120 |
+
Files changed:
|
| 2121 |
+
|
| 2122 |
+
- `side_by_side_review_panel_ui.py`
|
| 2123 |
+
- `tests/test_review_copy_polish_ui.py`
|
| 2124 |
+
- `tests/test_side_by_side_review_consolidation_dutch_sample.py`
|
| 2125 |
+
- `tests/test_side_by_side_review_ui_patch.py`
|
| 2126 |
+
- `WORKPACKAGES.md`
|
| 2127 |
+
- `RELEASE_NOTES.md`
|
| 2128 |
+
- `workpackage_claims/scrub_wp_review_surface_simplification_implementation.md`
|
| 2129 |
+
|
| 2130 |
+
Files added:
|
| 2131 |
+
|
| 2132 |
+
- `tests/test_review_surface_simplification_implementation.py`
|
| 2133 |
+
- `handover/workpackages/20260703_0000_review_surface_simplification_implementation.md`
|
| 2134 |
+
|
| 2135 |
+
Validation status:
|
| 2136 |
+
|
| 2137 |
+
- Source-level implementation tests added.
|
| 2138 |
+
- Related copy-polish and side-by-side tests updated.
|
| 2139 |
+
- PR #12 initially failed on stale copy expectations; a narrow test-expectation fix was applied.
|
| 2140 |
+
- PR #12 Tests passed after the narrow fix.
|
| 2141 |
+
- PR #12 merged to `main`.
|
| 2142 |
+
- Main Tests for commit `41cf304` passed.
|
| 2143 |
+
- GitHub to Hugging Face sync for commit `41cf304` passed.
|
| 2144 |
+
- App verification passed by coordinator screenshot.
|
| 2145 |
+
|
| 2146 |
+
Intentionally not changed:
|
| 2147 |
+
|
| 2148 |
+
- replacement logic;
|
| 2149 |
+
- review table data semantics;
|
| 2150 |
+
- export content;
|
| 2151 |
+
- download filenames;
|
| 2152 |
+
- download MIME types;
|
| 2153 |
+
- Scrub Key JSON semantics;
|
| 2154 |
+
- Scrub Key warning meaning;
|
| 2155 |
+
- reinsert behavior;
|
| 2156 |
+
- recognizer logic;
|
| 2157 |
+
- benchmark logic;
|
| 2158 |
+
- DOCX/PDF parsing behavior;
|
| 2159 |
+
- runtime/startup behavior;
|
| 2160 |
+
- dependencies.
|
| 2161 |
+
|
| 2162 |
+
Next recommended step:
|
| 2163 |
+
|
| 2164 |
+
- Do not start broader UI/reinsert/export work without a dedicated workpackage.
|
| 2165 |
+
- Decide whether further secondary-control grouping is desired as a separate small package, or return to recall/benchmark follow-up if product UI is good enough for the current MVP pass.
|
| 2166 |
+
|
| 2167 |
+
---
|
| 2168 |
+
|
| 2169 |
+
## 2026-06-23 20:52 Europe/Amsterdam — Full-suite validation update — SCRUB-WP_REINSERT_INTERFACE_SIMPLIFICATION_IMPLEMENTATION
|
| 2170 |
+
|
| 2171 |
+
## 2026-06-23 — SCRUB-WP_DOCX_SIDE_BY_SIDE_TEXT_ORDER_TRIAGE
|
| 2172 |
+
|
| 2173 |
+
- Fixed DOCX plain-text extraction order for side-by-side preview.
|
| 2174 |
+
- DOCX body paragraphs and tables are now read in document XML order instead of all paragraphs first and all tables afterwards.
|
| 2175 |
+
- Added synthetic regression coverage for interleaved paragraph/table order.
|
| 2176 |
+
- Preserved DOCX export, Scrub Key and reinsert semantics.
|
| 2177 |
+
- Validation: `python -m pytest tests -x -vv` → 649 passed in 102.51s.
|
| 2178 |
+
|
| 2179 |
+
- Full suite passed: `python -m pytest tests -x -vv` → 647 passed in 108.30s.
|
| 2180 |
+
- `git diff --check` passed.
|
| 2181 |
+
- Local implementation validation complete.
|
| 2182 |
+
- GitHub Actions, GitHub to Hugging Face sync and live app verification remain pending until PR/merge/sync.
|
| 2183 |
+
|
| 2184 |
+
## 2026-06-23 20:43 Europe/Amsterdam — SCRUB-WP_REINSERT_INTERFACE_SIMPLIFICATION_IMPLEMENTATION
|
| 2185 |
+
|
| 2186 |
+
- Implemented direct-source reinsert interface simplification.
|
| 2187 |
+
- Added `reinsert_mode_ui.py` with the visible four-step reinsert flow:
|
| 2188 |
+
1. Voeg Scrub Key toe
|
| 2189 |
+
2. Voeg tekst of document toe
|
| 2190 |
+
3. Controleer herstelrapport
|
| 2191 |
+
4. Download herstelde output
|
| 2192 |
+
- Added a minimal direct hook in `presidio_streamlit.py`.
|
| 2193 |
+
- Added a no-op guard to `fix_streamlit_pdf_text_reinsert.py` so startup does not mutate the direct-source reinsert UI.
|
| 2194 |
+
- Preserved Scrub Key warnings, acknowledgement gates, restored download filenames, MIME types and local-only/no-AI/no-cloud/no-OCR/no-restored-PDF boundaries.
|
| 2195 |
+
- Added `tests/test_reinsert_interface_simplification_ui.py`.
|
| 2196 |
+
|
| 2197 |
+
Validation so far:
|
| 2198 |
+
- `tests/test_reinsert_interface_simplification_ui.py`: 8 passed
|
| 2199 |
+
- reinsert patch tests: 39 passed
|
| 2200 |
+
- warning/two-mode UI tests: 23 passed
|
| 2201 |
+
|
| 2202 |
+
|
| 2203 |
+
# Changelog — SolidPrivacy Scrub
|
| 2204 |
+
|
| 2205 |
+
## SCRUB-WP_EXECUTION_INTERFACE_SIMPLIFICATION_IMPLEMENTATION_RESTART — Execution interface simplification
|
| 2206 |
+
|
| 2207 |
+
Status: completed and verified on `main` via PR #6 (`a34700c`).
|
| 2208 |
+
|
| 2209 |
+
Summary:
|
| 2210 |
+
|
| 2211 |
+
- Simplified the default Scrub flow toward `1. Voeg document of tekst toe`, `2. Controleer resultaat`, `3. Exporteer resultaat`.
|
| 2212 |
+
- Edited `presidio_streamlit.py` directly; no startup patch, runtime hook, sitecustomize hook or Dockerfile startup change was added.
|
| 2213 |
+
- Collapsed secondary controls by default while keeping them available:
|
| 2214 |
+
- control-mode explanation;
|
| 2215 |
+
- recognition details;
|
| 2216 |
+
- review guidance;
|
| 2217 |
+
- manual missed-value entry;
|
| 2218 |
+
- focus filter / extra control helpers;
|
| 2219 |
+
- candidate audit values;
|
| 2220 |
+
- replacement table;
|
| 2221 |
+
- technical replacement details;
|
| 2222 |
+
- step-by-step review;
|
| 2223 |
+
- reusable replacements;
|
| 2224 |
+
- Scrub Key download;
|
| 2225 |
+
- audit/technical downloads.
|
| 2226 |
+
- Kept side-by-side review visible as the main review surface.
|
| 2227 |
+
- Kept the replacement table as source of truth and export input.
|
| 2228 |
+
- Kept primary document downloads visible.
|
| 2229 |
+
|
| 2230 |
+
Tests:
|
| 2231 |
+
|
| 2232 |
+
- `tests/test_execution_interface_simplification_ui.py` — 6 passed.
|
| 2233 |
+
- Side-by-side, serial review and replace logic UI tests — 37 passed.
|
| 2234 |
+
- Export/download contract and implementation tests — 19 passed.
|
| 2235 |
+
- `git diff --check` — passed.
|
| 2236 |
+
- Full local test suite — 639 passed.
|
| 2237 |
+
- PR #6 checks — green.
|
| 2238 |
+
- Main Tests — green.
|
| 2239 |
+
- GitHub to Hugging Face sync — green.
|
| 2240 |
+
- Live app verification — passed by coordinator screenshot on 2026-06-23.
|
| 2241 |
+
|
| 2242 |
+
Intentionally not changed:
|
| 2243 |
+
|
| 2244 |
+
- export semantics;
|
| 2245 |
+
- download file contents;
|
| 2246 |
+
- download filenames;
|
| 2247 |
+
- Scrub Key JSON semantics;
|
| 2248 |
+
- Scrub Key warning meaning;
|
| 2249 |
+
- reinsert behavior;
|
| 2250 |
+
- recognizer logic;
|
| 2251 |
+
- benchmark thresholds;
|
| 2252 |
+
- document processing behavior;
|
| 2253 |
+
- cloud processing;
|
| 2254 |
+
- local packaging;
|
| 2255 |
+
- Dockerfile startup behavior;
|
| 2256 |
+
- runtime mutation behavior;
|
tests/test_mvp_document_fidelity_pr_final_contracts.py
CHANGED
|
@@ -3,10 +3,7 @@ from pathlib import Path
|
|
| 3 |
|
| 4 |
ROOT = Path(__file__).resolve().parents[1]
|
| 5 |
FIDELITY_HEADING = "## 2026-07-17 — SCRUB-WP_MVP_DOCUMENT_HYGIENE_FIDELITY_HARDENING"
|
| 6 |
-
|
| 7 |
-
"## 2026-07-17 22:30 Europe/Amsterdam — "
|
| 8 |
-
"SCRUB-WP_MVP_DOCUMENT_HYGIENE_FIDELITY_HARDENING"
|
| 9 |
-
)
|
| 10 |
|
| 11 |
|
| 12 |
def _heading_section(text: str, heading: str) -> str:
|
|
@@ -20,9 +17,11 @@ def test_temporary_document_fidelity_patch_scripts_are_absent() -> None:
|
|
| 20 |
assert not (ROOT / "scripts" / "finalize_mvp_document_fidelity_pr_cleanup.py").exists()
|
| 21 |
|
| 22 |
|
| 23 |
-
def
|
| 24 |
-
|
| 25 |
-
|
|
|
|
|
|
|
| 26 |
handover = (
|
| 27 |
ROOT
|
| 28 |
/ "handover"
|
|
@@ -30,22 +29,24 @@ def test_document_fidelity_governance_evidence_has_no_duplicate_lines() -> None:
|
|
| 30 |
/ "20260717_2230_mvp_document_hygiene_fidelity_hardening.md"
|
| 31 |
).read_text(encoding="utf-8")
|
| 32 |
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
workpackages,
|
| 36 |
-
FIDELITY_WORKPACKAGE_HEADING,
|
| 37 |
-
)
|
| 38 |
|
| 39 |
-
assert
|
| 40 |
-
assert
|
| 41 |
"The DOCX reinsert capability copy matches the supported body/table/header/footer scope."
|
| 42 |
) == 1
|
| 43 |
-
assert workpackage_section.count(
|
| 44 |
-
"Aligned existing DOCX reinsert copy with the supported body/table/header/footer scope without adding controls."
|
| 45 |
-
) == 1
|
| 46 |
assert handover.count("- `tests/test_mvp_document_fidelity_ui_copy.py`") == 1
|
| 47 |
assert handover.count("- Capability-copy contract tests.") == 1
|
| 48 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
|
| 50 |
def test_final_claim_preserves_verification_and_product_boundaries() -> None:
|
| 51 |
claim = (
|
|
|
|
| 3 |
|
| 4 |
ROOT = Path(__file__).resolve().parents[1]
|
| 5 |
FIDELITY_HEADING = "## 2026-07-17 — SCRUB-WP_MVP_DOCUMENT_HYGIENE_FIDELITY_HARDENING"
|
| 6 |
+
D030_HEADING = "## D030 — 2026-07-17 — Supported DOCX reinsert includes body, tables, headers and footers"
|
|
|
|
|
|
|
|
|
|
| 7 |
|
| 8 |
|
| 9 |
def _heading_section(text: str, heading: str) -> str:
|
|
|
|
| 17 |
assert not (ROOT / "scripts" / "finalize_mvp_document_fidelity_pr_cleanup.py").exists()
|
| 18 |
|
| 19 |
|
| 20 |
+
def test_document_fidelity_history_is_preserved_and_current_scope_remains_bound() -> None:
|
| 21 |
+
archived_changelog = (
|
| 22 |
+
ROOT / "history" / "CHANGELOG_PRE_CONVERGENCE_20260904.md"
|
| 23 |
+
).read_text(encoding="utf-8")
|
| 24 |
+
decisions = (ROOT / "DECISION_LOG.md").read_text(encoding="utf-8")
|
| 25 |
handover = (
|
| 26 |
ROOT
|
| 27 |
/ "handover"
|
|
|
|
| 29 |
/ "20260717_2230_mvp_document_hygiene_fidelity_hardening.md"
|
| 30 |
).read_text(encoding="utf-8")
|
| 31 |
|
| 32 |
+
archived_section = _heading_section(archived_changelog, FIDELITY_HEADING)
|
| 33 |
+
d030 = _heading_section(decisions, D030_HEADING)
|
|
|
|
|
|
|
|
|
|
| 34 |
|
| 35 |
+
assert archived_section.count("- `tests/test_mvp_document_fidelity_ui_copy.py`") == 1
|
| 36 |
+
assert archived_section.count(
|
| 37 |
"The DOCX reinsert capability copy matches the supported body/table/header/footer scope."
|
| 38 |
) == 1
|
|
|
|
|
|
|
|
|
|
| 39 |
assert handover.count("- `tests/test_mvp_document_fidelity_ui_copy.py`") == 1
|
| 40 |
assert handover.count("- Capability-copy contract tests.") == 1
|
| 41 |
|
| 42 |
+
# Current product authority lives in D030, not in a historical WORKPACKAGES slot.
|
| 43 |
+
assert "body, tables, headers and footers" in d030
|
| 44 |
+
assert "word/document.xml" in d030
|
| 45 |
+
assert "word/header*.xml" in d030
|
| 46 |
+
assert "word/footer*.xml" in d030
|
| 47 |
+
assert "comments" in d030
|
| 48 |
+
assert "footnotes/endnotes" in d030
|
| 49 |
+
|
| 50 |
|
| 51 |
def test_final_claim_preserves_verification_and_product_boundaries() -> None:
|
| 52 |
claim = (
|
tests/test_premium_core_flow_ui_realignment_plan.py
CHANGED
|
@@ -45,27 +45,37 @@ def test_plan_preserves_safety_and_roundtrip_semantics() -> None:
|
|
| 45 |
assert required in text
|
| 46 |
|
| 47 |
|
| 48 |
-
def
|
| 49 |
roadmap = ROADMAP.read_text(encoding="utf-8")
|
| 50 |
workpackages = WORKPACKAGES.read_text(encoding="utf-8")
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
assert "
|
| 54 |
-
assert "
|
| 55 |
-
assert "
|
| 56 |
-
assert "
|
| 57 |
-
assert "
|
| 58 |
-
assert "
|
| 59 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
|
| 61 |
|
| 62 |
def test_decision_log_records_global_view_and_stage_model() -> None:
|
| 63 |
text = DECISIONS.read_text(encoding="utf-8")
|
| 64 |
|
| 65 |
-
assert "D041" in text
|
| 66 |
-
assert "
|
| 67 |
-
assert "
|
| 68 |
-
assert "
|
|
|
|
|
|
|
| 69 |
|
| 70 |
|
| 71 |
def test_claim_is_planning_only() -> None:
|
|
|
|
| 45 |
assert required in text
|
| 46 |
|
| 47 |
|
| 48 |
+
def test_current_strategy_preserves_implemented_ui_contract_without_reactivating_old_queue() -> None:
|
| 49 |
roadmap = ROADMAP.read_text(encoding="utf-8")
|
| 50 |
workpackages = WORKPACKAGES.read_text(encoding="utf-8")
|
| 51 |
+
decisions = DECISIONS.read_text(encoding="utf-8")
|
| 52 |
+
|
| 53 |
+
assert "generation-bound Standard/Expert workflow state" in roadmap
|
| 54 |
+
assert "## D041" in decisions
|
| 55 |
+
assert "## D043" in decisions
|
| 56 |
+
assert "global `Standaard | Expert` presentation" in decisions
|
| 57 |
+
assert "exactly one Standard stage is dominant at a time" in decisions
|
| 58 |
+
assert "Shared Streamlit/review/export/runtime surfaces remain sequential" in workpackages
|
| 59 |
+
|
| 60 |
+
# Historical implementation packages are evidence, not the current queue.
|
| 61 |
+
for obsolete_current_package in [
|
| 62 |
+
"SCRUB-WP_PREMIUM_CORE_FLOW_UI_CONTRACT",
|
| 63 |
+
"SCRUB-WP_PREMIUM_CORE_FLOW_STATE_MODEL",
|
| 64 |
+
"SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION",
|
| 65 |
+
"SCRUB-WP_PREMIUM_CORE_FLOW_APP_VERIFY_CLOSEOUT",
|
| 66 |
+
]:
|
| 67 |
+
assert obsolete_current_package not in workpackages
|
| 68 |
|
| 69 |
|
| 70 |
def test_decision_log_records_global_view_and_stage_model() -> None:
|
| 71 |
text = DECISIONS.read_text(encoding="utf-8")
|
| 72 |
|
| 73 |
+
assert "## D041" in text
|
| 74 |
+
assert "global `Standaard | Expert` presentation" in text
|
| 75 |
+
assert "presentation-only switching" in text
|
| 76 |
+
assert "## D043" in text
|
| 77 |
+
assert "exactly one Standard stage is dominant at a time" in text
|
| 78 |
+
assert "Standard is lower cognitive load, not lower safety" in text
|
| 79 |
|
| 80 |
|
| 81 |
def test_claim_is_planning_only() -> None:
|
tests/test_premium_staged_workspace_decision.py
CHANGED
|
@@ -59,30 +59,39 @@ def test_decision_preserves_state_safety_and_product_semantics() -> None:
|
|
| 59 |
assert required in text
|
| 60 |
|
| 61 |
|
| 62 |
-
def
|
| 63 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
|
| 65 |
-
|
| 66 |
-
assert "SCRUB-WP_PREMIUM_STAGED_WORKSPACE_DECISION_FREEZE — URGENT CURRENT PACKAGE" in
|
| 67 |
-
assert "production integration paused until package 3 is incorporated" in
|
| 68 |
-
assert "PR #85 is **amended, not discarded**" in
|
| 69 |
-
assert "one persistent document workspace" in text
|
| 70 |
-
assert "no three routed pages" in text
|
| 71 |
|
| 72 |
|
| 73 |
-
def
|
| 74 |
workpackages = WORKPACKAGES.read_text(encoding="utf-8")
|
| 75 |
decisions = DECISION_LOG.read_text(encoding="utf-8")
|
| 76 |
|
| 77 |
-
assert "
|
| 78 |
-
assert "
|
| 79 |
-
assert "SCRUB-WP_PREMIUM_INPUT_STAGE_SIMPLIFICATION" in workpackages
|
| 80 |
-
assert "SCRUB-WP_PREMIUM_REVIEW_STAGE_SIMPLIFICATION" in workpackages
|
| 81 |
-
assert "SCRUB-WP_PREMIUM_EXPORT_STAGE_SIMPLIFICATION" in workpackages
|
| 82 |
-
assert "SCRUB-WP_PREMIUM_EXPERT_PARITY_REGRESSION" in workpackages
|
| 83 |
-
assert "SCRUB-WP_PREMIUM_CORE_FLOW_APP_VERIFY_CLOSEOUT" in workpackages
|
| 84 |
-
assert "do not run the shared Streamlit UI packages in parallel" in workpackages
|
| 85 |
-
|
| 86 |
-
assert "D043" in decisions
|
| 87 |
assert "single-page staged document workspace" in decisions
|
| 88 |
-
assert "three separate routed screens" in decisions
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 59 |
assert required in text
|
| 60 |
|
| 61 |
|
| 62 |
+
def test_current_roadmap_preserves_staged_workspace_without_reactivating_old_gate() -> None:
|
| 63 |
+
roadmap = ROADMAP.read_text(encoding="utf-8")
|
| 64 |
+
decisions = DECISION_LOG.read_text(encoding="utf-8")
|
| 65 |
+
|
| 66 |
+
assert "generation-bound Standard/Expert workflow state" in roadmap
|
| 67 |
+
assert "Stage 1 — Repository Convergence — CURRENT" in roadmap
|
| 68 |
+
assert "## D043" in decisions
|
| 69 |
+
assert "single-page staged document workspace" in decisions
|
| 70 |
+
assert "three routed pages" in decisions
|
| 71 |
|
| 72 |
+
# Old implementation-gate prose is historical, not current strategy.
|
| 73 |
+
assert "SCRUB-WP_PREMIUM_STAGED_WORKSPACE_DECISION_FREEZE — URGENT CURRENT PACKAGE" not in roadmap
|
| 74 |
+
assert "production integration paused until package 3 is incorporated" not in roadmap
|
| 75 |
+
assert "PR #85 is **amended, not discarded**" not in roadmap
|
|
|
|
|
|
|
| 76 |
|
| 77 |
|
| 78 |
+
def test_current_workpackages_preserve_shared_surface_sequencing_not_historical_package_order() -> None:
|
| 79 |
workpackages = WORKPACKAGES.read_text(encoding="utf-8")
|
| 80 |
decisions = DECISION_LOG.read_text(encoding="utf-8")
|
| 81 |
|
| 82 |
+
assert "Shared Streamlit/review/export/runtime surfaces remain sequential" in workpackages
|
| 83 |
+
assert "## D043" in decisions
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 84 |
assert "single-page staged document workspace" in decisions
|
| 85 |
+
assert "three separate routed screens" not in decisions
|
| 86 |
+
assert "three routed pages" in decisions
|
| 87 |
+
|
| 88 |
+
for obsolete_current_package in [
|
| 89 |
+
"SCRUB-WP_PREMIUM_STAGED_WORKSPACE_DECISION_FREEZE",
|
| 90 |
+
"SCRUB-WP_PREMIUM_APP_SHELL_IMPLEMENTATION",
|
| 91 |
+
"SCRUB-WP_PREMIUM_INPUT_STAGE_SIMPLIFICATION",
|
| 92 |
+
"SCRUB-WP_PREMIUM_REVIEW_STAGE_SIMPLIFICATION",
|
| 93 |
+
"SCRUB-WP_PREMIUM_EXPORT_STAGE_SIMPLIFICATION",
|
| 94 |
+
"SCRUB-WP_PREMIUM_EXPERT_PARITY_REGRESSION",
|
| 95 |
+
"SCRUB-WP_PREMIUM_CORE_FLOW_APP_VERIFY_CLOSEOUT",
|
| 96 |
+
]:
|
| 97 |
+
assert obsolete_current_package not in workpackages
|
tests/test_reinsert_auto_flow_app_verify_closeout.py
CHANGED
|
@@ -4,10 +4,11 @@ from pathlib import Path
|
|
| 4 |
ROOT = Path(__file__).resolve().parents[1]
|
| 5 |
|
| 6 |
|
| 7 |
-
def
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
|
|
|
| 11 |
claim = (
|
| 12 |
ROOT
|
| 13 |
/ "workpackage_claims"
|
|
@@ -20,13 +21,19 @@ def test_reinsert_auto_flow_closeout_is_recorded() -> None:
|
|
| 20 |
/ "20260727_1706_mvp_reinsert_auto_flow_simplification_implementation.md"
|
| 21 |
).read_text(encoding="utf-8")
|
| 22 |
|
| 23 |
-
assert "
|
| 24 |
-
assert "SCRUB-WP_MVP_SCRUB_KEY_ROUNDTRIP_VALIDATION" in roadmap
|
| 25 |
-
assert "SCRUB-WP_MVP_REINSERT_AUTO_FLOW_SIMPLIFICATION_APP_VERIFY_CLOSEOUT" in workpackages
|
| 26 |
-
assert "coordinator reported `getest en werkend`" in changelog
|
| 27 |
assert "Status: completed and app-verified" in claim
|
| 28 |
assert "Completed and app-verified." in handover
|
| 29 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
|
| 31 |
def test_closeout_preserves_product_boundaries() -> None:
|
| 32 |
closeout = (
|
|
|
|
| 4 |
ROOT = Path(__file__).resolve().parents[1]
|
| 5 |
|
| 6 |
|
| 7 |
+
def test_reinsert_auto_flow_closeout_history_and_current_contract_are_preserved() -> None:
|
| 8 |
+
archived_changelog = (
|
| 9 |
+
ROOT / "history" / "CHANGELOG_PRE_CONVERGENCE_20260904.md"
|
| 10 |
+
).read_text(encoding="utf-8")
|
| 11 |
+
decisions = (ROOT / "DECISION_LOG.md").read_text(encoding="utf-8")
|
| 12 |
claim = (
|
| 13 |
ROOT
|
| 14 |
/ "workpackage_claims"
|
|
|
|
| 21 |
/ "20260727_1706_mvp_reinsert_auto_flow_simplification_implementation.md"
|
| 22 |
).read_text(encoding="utf-8")
|
| 23 |
|
| 24 |
+
assert "coordinator reported `getest en werkend`" in archived_changelog
|
|
|
|
|
|
|
|
|
|
| 25 |
assert "Status: completed and app-verified" in claim
|
| 26 |
assert "Completed and app-verified." in handover
|
| 27 |
|
| 28 |
+
# Current authority lives in accepted decisions, not old roadmap/workpackage slots.
|
| 29 |
+
assert "## D031" in decisions
|
| 30 |
+
assert "source document/text" in decisions
|
| 31 |
+
assert "corresponding Scrub Key" in decisions
|
| 32 |
+
assert "deterministic validation/reinsert" in decisions
|
| 33 |
+
assert "meaningful confidentiality acknowledgement" in decisions
|
| 34 |
+
assert "## D037" in decisions
|
| 35 |
+
assert "fails closed with zero replacements" in decisions
|
| 36 |
+
|
| 37 |
|
| 38 |
def test_closeout_preserves_product_boundaries() -> None:
|
| 39 |
closeout = (
|
tests/test_repository_convergence_bootstrap_contracts.py
ADDED
|
@@ -0,0 +1,228 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import re
|
| 2 |
+
from pathlib import Path
|
| 3 |
+
|
| 4 |
+
|
| 5 |
+
ROOT = Path(__file__).resolve().parents[1]
|
| 6 |
+
|
| 7 |
+
|
| 8 |
+
def read(name: str) -> str:
|
| 9 |
+
return (ROOT / name).read_text(encoding="utf-8")
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
def test_roadmap_has_only_the_five_current_macro_stages() -> None:
|
| 13 |
+
roadmap = read("ROADMAP.md")
|
| 14 |
+
|
| 15 |
+
stage_headings = re.findall(
|
| 16 |
+
r"^## Stage (\d+) — (.+?)(?: — CURRENT)?$",
|
| 17 |
+
roadmap,
|
| 18 |
+
flags=re.MULTILINE,
|
| 19 |
+
)
|
| 20 |
+
assert stage_headings == [
|
| 21 |
+
("1", "Repository Convergence"),
|
| 22 |
+
("2", "Scrub Private Application"),
|
| 23 |
+
("3", "Private Service"),
|
| 24 |
+
("4", "External Product & Service Assurance"),
|
| 25 |
+
("5", "Pilot"),
|
| 26 |
+
]
|
| 27 |
+
|
| 28 |
+
for milestone in [
|
| 29 |
+
"SCRUB_REPOSITORY_CONVERGED",
|
| 30 |
+
"SCRUB_HF_APPLICATION_COMPLETE",
|
| 31 |
+
"SCRUB_PRIVATE_SERVICE_CANDIDATE",
|
| 32 |
+
]:
|
| 33 |
+
assert milestone in roadmap
|
| 34 |
+
|
| 35 |
+
assert "Phase 9 — Final local desktop/offline installer path" not in roadmap
|
| 36 |
+
assert "active draft PR #85" not in roadmap
|
| 37 |
+
assert "The broader direction is a local-first" not in roadmap
|
| 38 |
+
|
| 39 |
+
|
| 40 |
+
def test_workpackages_contains_one_current_queue_not_historical_overrides() -> None:
|
| 41 |
+
workpackages = read("WORKPACKAGES.md")
|
| 42 |
+
|
| 43 |
+
assert "# SolidPrivacy Scrub — Current Workpackages" in workpackages
|
| 44 |
+
assert "Repository Convergence" in workpackages
|
| 45 |
+
assert "SCRUB-WP_REPOSITORY_CONVERGENCE_BOOTSTRAP" in workpackages
|
| 46 |
+
assert "Current execution status override" not in workpackages
|
| 47 |
+
assert "WP-CONVERGENCE-02..N" in workpackages
|
| 48 |
+
assert "DERIVED, NOT PRE-INVENTED" in workpackages
|
| 49 |
+
assert "SCRUB_REPOSITORY_CONVERGED" in workpackages
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
def test_project_prompt_preserves_start_governance_and_safety_contracts() -> None:
|
| 53 |
+
prompt = read("PROJECT_PROMPT.md")
|
| 54 |
+
|
| 55 |
+
ordered = [
|
| 56 |
+
"1. `PROJECT_PROMPT.md`",
|
| 57 |
+
"2. `ROADMAP.md`",
|
| 58 |
+
"3. `WORKPACKAGES.md`",
|
| 59 |
+
"4. `CHANGELOG.md`",
|
| 60 |
+
]
|
| 61 |
+
positions = [prompt.index(marker) for marker in ordered]
|
| 62 |
+
assert positions == sorted(positions)
|
| 63 |
+
|
| 64 |
+
for marker in [
|
| 65 |
+
"implementation_operations",
|
| 66 |
+
"governance_release_assurance",
|
| 67 |
+
"PASS`, `FAIL` or `INDETERMINATE",
|
| 68 |
+
"Scrub Private",
|
| 69 |
+
"Repository Convergence",
|
| 70 |
+
"human review",
|
| 71 |
+
"false negatives",
|
| 72 |
+
"GitHub→Hugging Face",
|
| 73 |
+
"handover/workpackages/YYYYMMDD_HHMM_<workpackage_slug>.md",
|
| 74 |
+
]:
|
| 75 |
+
assert marker in prompt
|
| 76 |
+
|
| 77 |
+
assert "Immediate line: v12 Review UX" not in prompt
|
| 78 |
+
assert "broader direction is a local-first" not in prompt
|
| 79 |
+
|
| 80 |
+
|
| 81 |
+
def test_short_prompt_routes_workers_to_the_same_current_direction() -> None:
|
| 82 |
+
short = read("PROJECT_PROMPT_SHORT.md")
|
| 83 |
+
|
| 84 |
+
for marker in [
|
| 85 |
+
"Repository Convergence",
|
| 86 |
+
"Scrub Private Application",
|
| 87 |
+
"Private Service",
|
| 88 |
+
"External Product & Service Assurance",
|
| 89 |
+
"Pilot",
|
| 90 |
+
"governance_release_assurance",
|
| 91 |
+
"synthetic/approved-test application-validation",
|
| 92 |
+
]:
|
| 93 |
+
assert marker in short
|
| 94 |
+
|
| 95 |
+
assert "Immediate line: v12 Review UX" not in short
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
def test_agents_routes_claims_and_execution_to_convergence() -> None:
|
| 99 |
+
agents = read("AGENTS.md")
|
| 100 |
+
|
| 101 |
+
for marker in [
|
| 102 |
+
"Repository Convergence",
|
| 103 |
+
"SCRUB_REPOSITORY_CONVERGED",
|
| 104 |
+
"implementation_operations",
|
| 105 |
+
"governance_release_assurance",
|
| 106 |
+
"workpackage_claims/",
|
| 107 |
+
"Do not refactor for aesthetics",
|
| 108 |
+
]:
|
| 109 |
+
assert marker in agents
|
| 110 |
+
|
| 111 |
+
assert "local-first" not in agents.lower()
|
| 112 |
+
|
| 113 |
+
|
| 114 |
+
def test_temporary_ledger_is_explicitly_non_authoritative_and_capability_level() -> None:
|
| 115 |
+
ledger = read("REPOSITORY_CONVERGENCE_DEBT_LEDGER.md")
|
| 116 |
+
|
| 117 |
+
assert "TEMPORARY EXECUTION ARTIFACT — NON-AUTHORITATIVE AFTER CONVERGENCE" in ledger
|
| 118 |
+
assert "54c73e0ebf5a3a3ed7039a50596fb57694add3cd" in ledger
|
| 119 |
+
|
| 120 |
+
for classification in [
|
| 121 |
+
"CANONICAL",
|
| 122 |
+
"RECONCILE",
|
| 123 |
+
"RETIRE",
|
| 124 |
+
"VARIANT-SPECIFIC",
|
| 125 |
+
]:
|
| 126 |
+
assert classification in ledger
|
| 127 |
+
|
| 128 |
+
for capability in [
|
| 129 |
+
"Persistent replacement memory",
|
| 130 |
+
"Azure AI Language document recognition",
|
| 131 |
+
"OpenAI/Azure OpenAI synthesis operator",
|
| 132 |
+
"Legacy startup patch script invocation in Dockerfile",
|
| 133 |
+
"Recognizer-backed recall benchmark",
|
| 134 |
+
]:
|
| 135 |
+
assert capability in ledger
|
| 136 |
+
|
| 137 |
+
|
| 138 |
+
def test_decision_log_keeps_current_binding_decisions_and_adds_d044() -> None:
|
| 139 |
+
decisions = read("DECISION_LOG.md")
|
| 140 |
+
|
| 141 |
+
for marker in [
|
| 142 |
+
"D044 — 2026-09-04 — Repository Convergence",
|
| 143 |
+
"D043",
|
| 144 |
+
"D042",
|
| 145 |
+
"D041",
|
| 146 |
+
"D040",
|
| 147 |
+
"D039",
|
| 148 |
+
"D037",
|
| 149 |
+
"D036",
|
| 150 |
+
"D034",
|
| 151 |
+
"D033",
|
| 152 |
+
"D031",
|
| 153 |
+
"D030",
|
| 154 |
+
"D021",
|
| 155 |
+
"54c73e0ebf5a3a3ed7039a50596fb57694add3cd",
|
| 156 |
+
]:
|
| 157 |
+
assert marker in decisions
|
| 158 |
+
|
| 159 |
+
assert "main` becomes the active Scrub Private development line" in decisions
|
| 160 |
+
assert "human review" in decisions.lower()
|
| 161 |
+
assert "fail" in decisions.lower() and "closed" in decisions.lower()
|
| 162 |
+
|
| 163 |
+
|
| 164 |
+
def test_risk_register_keeps_critical_product_risks_and_adds_current_truth_risks() -> None:
|
| 165 |
+
risks = read("RISK_REGISTER.md")
|
| 166 |
+
|
| 167 |
+
assert "R1 — False negatives / missed sensitive data" in risks
|
| 168 |
+
assert "R2 — Scrub Key leakage, mismatch or misuse" in risks
|
| 169 |
+
assert "R10 — Zorg under-detection and clinical over-masking" in risks
|
| 170 |
+
assert "R11 — Repository/source-of-truth drift" in risks
|
| 171 |
+
assert "R12 — Legacy runtime mutation / hidden startup authority" in risks
|
| 172 |
+
|
| 173 |
+
for heading in [
|
| 174 |
+
"R1 — False negatives / missed sensitive data",
|
| 175 |
+
"R2 — Scrub Key leakage, mismatch or misuse",
|
| 176 |
+
"R10 — Zorg under-detection and clinical over-masking",
|
| 177 |
+
]:
|
| 178 |
+
start = risks.index(heading)
|
| 179 |
+
section = risks[start : start + 700]
|
| 180 |
+
assert "Impact: `critical`" in section, heading
|
| 181 |
+
|
| 182 |
+
assert "R5 — Scrub Private content-retention / external-egress trust boundary" in risks
|
| 183 |
+
r5 = risks[risks.index("R5 — Scrub Private") : risks.index("R6 — Review UX")]
|
| 184 |
+
assert "Impact: `critical`" in r5
|
| 185 |
+
assert "replacement_memory.py" in r5
|
| 186 |
+
assert "Azure AI Language" in r5
|
| 187 |
+
|
| 188 |
+
|
| 189 |
+
def test_changelog_preserves_preconvergence_history_without_using_it_as_current_queue() -> None:
|
| 190 |
+
changelog = read("CHANGELOG.md")
|
| 191 |
+
archived_path = ROOT / "history" / "CHANGELOG_PRE_CONVERGENCE_20260904.md"
|
| 192 |
+
|
| 193 |
+
assert archived_path.exists()
|
| 194 |
+
archived = archived_path.read_text(encoding="utf-8")
|
| 195 |
+
|
| 196 |
+
assert "SCRUB-WP_REPOSITORY_CONVERGENCE_BOOTSTRAP" in changelog
|
| 197 |
+
assert "history/CHANGELOG_PRE_CONVERGENCE_20260904.md" in changelog
|
| 198 |
+
assert "54c73e0ebf5a3a3ed7039a50596fb57694add3cd" in changelog
|
| 199 |
+
assert "SCRUB-WP_REVIEW_MARKER_COMPACTION_LIVE_REGRESSION_REPAIR" in archived
|
| 200 |
+
assert "historical provenance only" in changelog
|
| 201 |
+
|
| 202 |
+
|
| 203 |
+
def test_bootstrap_does_not_redefine_hugging_face_as_production_assurance() -> None:
|
| 204 |
+
combined = "\n".join(
|
| 205 |
+
[
|
| 206 |
+
read("ROADMAP.md"),
|
| 207 |
+
read("PROJECT_PROMPT.md"),
|
| 208 |
+
read("WORKPACKAGES.md"),
|
| 209 |
+
]
|
| 210 |
+
).lower()
|
| 211 |
+
|
| 212 |
+
assert "hugging face" in combined
|
| 213 |
+
assert "application-validation" in combined or "application validation" in combined
|
| 214 |
+
assert "not the final confidential-production trust environment" in combined
|
| 215 |
+
assert "zero retention" not in combined
|
| 216 |
+
|
| 217 |
+
|
| 218 |
+
def test_bootstrap_does_not_authorize_source_cloning_or_a_new_evidence_framework() -> None:
|
| 219 |
+
roadmap = read("ROADMAP.md")
|
| 220 |
+
ledger = read("REPOSITORY_CONVERGENCE_DEBT_LEDGER.md")
|
| 221 |
+
|
| 222 |
+
assert "no source-tree clone" in roadmap
|
| 223 |
+
assert "No separate Evidence Framework" in roadmap
|
| 224 |
+
assert "/app_v2" not in roadmap
|
| 225 |
+
assert "/scrub-new" not in roadmap
|
| 226 |
+
# One stable semantic prohibition is enough; do not make duplicate prose
|
| 227 |
+
# across documents a release requirement.
|
| 228 |
+
assert "no new framework" in ledger.lower()
|
workpackage_claims/SCRUB-WP_REPOSITORY_CONVERGENCE_BOOTSTRAP.md
ADDED
|
@@ -0,0 +1,83 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Workpackage claim — SCRUB-WP_REPOSITORY_CONVERGENCE_BOOTSTRAP
|
| 2 |
+
|
| 3 |
+
Status: implementation_remediated; final exact-head CI and fresh independent assurance required
|
| 4 |
+
Role: `implementation_operations`
|
| 5 |
+
Issue: #113
|
| 6 |
+
PR: #114
|
| 7 |
+
Branch: `wp/repository-convergence-bootstrap`
|
| 8 |
+
Starting main: `54c73e0ebf5a3a3ed7039a50596fb57694add3cd`
|
| 9 |
+
Started: 2026-09-04 Europe/Amsterdam
|
| 10 |
+
|
| 11 |
+
## Objective
|
| 12 |
+
|
| 13 |
+
Preserve the exact pre-convergence baseline, reconstruct reachable current truth, and reset canonical strategy/execution documentation to the approved Repository Convergence → Scrub Private direction without changing runtime product semantics.
|
| 14 |
+
|
| 15 |
+
## Scope
|
| 16 |
+
|
| 17 |
+
Documentation, governance, issue/current-truth inventory, audit-ledger and governance/contract-test convergence only.
|
| 18 |
+
|
| 19 |
+
No recognizer, review authority, export, Scrub Key, reinsert, document-processing, dependency, runtime or Hugging Face product behavior change is authorized in this package.
|
| 20 |
+
|
| 21 |
+
## Independent assurance history
|
| 22 |
+
|
| 23 |
+
Fresh blind assurance on prior exact head:
|
| 24 |
+
|
| 25 |
+
```text
|
| 26 |
+
ea203abb04f008a7e583387242a6f4917c72e591
|
| 27 |
+
```
|
| 28 |
+
|
| 29 |
+
returned:
|
| 30 |
+
|
| 31 |
+
```text
|
| 32 |
+
FAIL
|
| 33 |
+
```
|
| 34 |
+
|
| 35 |
+
Release-blocking findings:
|
| 36 |
+
|
| 37 |
+
1. exact-head full regression was red: `8 failed, 1256 passed`;
|
| 38 |
+
2. two new convergence tests encoded brittle global-string/prose placement rather than stable semantic contracts;
|
| 39 |
+
3. six legacy tests still required historical Premium/fidelity/reinsert status to remain in current ROADMAP/WORKPACKAGES/CHANGELOG locations.
|
| 40 |
+
|
| 41 |
+
The reviewer found the underlying roadmap/scope/product semantics materially sound and explicitly required a narrow test/governance remediation rather than runtime changes.
|
| 42 |
+
|
| 43 |
+
## Remediation
|
| 44 |
+
|
| 45 |
+
Implementation changed only governance/contract tests:
|
| 46 |
+
|
| 47 |
+
- roadmap macro-stage validation now parses formal `## Stage N — ...` headings and compares the ordered five-stage set;
|
| 48 |
+
- Evidence Framework prohibition is bound to one stable semantic prohibition rather than duplicated prose;
|
| 49 |
+
- historical fidelity completion evidence now reads from the exact archived pre-convergence changelog/handovers, while current DOCX scope is bound to D030;
|
| 50 |
+
- Premium staged-workspace/core-flow tests now bind current semantics to D041/D043 and current shared-surface sequencing instead of requiring superseded package queues to stay current;
|
| 51 |
+
- reinsert completion evidence now reads from archived history/handovers, while current document-first/fail-closed behavior is bound to D031/D037.
|
| 52 |
+
|
| 53 |
+
No runtime/product file was changed by this remediation.
|
| 54 |
+
|
| 55 |
+
## Pre-final validation evidence
|
| 56 |
+
|
| 57 |
+
After the test/governance remediation, branch head:
|
| 58 |
+
|
| 59 |
+
```text
|
| 60 |
+
dcdfb2c84bbaafe0beca79191bee509b9607b461
|
| 61 |
+
```
|
| 62 |
+
|
| 63 |
+
was tested by GitHub Actions `Tests` run:
|
| 64 |
+
|
| 65 |
+
```text
|
| 66 |
+
33922952965
|
| 67 |
+
```
|
| 68 |
+
|
| 69 |
+
Raw job output:
|
| 70 |
+
|
| 71 |
+
```text
|
| 72 |
+
1264 passed in 12.34s
|
| 73 |
+
```
|
| 74 |
+
|
| 75 |
+
That run proves the remediation itself is functionally green. Administrative claim/handover/changelog updates follow, so a **new final exact-head full regression is still mandatory** before the release candidate is frozen.
|
| 76 |
+
|
| 77 |
+
## Required final validation
|
| 78 |
+
|
| 79 |
+
- compare final branch against exact starting main and confirm documentation/governance/tests/administration-only scope;
|
| 80 |
+
- full GitHub Actions regression on the final exact PR head;
|
| 81 |
+
- freeze that exact head;
|
| 82 |
+
- fresh independent `governance_release_assurance` on the new SHA before merge;
|
| 83 |
+
- no prior FAIL/PASS verdict transfers to a changed head.
|