docs/anonymization-gate03e.md

Gate03e — error taxonomy and private review readiness

Offline synthetic development: GO. Tested fail-closed behavior: GO. Model qualification: NO-GO. Review readiness: PARTIAL. Private deployment, real sensitive documents and automated release: NO-GO. No new independent holdout was created.

Synchronization and immutable history

Reviewed start ad9a0003cea9e5dbe39183afbca869c2926b9fff; existing feature worktree and draft PR #3 retained. All remote refs fetched. Integration target remains 281c84172281a2294227f15287a4aded8a7ea948; ancestry verified, no rebase required. No reset, force push, other-worktree modification, merge, deployment, service restart, public listener, remote inference or real sensitive-document processing.

Actual committed recognition/evaluation source: 094ad6052cde66e5a7204fb614eff10c5572f313. Final source-reference classification contract/security verification source: 25064678071d00e7cd6d99ac49fb5e3f6d6a16e2 (only the private review bridge and tests changed thereafter; recognition, candidate provenance and evaluation sources are byte-identical).

Gate03c/03d first reports, first provenance, datasets and freeze files remain byte-for-byte unchanged. historical.py pins and verifies their original hashes. The old frozen 03d runner still refuses updated code instead of presenting a new run as its first result. Historical failing assertions are explicit controls; the current candidate coverage gate is a different, also failing, assertion. All 40+10+120+160 cases are now development evidence, not untouched qualification.

Separate outcomes and measurement scope

score_outcomes reports six mutually exclusive document outcomes:

1. Complete candidate with no measured source-canary leakage.
2. Returned candidate with a partial mask.
3. Returned candidate with a fully unmasked canary (takes precedence if both occur).
4. Processing blocked by conflicting spans.
5. Processing blocked by model/runtime failure.
6. Other incomplete processing.

Per-canary counts retain both partial/full failures when they coexist. A category or exact-boundary error can be completely covered and thus have zero measured canary leakage. A candidate with no measured leakage can still be inexact or unsafe on other text. Blocked cases return no candidate; their gold spans are not leaked candidate output. Empty-gold failures still count as incomplete, never success.

Reports separate raw recognizer exact-span P/R/F1, all-document replacement metrics (incomplete documents count as missing), and returned-candidate-only metrics with an explicit availability/abstention denominator. Conservative all-source coverage is named uncovered_source_canaries_including_incomplete, not emitted leakage. The original aggregate evaluator and saved reports remain unchanged. Unexpected runtime errors are converted to constant safe-blocking codes, with no exception payload in results. Source-offset and overlap validation are unchanged.

Failure taxonomy and limited solutions

The synthetic error taxonomy records exact gold slices, genuine old predictions/scores and causes for all nine problem documents. It derives from the immutable first03d result, not a new model claim or changed gold.

Case/class	Diagnosis and action
025, keine Religion	Boundary/terminology/taxonomy: model masks Religion only, leaving the negative-status word. Bounded status grammar protects the full phrase.
026, religiously unaffiliated	Semantic/terminology miss; Affiliation is falsely ORGANIZATION. Grammar protects the full status expression. The model's header false positive is retained and reported; exact document still fails.
111, konfesslonsfrei	OCR i/l substitution defeats literal terminology; model returns no entities. One-confusable long-status matching covers the original slice without rewriting the source.
125, Hafenbogen Archivs	Entity boundary/semantic head: model masks Hafenbogen only, omitting seven letters in the genitive head Archivs. No OCR corruption. Unresolved; blindly padding following nouns could overmask/cross other entities.
055/063/078/127/137	Identical bounds conflict: model PERSON (four documents) or ORGANIZATION (one, two occurrences) versus terminology RELIGION. All five still block. No span/category is discarded to improve pass rates.

Religion-topic gold protects terminology even without personal attribution. A model PERSON score of 0.9998 on the Bahai topic demonstrates why genuine scores are not calibrated privacy/affiliation evidence. General words/headers also cause false positives; do not relabel the gold to make them correct. Organization-head expansion requires context and independent safety evidence, not fixture suffix padding. Prior company/organization prompt regressions remain historical evidence; no global threshold reduction or organization dictionary was added here.

Three policies were compared: old literal hybrid control, contextual status grammar, and grammar plus scoped OCR recognition. Selection criteria were documented before comparison in the acceptance specification: no new returned canary, no lower candidate availability/exact match, no new exact-span false positive in any suite, and at least one canary improvement. Both proposals pass those development criteria; select grammar plus scoped OCR, improving three canaries. No post-comparison tuning was performed. These criteria do not qualify an independently evaluated model or authorize processing/release.

Status grammar protects explicit DE/EN noun/modifier phrases (including declined German negative determiners and religiously modified English non-affiliation), with bounded whitespace and Unicode word boundaries. It does not infer a person's belief. OCR matching accepts exactly one i/l/1 or o/0 substitution in configured lowercase ASCII status words of 12–40 characters, at whole-word boundaries. Default terms are status vocabulary, not names; no edit distance, arbitrary fuzzy-name matching, insertions/deletions, short-word matching or source correction. The source span is mapped through dehyphenation and labeled deterministic with no fabricated score. Configuration, bounds, confusable classes and versions enter the candidate digest. Independent unit negatives cover nonreligious ambiguity, identifier substrings, combining marks, multiple changes, unsupported substitutions and political status.

Controlled development results

One actual pinned inference per each of 330 unique texts is shared by the three fixed policies; replay times are never described as model inference. A preliminary uncommitted comparison selected the same fixed solution; the final authoritative comparison repeats it from committed clean source for reproducible provenance. The final literal-control model span predictions match the first03d raw predictions for all 160 cases. Model weights/prompts/threshold are unchanged.

Policy	Suite	No measured leak	Partial-mask docs	Fully-unmasked docs	Safely blocked	Document exact match
gate03d_literal_control	qualification (40)	40	0	0	0	87.500%
gate03d_literal_control	validation (10)	10	0	0	0	80.000%
gate03d_literal_control	development03c (120)	120	0	0	0	90.000%
gate03d_literal_control	development03d (160)	151	2	2	5	82.500%
contextual_status	qualification (40)	40	0	0	0	87.500%
contextual_status	validation (10)	10	0	0	0	80.000%
contextual_status	development03c (120)	120	0	0	0	90.000%
contextual_status	development03d (160)	153	1	1	5	83.125%
status_scoped_ocr	qualification (40)	40	0	0	0	87.500%
status_scoped_ocr	validation (10)	10	0	0	0	80.000%
status_scoped_ocr	development03c (120)	120	0	0	0	90.000%
status_scoped_ocr	development03d (160)	154	1	0	5	83.750%

Model/runtime blocks and other incomplete cases are zero in this actual development run; their distinct paths are covered by deterministic tests. On the 160-case set: returned candidates remain 155/160 (96.875%), safe abstention 5/160 (3.125%). Returned religion canaries improve 1 partial + 2 fully unmasked → zero. One partially masked organization remains; no completely unmasked canary remains in returned candidates. No-measured-leakage returned candidates improve 151→154; this does not make those texts universally safe. Exact match improves 132→134/160 (82.5→83.75%). Cases 025 and 111 become exact; 026 retains the ORGANIZATION header false positive. The same five conflicted documents remain unavailable.

All-document exact source-span comparison on the 160 cases (incomplete counts as missing). Other suites/categories are fully recorded in the linked machine report:

Category	Before TP/FP/FN	After TP/FP/FN	Before P/R/F1	After P/R/F1
PERSON	20/5/2	20/5/2	0.800000/0.909091/0.851064	0.800000/0.909091/0.851064
CITY	7/2/1	7/2/1	0.777778/0.875000/0.823529	0.777778/0.875000/0.823529
RELIGION	137/3/13	140/2/10	0.978571/0.913333/0.944828	0.985915/0.933333/0.958904
EMAIL	3/0/1	3/0/1	1.000000/0.750000/0.857143	1.000000/0.750000/0.857143
PHONE	2/0/0	2/0/0	1.000000/1.000000/1.000000	1.000000/1.000000/1.000000
ORGANIZATION	5/12/1	5/12/1	0.294118/0.833333/0.434783	0.294118/0.833333/0.434783

Returned-candidate-only religion P/R/F1 is 0.985915/0.972222/0.979021, FP2/FN4; that subset excludes the five blocked documents and cannot substitute for global availability or quality metrics. Fully covered wrong-category/boundary predictions still count as exact-span errors. All other categories' aggregate metrics are unchanged; no new false positive is introduced on any of the four suites. The previous literal-hybrid status-control false positives remain development debt.

Full per-case predictions, errors, outcomes and all metrics, hashes/source/configuration provenance.

Candidate and human-review boundary

New candidate v2 commits the actual model revision, threshold, prompts, terminology, status/OCR configuration, replacement version/placeholders and normalization into canonical SHA-256. Only digest, pinned revision and safe policy-version enums enter ordinary candidate provenance; raw configuration strings and source entities do not. Configuration drift blocks creation. Complete provenance participates in the existing release digest. All original sensitivity and false release/anonymity/automation flags are retained.

The review/provenance interface and migration specification describes the actual typed private bridge. Current authenticated identity, private operator environment and review ACL are checked before loading candidate text and again before return. Source-page retrieval verifies authoritative immutable byte hash, source classification, strict UTF-8 and original character count, then reauthorizes. An authenticated reviewer can inspect complete original pages and sanitized candidate, categories, span references, uncertainty, blocking reasons and configuration provenance within the authorized private application.

This contract has an isolated synthetic test adapter, no production provider/UI. Displaying review material neither completes an immutable review nor creates an approval. Blocked records have no candidate/digest. Host completion must bind the exact displayed source evidence/candidate/configuration and current actor, then use the unchanged reviewed-release authority. Original evidence remains separate, classified and inaccessible in public metadata/export JSON. Export still requires its own verified review/ACL/destination/expiry/single-use ledger authorization; exported JSON never grants transfer permission.

Original v1 candidate serialization and approval signatures remain valid for that exact object. New v2 provenance cannot be retrofitted into old approvals. Unknown versions and malformed/missing provenance fail closed. A golden v1 fixture from the actual reviewed schema and real SQLite consumption verify compatibility; no key, nonce ledger, stored approval or old candidate was silently migrated/reset. Legacy review explicitly marks configuration unknown. Full rollback/quarantine requirements are in the linked contract. No migration/deployment was performed.

Runtime and tests

Pinned model VAGOsolutions/SauerkrautLM-GLiNER@ba967ee030b7e679dd5422cb16fcd1b379da38e1, CPU, two threads, threshold 0.5; Python 3.11.17, GLiNER 0.2.29, torch 2.8.0+cpu, transformers 4.57.1, NumPy 2.4.6, Pydantic 2.14.1; Ruff 0.17.1. Verified local cache and existing locked environments reused. No dependency/service setup or environment configuration change was necessary.

Final model load 15.254 s, peak RSS 3018.828 MiB versus first03d 3018.203 MiB. On the 160 texts, actual model median/max 95.934/392.201 ms versus historical03d median/max 93.217/125.148 ms; scheduling variance is visible, so no throughput/SLA improvement or causal neural-runtime regression is claimed. Same cached predictions, deterministic/replacement replay median is 0.0883 ms control → 0.1101 ms selected. The 40/10/120 suites' actual model median latencies are 92.660/ 141.536/ 119.453 ms.

The offline audit denies one urllib3 IPv6 capability-check bind attempt; no connect or DNS event is recorded. No public endpoint/listener or remote model was enabled. No new container/service was started in this gate. Prior constrained cloud 8 GiB memory/headroom observations remain historical, not private-host or worst-case current-workload qualification. Maximum-length throughput, actual private-host load, identity/UI, encrypted source retention and operational recovery remain blockers.

Exact argv, environment overrides, UTC timing, source SHA and exit statuses: recognition/full verification, review-classification verification, final request-reference security verification.

Check	Result
Ruff check/format	PASS; 91 Python files formatted
Final default OCR suite	310 passed, 12 skipped, one existing Starlette deprecation warning
Final explicit security subset	241 passed
Offline/UTF-8/atomic-output subset	18 passed
Actual pinned smoke/two-page handoff, old50 canaries, selected170 regression	3 passed
Historical03c control acceptance	FAILED (original model-only control)
Historical03d first-result acceptance	FAILED (immutable first report, not updated candidate inference)
Current03e returned-candidate coverage gate	FAILED (remaining partial organization canary)
Whole opt-in model invocation	3 passed, 3 failed, exit 1; no xfail or relabel
Fixed-policy development comparison	Selection criteria PASS; not independent model qualification

Default skips are six explicit heavy/model/report gates, two optional NLTK legacy characterizations, one legacy OCR opt-in, and three live OCR-backend opt-ins. Unit/security evidence includes actual temporary durable SQLite, concurrency, termination/restart, key rotation, revocation, destination transport and atomic files. Identity, review and source storage use isolated synthetic adapters only; no real production authentication/host test is claimed. The 70 new deterministic regressions cover outcome separation, scoped status/OCR negatives, provenance, legacy approvals, unknown versions, source classification, Unicode, authorization before reads and rechecks, integrity, no approval from review metadata and raw-log suppression. GitHub CI remains lightweight; exact-head CI is checked after push. Its green status does not satisfy any of the three deliberately red quality gates.

Acceptance and independent decisions

The measurable staged acceptance proposal separates model-quality targets from live identity/authorization and host readiness. No new independent corpus is authorized/created before the proposed changes and thresholds are reviewed. Future thresholds require domain-owner acceptance and independent representative testing; finite synthetic zero-leakage is no guarantee.

Decision	Status	Remaining reason
Offline synthetic development	GO	Bounded local runtime and deterministic/security tests pass; known failures explicit.
Fail-closed behavior within tested boundaries	GO	Same five conflicts abstain; runtime/incomplete errors never count as successful candidate output.
Model recognition qualification	NO-GO	Organization canary survives; 83.75% exact match, precision/recall/availability targets unmet; no independent evaluation of this fix.
Human review readiness	PARTIAL	Typed authorized original/candidate access is tested; real IdP, source store, completion evidence and private UI integration BLOCKED.
Private deployment	NO-GO	Actual application/host security, bounded workload and operational acceptance not qualified.
Manually reviewed real sensitive-document processing	NO-GO	Both model-quality and real host/review integration gates unmet.
Automated release	NO-GO unconditionally	anonymous=false, automated_release_eligible=false; no automatic exporter or remote processing.

PR #3 stays draft/open. No merge, deployment, production activation or automatic export.