# Dataset README Classifier Progress ## Milestone 1: Dataset Foundations, Models, Errors, and Schemas Status: completed ### Files created - `DATASET_README_PROGRESS.md` - `src/gcmd_classifier/datasets/__init__.py` - `src/gcmd_classifier/datasets/errors.py` - `src/gcmd_classifier/datasets/models.py` - `src/gcmd_classifier/datasets/schemas.py` - `schemas/dataset_classification_result.schema.json` - `schemas/dataset_extraction_manifest.schema.json` - `schemas/dataset_evidence_packet.schema.json` - `schemas/dataset_comparison.schema.json` - `tests/__init__.py` - `tests/datasets/__init__.py` - `tests/datasets/fixture_manifest.py` - `tests/datasets/test_fixture_manifest.py` - `tests/datasets/test_models_and_schemas.py` ### Files modified - `src/gcmd_classifier/config.py` No files were deleted. ### Requirements implemented - Added a dedicated dataset package without extending or repurposing publication article models. - Added dataset-only enumerations and typed contracts for: - processing, classification, final-candidate, and review status scopes; - dataset stages, README coverage modes, product scopes, evidence roles, and comparison categories; - CMR requests, exact source artifacts, Concept ID-based collection identity, sealed expert-keyword references, and blind collection views; - README candidates, explicit selections, redirect provenance, and retrieval metadata contracts; - extracted pages, sections, blocks, and extraction manifests; - product-scope decisions, separate uncalibrated confidence signals, and source-faithful evidence packets; - blind GCMD classifications, deterministic validation, blind-result seals, processing metadata, and dataset results; and - resolved/unresolved expert inputs and additive post-inference comparison output. - Added dataset-specific exception types under a `DatasetError` hierarchy without changing publication exceptions. - Added namespaced `DatasetCMRSettings`, `DatasetDocumentSettings`, `DatasetExtractionSettings`, `DatasetEvidenceSettings`, and aggregate `DatasetSettings`. - Added four separate formal dataset JSON Schemas. - Added dependency-neutral schema generation and typed-payload validation helpers. - Added a test-only typed fixture-manifest loader that validates metadata only and: - requires unique case IDs and Native IDs; - validates required provenance fields, coverage modes, and SHA-256 formatting; - requires relative paths; - rejects paths escaping the fixture root; - requires resolved paths to identify existing regular files; and - does not read CMR/PDF contents, retrieve resources, parse PDFs, classify content, or use expert ScienceKeywords as expected answers. - Added typed and formal-schema enforcement of the rule that `classification_attempted: false` forbids a classification outcome, classifications, blind result, comparison reference, and nonzero model-call count. - Added regression assertions for unchanged representative `ArticleRecord` serialization and unchanged article schema bytes. ### Important design decisions - CMR Concept ID is represented separately from Native ID and is the authoritative `dataset_key`. - Native ID must equal the exact `short_name + "_" + version` representation; an optional legacy Entry ID remains separately named. - Dataset output models are independent of `ArticleRecord`, `ArticleResult`, and the publication JSON Schema. - Product-match confidence and keyword-evidence confidence are separate explicitly uncalibrated contracts. - Extraction and evidence models contain source-faithful text fields and provenance only; no extraction or selection behavior was implemented. - Formal dataset-result JSON Schema includes a conditional constraint matching the Pydantic blind-attempt invariant. - The empty `tests/__init__.py` marker permits focused tests to import the test-only manifest validator without moving it into application code. ### Test commands and results - `python -m ruff check .` - Passed: `All checks passed!` - `python -m ruff format --check .` - Passed: `66 files already formatted` - `python -m pytest` - Passed: `327 passed, 1 skipped in 0.76s` - The skipped test is the existing optional live integration test. - Focused dataset suite: - `python -m pytest tests/datasets` - Passed: `23 passed in 0.19s` No live network, PDF extraction, model, CMR, README, DNS, or UI calls were made. ### Formatting and static analysis Ruff lint and formatting checks pass for the complete repository. ### Protected source and fixture status The following remained unchanged from their pre-implementation SHA-256 baselines: - `tests/fixtures/datasets/README.md` - `tests/fixtures/datasets/cases.json` - all four `tests/fixtures/datasets/cmr/*.json` files; - all four `tests/fixtures/datasets/pdf/*.pdf` files; - `data/gcmd_hierarchy.json`; - `data/articles.json`; - `prototype/app_hf_poc.py`; and - `schemas/classification_result.schema.json`. `src/gcmd_classifier/models.py`, including `ArticleRecord` and `ArticleResult`, was not modified. ### Deviations from DATASET_README_IMPLEMENTATION_PLAN.md - No scope deviations. - Added `tests/__init__.py` solely as an import marker for the test-only typed manifest loader. - The dataset result schema generator adds the Pydantic cross-field blind-attempt invariant as a JSON Schema conditional because Pydantic model validators are not emitted automatically into generated JSON Schema. ### Known limitations - These are contract foundations only; they do not retrieve, parse, classify, compare, persist, cache, or present dataset content. - Timestamp fields are currently string contracts; stricter timestamp parsing may be added in the milestone that owns source retrieval. - Source artifact storage references and blind-result persistence references are contracts only. - Confidence values are explicitly uncalibrated. - The fixture-manifest loader validates manifest metadata and safe file references only; it intentionally does not verify source-file content hashes or inspect source envelopes. ### Deferred work All Milestones 2 through 11 remain deferred, including CMR and README retrieval, URL/DNS security behavior, PDF extraction, product resolution, evidence selection, shared-classifier adaptation, dataset orchestration, persistence and caching, expert-keyword comparison, and UI changes. ### Acceptance criteria All requested Milestone 1 acceptance criteria are satisfied: - representative typed outputs validate against all four dataset schemas; - unattempted classification invariants are enforced by typed and formal schemas; - the four frozen cases load through one safe manifest validator; - publication models and schema remain unchanged; - the full publication regression suite passes; and - no Milestone 2 work was started. ## Milestone 2: CMR Retrieval, Identity, and Blind Boundary Status: completed ### Files created - `src/gcmd_classifier/datasets/cmr.py` - `tests/datasets/test_cmr.py` ### Files modified - `pyproject.toml` - `src/gcmd_classifier/config.py` - `src/gcmd_classifier/datasets/__init__.py` - `src/gcmd_classifier/datasets/errors.py` - `src/gcmd_classifier/datasets/models.py` - `DATASET_README_PROGRESS.md` No files were deleted. ### Requirements implemented - Added strict CMR `collections.umm_json` URL validation: - exact HTTPS scheme, CMR host, collection-search path, and default HTTPS port; - exactly one decoded non-empty `native_id`; - rejection of user information, fragments, duplicate/missing Native IDs, and authentication-like query parameters; - exact preservation of Native ID capitalization, punctuation, and leading zeros; and - preservation of harmless additional query parameters. - Added public CMR retrieval using `httpx` with an injected client/transport, UTC clock, and monotonic operation clock. - Added approved configurable CMR defaults: - 5-second connection timeout; - 20-second read timeout; - 30-second overall operation limit; - three redirects; - 10 MiB response limit; and - configurable `Client-Id` defaulting to `gcmd-keyword-classifier-mvp`. - Added GET-only manual redirect handling with automatic redirects disabled and complete URL-policy validation for every target. - Added rejection of changed redirect Native IDs, redirect loops, excess redirects, missing/unsupported redirects, non-public statuses, login responses, unsupported media types, encoded entities, unsafe client authorization/cookie state, timeout/transport failures, and oversized responses. - Requests use `Accept: application/json`, `Accept-Encoding: identity`, and the configured `Client-Id`; no authorization or cookie state is permitted. - Added streaming raw-byte size enforcement for transport responses, with support for deterministic pre-buffered injected test responses. - Preserved exact response bytes, decoded UTF-8 source text, submitted/final URL, validated redirect history, retrieval timestamp, HTTP status, allow-listed non-sensitive headers, SHA-256, and derived parsed JSON. - Added complete CMR envelope validation for object root, `hits`, `items`, exactly-one cardinality, `meta`, `umm`, required identity fields, positive revision ID, and valid source arrays used by the blind boundary. - Enforced character-for-character equality among submitted Native ID, returned `meta["native-id"]`, and derived `ShortName + "_" + Version`. - Used CMR Concept ID as the authoritative dataset key. - Preserved all unknown source fields in the trusted exact-source and selected-item objects. - Represented the derived Native ID explicitly without fabricating a preserved source `entry_id`; the optional Milestone 1 legacy mapping field is excluded from serialization. - Added an explicit allow-listed `BlindCollectionView` containing only validated identity, derived identity, approved display fields, and `RelatedUrls` metadata. - Added a separate sealed expert-keyword payload containing the original `umm.ScienceKeywords`, source path, dataset provenance, and deterministic payload hash. - Added safe CMR event logging that records only stage, identities, status, and hashes—not payloads, expert keywords, credentials, protected headers, or local paths. - Added explicit `CMRPreclassificationFailure` output with `classification_attempted: false`, zero model calls, no classifications, no semantic outcome, and no comparison. ### Important design decisions - The adapter separates trusted source preservation, blind-stage metadata, and sealed expert inputs into distinct typed objects. - Blind material is constructed by positive allow-listing; the full collection is never passed into `BlindCollectionView` and then filtered. - The trusted resolution bundle contains both blind and sealed outputs so an orchestration milestone can route them separately; blind-stage interfaces accept only `BlindCollectionView`. - Exact entity-byte preservation uses `Accept-Encoding: identity` and raw response streaming. Non-identity content encoding is rejected rather than silently hashing decoded bytes. - Expert payload hashes cover the dataset key, exact source path, and original expert assignments, not only the keyword list. - Harmless CMR query parameters remain allowed, but redirects may not change the decoded Native ID and the returned envelope must still resolve exactly one matching collection. - `httpx>=0.27,<1` is now an explicit core dependency because public CMR retrieval is an approved Dataset MVP capability. ### Tests and results Focused tests cover: - valid and percent-encoded URLs; - every required URL rejection; - configured headers and absence of credentials/cookies; - manual valid and invalid redirects, loops, and limits; - timeout, connection/TLS-class transport, size, status, login, media-type, and content-encoding failures; - malformed UTF-8/JSON and non-object roots; - zero, multiple, and mismatched response cardinalities; - malformed `meta`/`umm` and every required identity field; - exact Native ID consistency, capitalization, and leading zeros; - exact source bytes, stable hashes, timestamps, safe response metadata, and unknown source fields; - no fabricated `entry_id` plus the documented future legacy Entry ID consistency relationship; - allow-listed blind serialization and stable sealed payload hashing; - expert-keyword sentinel isolation from blind outputs and logs; - explicit zero-inference failure semantics; and - all four frozen real-shape CMR fixtures and their immutable bytes. Commands executed: - `python -m ruff check .` - Passed: `All checks passed!` - `python -m ruff format --check .` - Passed: `68 files already formatted` - `python -m pytest` - Passed: `389 passed, 1 skipped in 0.85s` - Focused CMR suite: - `python -m pytest tests/datasets/test_cmr.py` - Passed: `62 passed` The skipped test is the existing optional live integration test. ### Fixture and protected-file confirmation Pre- and post-implementation SHA-256 verification confirmed no changes to: - `tests/fixtures/datasets/README.md`; - `tests/fixtures/datasets/cases.json`; - all four frozen CMR response files; - all four frozen README PDFs; - `data/gcmd_hierarchy.json`; - `data/articles.json`; - `prototype/app_hf_poc.py`; and - `schemas/classification_result.schema.json`. Publication `ArticleRecord`, `ArticleResult`, shared publication models, and publication behavior were not modified. ### Network, model, and blind-boundary confirmation - Routine tests made no live HTTP, DNS, CMR, README, or model calls. - All HTTP behavior used injected `httpx.MockTransport` handlers. - No README discovery/retrieval, PDF extraction, model prompt, classification, cache, persistence, comparison, or UI behavior was implemented. - Expert ScienceKeyword sentinel values remained present only in the sealed payload and trusted exact source; they did not appear in blind serialization, safe logs, error messages, or blind identity material. ### Deviations from DATASET_README_IMPLEMENTATION_PLAN.md - No scope deviations. - The adapter supports both truly streamed transport responses and already-buffered responses supplied by deterministic injected clients; size validation remains mandatory for both. - CMR responses using a non-identity content encoding are rejected to ensure that the preserved and hashed bytes are the exact JSON entity bytes parsed by the adapter. ### Known limitations - This milestone implements only the public CMR collection stage; it does not discover or retrieve a README. - CMR retrieval supports only the approved Native-ID `collections.umm_json` search representation. - Direct Concept ID and legacy JSON adapters remain unimplemented. - Blind and sealed objects are returned together in a trusted resolution bundle; later orchestration must persist and route them through separate typed interfaces. - No cache or persistence behavior exists yet. - Retrieval timestamps are supplied by the injected clock as UTC strings; persistence policy remains deferred. - Network destination/DNS validation for arbitrary README hosts belongs to Milestone 3; CMR is restricted to one exact approved HTTPS host. ### Deferred work Milestones 3 through 11 remain deferred, including README discovery and selection, public README retrieval and DNS controls, PDF validation/extraction, product resolution, evidence selection, classifier adaptation, blind persistence/caching, expert comparison, and UI changes. ### Acceptance criteria Every Milestone 2 acceptance criterion is satisfied: - a valid public CMR request resolves one typed collection with Concept ID as the authoritative key; - invalid and non-public cases stop explicitly before classification with zero model calls; - all four frozen real-shape fixtures pass identity, revision, hash, and immutability checks; - expert keywords are inaccessible from blind-stage types and absent from blind outputs/logs; - complete dataset and publication tests pass; - protected source and fixture files are unchanged; and - no Milestone 3 work was started. ## Milestone 3: README Discovery, Selection, and Safe PDF Retrieval Status: completed ### Files created - `src/gcmd_classifier/datasets/readme_discovery.py` - `src/gcmd_classifier/datasets/documents.py` - `tests/datasets/test_readme_discovery.py` - `tests/datasets/test_documents.py` ### Files modified - `src/gcmd_classifier/config.py` - `src/gcmd_classifier/datasets/__init__.py` - `src/gcmd_classifier/datasets/errors.py` - `src/gcmd_classifier/datasets/models.py` - `DATASET_README_PROGRESS.md` No files were deleted. ### Requirements implemented - Added blind-safe README discovery that accepts only `BlindCollectionView`, matches only exact case-sensitive `Subtype == "READ-ME"`, and preserves every usable candidate's complete source entry, stable zero-based source position, exact URL, description, unknown fields, duplicates, and source order. - Added explicit zero-, one-, and multiple-candidate states. Missing or empty candidate input produces the structured `NO_README_AVAILABLE` skipped, zero-inference outcome; exact matches without usable URL strings produce `NO_USABLE_README_URL`. - Added explicit selection that requires exactly one source position and exact URL, binds the selection to the complete collection identity, CMR source hash, candidate ID, complete source entry, and timestamp, and rejects stale, altered, arbitrary, normalized, local-path, and multiple selections. - Kept the existing dataset-result `READMESelection` schema reference backward compatible and added a separate `ValidatedREADMESelection` required by retrieval. The retriever has no raw-URL or local-path production entry point. - Added selected-URL-only public HTTP/HTTPS retrieval with default ports only, no user information, fragments, authentication query data, authorization headers, cookies, sessions, or credential requests. - Added injectable HTTP client, DNS resolver, UTC clock, monotonic clock, artifact directory, and run-ID factory. - Added strict hostname syntax checks, two public-address checks immediately before each request, all-address global-policy enforcement for IPv4 and IPv6, mixed-result rejection, and redirect-target revalidation. - Added manual redirects, loop/count controls, authentication-redirect detection, configurable connection/read/overall timeouts, streamed size enforcement, response-metadata limits, declared-length/truncation checks, and identity content encoding. - Added typed handling for authentication responses, login pages, transport/TLS-class failures, timeouts, HTTP failures, unsafe redirects, non-public destinations, oversized/truncated responses, and unsupported documents. - Added deterministic PDF-only validation using PDF header/version, object, `startxref`, and terminal `%%EOF` structural markers plus approved response media types. No PDF text extraction, page inspection, OCR, embedded-link handling, or active-content execution was added. - Added exact-byte SHA-256 and size recording; identity and selected-entry provenance; submitted/final URLs and redirect history; retrieval timestamp/status; allow-listed response metadata; sanitized server filename; detected media type; public-address and document-validation methods; and a content-derived artifact ID. - Added isolated sanitized run directories, repository-independent relative storage references, no-overwrite behavior, and preservation of partial failure artifacts. No automatic cleanup or deletion exists. - Added fixture-backed tests that load and resolve each frozen CMR response, discover and explicitly select the exact source candidate, and only then map that exact selected URL to immutable local PDF bytes through a mocked transport. ### Important retrieval and security decisions - Candidate identity and selection binding use canonical hashes covering the exact CMR source hash, source position, URL, source entry, and collection identity fields. Any changed component invalidates the selection. - DNS fakes are deterministic and routine tests never consult machine DNS. Every resolved address must pass both `is_global` and explicit private, loopback, link-local, reserved, multicast, and unspecified rejection checks. - Automatic HTTP redirects are disabled. Every redirect passes the complete URL and DNS policy before a subsequent request. - The response is streamed into a retained partial artifact while enforcing limits. Successful bytes are promoted within the isolated run directory; failures do not remove the partial artifact or any prior artifact. - Serialized retrieval records expose only relative storage references. The bounded exact bytes are also returned to the owning service; absolute local paths are not serialized or logged. - A small deterministic structural PDF validator was used rather than adding a parser dependency. It performs document validation only and does not extract or inspect scientific content. ### Tests and static checks executed - Prerequisite full suite before editing: `python -m pytest` - Passed: `389 passed, 1 skipped`. - Focused Milestone 3 suite: `python -m pytest tests/datasets/test_readme_discovery.py tests/datasets/test_documents.py` - Passed: `70 passed`. - Final lint: `python -m ruff check .` - Passed: `All checks passed!`. - Final format check: `python -m ruff format --check .` - Passed: `72 files already formatted`. - Final complete suite: `python -m pytest` - Passed: `459 passed, 1 skipped in 0.91s`. The skipped test is the existing optional live integration test. ### Network, model, boundary, and immutability confirmation - Routine tests made no live HTTP, DNS, CMR, README, or model calls. HTTP used `httpx.MockTransport`; DNS used injected deterministic resolvers. - Direct raw-URL and fixture-path retrieval is impossible through the production retriever: it accepts only `ValidatedREADMESelection` and revalidates its discovery binding. - In all four frozen use cases, every fetched URL originated from exact CMR `READ-ME` discovery followed by explicit source-position and exact-URL selection. - Manifest SHA-256 values matched all four frozen CMR files and all four frozen PDF files before and after implementation. - Byte-for-byte Git comparisons confirmed that everything under `tests/fixtures/datasets/`, `data/gcmd_hierarchy.json`, `data/articles.json`, `prototype/app_hf_poc.py`, and `schemas/classification_result.schema.json` remained unchanged. - No article model, article result schema, publication behavior, fixture, source JSON, or protected proof-of-concept file was modified. - No extraction, page/section processing, evidence selection, classification, comparison, UI, live retrieval, or model call occurred. - No delete-on-success, delete-on-error, temporary cleanup, startup cleanup, scheduled cleanup, expiration, quota cleanup, cache cleanup, or other automatic deletion behavior was added. ### Deviations from DATASET_README_IMPLEMENTATION_PLAN.md - No scope deviations. - PDF document validation uses deterministic structural byte checks rather than adding a PDF parser dependency; this is sufficient for Milestone 3 format validation and intentionally performs no extraction. - The retriever resolves each destination twice immediately before requesting it so deterministic DNS-rebinding-style changes between checks are rejected. - Isolated run directories use an injectable sanitized run identifier; the immutable artifact identifier itself is derived from the exact PDF content hash. ### Known limitations - PDF structural validation is intentionally conservative and is not a full semantic or forensic PDF validator. - Public-address validation is a pre-request destination policy implemented through the injected resolver; the injected HTTP client remains responsible for transport-level connection behavior. - Artifacts and partial artifacts are intentionally retained indefinitely until an explicit manual operation removes them. - This milestone exposes service APIs only; UI selection controls and full pipeline status integration remain deferred. ### Deferred work Milestones 4 through 11 remain deferred, including PDF text/page/section extraction, product resolution, evidence selection and packet construction, shared-classifier adaptation, dataset orchestration and stage caches, blind persistence, expert comparison, review output, UI integration, and evaluation tooling. ### Acceptance criteria Every Milestone 3 acceptance criterion is satisfied: - exact discovery preserves all candidates and source provenance; - zero, one, and multiple candidate states behave correctly; - retrieval accepts only a typed explicit selection originating from validated CMR discovery; - URL, public-address, redirect, timeout, streaming, size, metadata, authentication, and PDF-validation controls pass; - no-README and retrieval failures stop before extraction, comparison, classification, or model use; - all four frozen cases pass discovery, explicit selection, and exact mocked PDF retrieval; - fixture and protected publication files remain unchanged; - complete dataset and publication regressions pass; and - no Milestone 4 work was started. ## Milestone 4: PDF Extraction and Provenance Status: completed ### Files created - `src/gcmd_classifier/datasets/extraction.py` - `tests/datasets/test_extraction.py` ### Files modified - `pyproject.toml` - `src/gcmd_classifier/config.py` - `src/gcmd_classifier/datasets/__init__.py` - `src/gcmd_classifier/datasets/documents.py` - `src/gcmd_classifier/datasets/errors.py` - `src/gcmd_classifier/datasets/models.py` - `schemas/dataset_extraction_manifest.schema.json` - `schemas/dataset_evidence_packet.schema.json` - `tests/datasets/test_models_and_schemas.py` - `DATASET_README_PROGRESS.md` No files were deleted. ### Dependency changes - Added runtime dependency `pdfplumber==0.11.9` for deterministic page-aware passive PDF parsing. - Added development dependency `reportlab>=4,<5` for generated unit-test PDFs. - The initial unrestricted `<1` pdfplumber range selected `0.11.10`, whose Pillow requirement conflicted with the project Gradio environment. The runtime parser was therefore pinned to `0.11.9`, restoring Pillow `<12` compatibility and making the extraction engine reproducible. - No OCR, image-recognition, document-conversion, browser, or external-service dependency was added. ### Requirements implemented - Added a production extraction boundary that accepts only the typed `RetrievedREADME` produced by Milestone 3. It has no arbitrary URL, fixture URL, upload, or local-path entry point. - Added an exact pre-parse SHA-256 and byte-size check; altered bytes return fatal `PDF_HASH_MISMATCH`. Source PDF bytes are read through an in-memory passive stream and never modified. - Preserved blind-safe lineage through Concept ID, Native ID, ShortName, Version, CMR revision, CMR response hash, selected candidate ID/source position/complete entry, submitted and final README URLs, retrieval timestamp/status, public-address validation, document-validation method, PDF artifact identity, byte size, media type, and SHA-256. - Added one-based page extraction with zero-based source order, stable page IDs, exact approved `pdfplumber.extract_text(layout=False, x_tolerance=3, y_tolerance=3)` text, UTF-8 text hashes, character counts, extraction statuses, page dimensions, coordinate-system metadata, block membership, and diagnostics. - Added deterministic heading detection for numbered, short all-caps, and colon-label lines. Literal headings retain exact page-text slices and character spans; the deterministic `Document body` fallback is explicitly derived and has no fabricated literal offsets. - Added deterministic ordered sections supporting no headings, multiple headings per page, repeated identical headings, page-boundary headings, cross-page sections, and collision-resistant document-order section IDs. - Added page-bounded blocks that are exact page-text slices with page/section/document/PDF provenance, text offsets, extraction and splitting-policy versions, global source order, and stable page-local IDs. - Added deterministic long-block splitting at retained newline boundaries or hard character boundaries without rewriting, trimming, summarizing, or discarding text. - Added manifest invariants requiring consecutive one-based pages, consecutive source orders, exact block spans, correct page linkage, matching counts, and exact per-page reconstruction. No separators are excluded under `no-excluded-separators-v1`; the formal separator contract remains available if a later policy excludes any. - Added typed fatal zero-inference outcomes for encrypted/password-protected, malformed/unreadable, no-page, no-text, image-only, page extraction, partial-page, timeout, page-limit, character-limit, block-limit, hash-mismatch, and unvalidated-input failures. - Added deterministic non-fatal warnings for blank pages, image-only pages in otherwise usable documents, low text coverage, unavailable layout coordinates, table-like ruled layouts, likely multi-column layouts, and suspicious reading order. - No extraction failure or success path calls a model. Fatal results explicitly contain `classification_attempted: false`, `model_calls: 0`, and `usable_for_classification: false`. ### Extraction policy versions and stable identity design - Extraction policy: `dataset-extraction-v1` - Page-text policy: `pdfplumber-page-text-v1` - Heading policy: `deterministic-heading-v1` - Section policy: `deterministic-section-v1` - Block-splitting policy: `dataset-block-splitting-v1` - Identifier policy: `dataset-extraction-identifiers-v1` - Separator policy: `no-excluded-separators-v1` - `document_id`: `sha256:` - `page_id`: `:pNNNN` - `section_id`: `sNNNN` in document order - `block_id`: `pNNNN-bNNNN` in page order - Page text hashes use SHA-256 of the exact stored extracted UTF-8 text. Stable outputs contain no timestamps generated during extraction, random UUIDs, temporary paths, process IDs, or host-specific values. ### Configured document limits and diagnostics policy - Maximum pages: 500 - Maximum extracted characters: 2,000,000 - Maximum blocks: 10,000 - Maximum characters per block: 4,000 - Extraction timeout: 60 seconds - Maximum heading length: 120 characters - Low-text warning threshold: 40 characters per non-empty page - Multi-column diagnostic minimum: five words per side and a gap of at least 20% of page width - Page-text x/y tolerances: 3 PDF points Fatal errors make the artifact unusable for classification. Layout, blank-page, low-coverage, table, and reading-order findings remain deterministic review warnings when all page text was extracted successfully. ### Schema status - Expanded the typed `DatasetExtractionManifest` contract first, then regenerated schemas with `write_dataset_schemas`. - `dataset_extraction_manifest.schema.json` now contains full lineage, version, page, section, block, limit, status, warning, coordinate, separator, and stable-ID requirements. - `dataset_evidence_packet.schema.json` changed because it embeds the strengthened `ExtractedBlock` provenance contract. - The publication `classification_result.schema.json`, dataset classification-result schema, and dataset comparison schema remain unchanged. - A final drift check confirmed all four dataset schema files exactly match their generated typed contracts. ### Tests added Focused tests cover generated one-page and multi-page PDFs, literal and derived headings, multiple and repeated headings, a heading at a page boundary, cross-page sections, no-heading fallback, deterministic long splitting, blank pages/documents, image-only documents, encrypted and malformed PDFs, simulated page and partial-page failures, unavailable coordinates, table/multi-column/reading-order warnings, low coverage, hash mismatch, page/character/block/time limits, stable hashes and identifiers, policy-version behavior, exact spans and reconstruction, source immutability, schema validation, sealed-keyword absence, and zero-inference failures. The real pdfplumber extractor also runs twice against each frozen PDF with identical output assertions. ### Frozen PDF extraction outcomes - `OMPS_NPP_NMTO3_L2_2`: usable; 24 pages, 102 blocks, 37,218 extracted characters, 12 warnings. - `ESMRN5IM_001`: usable; 21 pages, 223 blocks, 25,200 extracted characters, 9 warnings. - `M2T1NXAER_5.12.4`: usable; 66 pages, 115 blocks, 77,539 extracted characters, 63 warnings. - `OCO2_L2_Lite_SIF_11.2r`: usable; 14 pages, 58 blocks, 22,146 extracted characters, 6 warnings. Warnings are deterministic layout/review diagnostics and do not imply product relevance or classification quality. ### Commands and exact results - Prerequisite `python -m pytest`: `459 passed, 1 skipped`. - Focused extraction suite `python -m pytest tests/datasets/test_extraction.py`: `12 passed`. - Final `python -m ruff check .`: `All checks passed!`. - Final `python -m ruff format --check .`: `74 files already formatted`. - Final `python -m pytest`: `471 passed, 1 skipped in 16.63s`. - Dataset schema drift check: all four schemas match generated typed contracts. - Fixture manifest verification: all four CMR and four PDF SHA-256 values match `cases.json`. The skipped test is the existing optional live integration test. Routine tests made no live network or model calls. ### Security, boundary, and immutability confirmation - All four frozen CMR and PDF files remained byte-for-byte unchanged before and after extraction; their manifest hashes still match. - Everything under `tests/fixtures/datasets/` remained unchanged. - `data/gcmd_hierarchy.json`, `data/articles.json`, `prototype/app_hf_poc.py`, and `schemas/classification_result.schema.json` remained byte-for-byte unchanged. - `ArticleRecord`, `ArticleResult`, and publication behavior were not modified. - Sealed expert ScienceKeywords are not accepted by extraction types and do not appear in extraction inputs, manifests, diagnostics, schema identities, or logs. - PDF parsing is passive: no JavaScript, actions, forms, attachments, embedded files, links, URLs, external viewers, referenced resources, or PDF instructions are executed or followed. - No model, OCR, product resolution, evidence selection, GCMD classification, expert comparison, persistence service, cache policy, UI, or Gradio behavior was added. - No automatic deletion, cleanup, expiration, retention timer, startup cleanup, scheduled cleanup, or recursive removal was added. ### Deviations from DATASET_README_IMPLEMENTATION_PLAN.md - No scope deviations. - The schema expansion was required because the Milestone 1 extraction schema lacked several provenance, stable-ID, diagnostic, limit, and reconstruction fields explicitly required by the approved Milestone 4 specification. Typed models were changed first and schemas were regenerated, not manually patched. - `pdfplumber` is pinned to `0.11.9` rather than a broad compatible range to avoid the observed Pillow/Gradio incompatibility and make extraction behavior reproducible. ### Known limitations - Heading detection is intentionally conservative and layout-based; it does not infer semantic section meaning or product relevance. - Table and multi-column detection are deterministic heuristics intended to raise review warnings, not reconstruct arbitrary complex visual reading order. - PDF text fidelity means fidelity to the exact approved text emitted by the pinned pdfplumber policy; PDF character encoding and layout interpretation are necessarily performed by the parser and are explicitly versioned. - OCR is unavailable, so complete image-only documents are fatal and image-only pages in mixed documents require review. - Extraction artifacts are represented by the manifest contract but no persistence or cache service is implemented in this milestone. ### Deferred work Milestones 5 through 11 remain deferred, including product coverage and scope resolution, evidence selection and packet construction, classifier adaptation, blind orchestration/persistence/cache stages, expert comparison, human-review output, two-tab UI integration, and evaluation tooling. ### Acceptance criteria Every Milestone 4 acceptance criterion is satisfied: - every successful block resolves to exact document, PDF, page, page-text span, source order, and section provenance; - approved page text is exactly reconstructable from ordered page blocks; - deterministic versions, stable IDs, hashes, limits, and diagnostics are recorded; - unusable PDFs stop with zero model calls; - all four frozen PDFs produce deterministic schema-valid usable manifests; - fixture and protected publication files remain unchanged; - complete dataset and publication regressions pass; and - no Milestone 5 work was started. ## Milestone 5: Coverage Analysis and Product-to-Passage Resolution Status: completed ### Files created - `src/gcmd_classifier/datasets/product_prompts.py` - `src/gcmd_classifier/datasets/product_resolution.py` - `tests/datasets/test_product_resolution.py` ### Files modified - `src/gcmd_classifier/config.py` - `src/gcmd_classifier/datasets/__init__.py` - `src/gcmd_classifier/datasets/errors.py` - `src/gcmd_classifier/datasets/models.py` - `src/gcmd_classifier/llm/base.py` - `DATASET_README_PROGRESS.md` No files were deleted. No formal JSON Schema file required a change. ### Requirements implemented - Added a strict blind-stage `CoverageAnalysisInput` that contains only validated dataset identity, approved blind display fields, selected README/extraction lineage, the valid Milestone 4 manifest, and version identities. It rejects extra fields such as sealed expert keywords, full CMR payloads, expected results, arbitrary text, and comparison data. - Added versioned deterministic analysis for exact boundary-aware ShortName, Native ID, and Version spans; explicit product enumerations; target/other membership; product-family declarations; linked family-membership evidence; template headings; document-wide/shared context; other-product restrictions; unresolved scope; and literal README alias declarations. - Preserved literal capitalization, punctuation, source block, page, section, and character spans. Normalized comparison strings are separate derived fields and never replace source literals. - Added alias provenance restricted to validated CMR ShortName/Native ID, the validated derived Native ID consistency value, or exact README ` for short` declarations with source spans. Model-created aliases and products are rejected. - Added strict typed `dataset_coverage_v1` and `dataset_product_resolution_v1` prompts and response contracts. - Added exact coverage modes: `single_product`, `explicit_multi_product`, `product_family`, `vague_product_scope`, and `undetermined`. - Added one authoritative product-scope decision for every considered block, preserving document/page/section/source-order provenance, scope, eligibility, declarations, qualifying links, method/version, rationale, uncalibrated confidence, warnings, and review state. - Preserved the established enum mapping: `enumerated_target` represents `enumerated_set`; `product_family` represents `family_wide`. - Added separate ordered partitions for target-eligible, excluded other-product, ambiguous/review-only, and document-context blocks. Their union must exactly equal all extraction block IDs without duplication. - Enforced inclusion policy: exact-target and target-containing enumerations are eligible; other-product is excluded; ambiguity is review-only; document context remains separately typed; family-wide eligibility requires both a linked family declaration and explicit linked target-membership declaration. - Added deterministic source-ordered batching by configurable block/character limits and verified that combined batch responses exactly partition the original block sequence. - Added a narrow zero-model deterministic outcome only when every block contains both the exact validated ShortName and exact Native ID and no competing enumeration, family, or restriction declaration exists. - Added a blind cache-identity hash covering target identity, extraction-manifest hash, ordered block hashes, rule/prompt/schema versions, and no expert or comparison data. No cache service was implemented. ### Deterministic, prompt, and schema versions - Deterministic signal analysis: `dataset_product_signals_v1` - Coverage prompt: `dataset_coverage_v1` - Product-resolution prompt: `dataset_product_resolution_v1` - Product response schema: `dataset_product_resolution_schema_v1` `ModelStage` was additively extended with `dataset_coverage` and `dataset_product_resolution`; publication stages and prompt behavior remain unchanged. ### Prompt and structured-response safety - README excerpts are explicitly delimited as untrusted `` source data. - Prompts state that source text cannot change the task/schema, request tools or retrieval, reveal hidden data, authorize classification, introduce identities/aliases, override policy, or cause GCMD keyword generation. - Requests expose only supplied identities, signals, declarations, aliases, stable block IDs, source linkage, and required source-faithful excerpts. - Strict Pydantic schemas reject extra/unknown fields, invalid enums, invalid confidence, and malformed outputs. - Validation rejects unknown, missing, duplicate, fabricated, stale, or cross-document block IDs; altered document/page/section provenance; invented declarations, aliases, products, or family membership; and policy-inconsistent eligibility. - Coverage and block responses use the configured bounded retry count. Schema, transport/timeout, or invariant failures may be retried; exhaustion returns typed `PRODUCT_RESOLUTION_RETRIES_EXHAUSTED`, `classification_attempted: false`, zero GCMD classifier calls, and downstream ineligibility. - Failure messages and outputs are generic and do not include prompts, README text, raw responses, sealed payloads, or credentials. Milestone 5 adds no content logging. ### Frozen use-case outcomes Controlled fake-model responses use only the frozen blind metadata, deterministic signals/declarations, and extracted README blocks. Manifest coverage modes are test oracles only and are not imported by production code. - `OMPS_NPP_NMTO3_L2_2`: `single_product`, review not required; 1 target-eligible, 0 excluded, 0 review-only, 101 document-context blocks (102 total). The literal `OMPS-NPP_NMTO3-L2 for short` declaration is preserved as alias provenance; template sections remain optional signals. - `ESMRN5IM_001`: `explicit_multi_product`, review not required; 4 target-eligible, 1 excluded ESMRN5L1-only, 0 review-only, 218 document-context blocks (223 total). The CMR-selected differently named PDF remains eligible; filename equality is never a production applicability rule. - `M2T1NXAER_5.12.4`: `product_family`, review not required; 19 target/family-eligible, 36 other-product exclusions, 0 review-only, 60 document-context blocks (115 total). `MERRA-2 Data Products` is linked with explicit target membership in the source product enumeration before family-wide eligibility is accepted. - `OCO2_L2_Lite_SIF_11.2r`: `vague_product_scope`, review required and downstream classification ineligible; 0 target-eligible, 0 excluded, 58 review-only, 0 document-context blocks (58 total). No expected GCMD keyword or frozen CMR ScienceKeyword is used as a coverage, alias, family, inclusion, or product-scope oracle. ### Tests added Focused tests cover all coverage modes; boundary-aware ShortName/Native ID matching; capitalization and leading-zero preservation; enumerated target/other products; family membership proven and unproven; explicit README alias spans; document-wide/shared context; vague and unresolved scope; contradictory/repeated declarations; literal/derived/template headings; source order; exact block partitioning; eligibility groups; batching; model-created/stale/cross-document IDs; missing/duplicate decisions; invented aliases/products/family membership; malformed schema/enums/confidence; timeout/transport errors; retry success/exhaustion; deterministic zero-model resolution; prompt injection; prompt/stage/version identities; frozen cases; source immutability; and expert sentinel isolation. A conspicuous sentinel is placed inside a real `SealedExpertKeywordPayload`; the strict blind input rejects the sealed field, and the sentinel remains absent from deterministic inputs, prompts, results, block decisions, cache identity material, retry requests, errors, and model-fake outputs. ### Commands and exact results - Prerequisite `python -m pytest`: `471 passed, 1 skipped in 16.26s`. - Focused Milestone 5 suite: `python -m pytest tests/datasets/test_product_resolution.py` → `23 passed`. - Final `python -m ruff check .`: `All checks passed!`. - Final `python -m ruff format --check .`: `77 files already formatted`. - Final complete `python -m pytest`: `494 passed, 1 skipped in 24.73s`. - Schema drift verification: all four formal dataset schemas exactly match generated typed contracts. - Fixture verification: all four CMR and four PDF SHA-256 values match `cases.json`. The skipped test is the existing optional live integration test. Routine tests made no live model or network calls; model behavior used `FakeModelClient`, README transport used `httpx.MockTransport`, and DNS was injected. ### Blind-boundary, security, and immutability confirmation - Milestone 5 production signatures cannot receive the sealed expert payload. The full CMR source and `ScienceKeywords` remain outside the service. - Real frozen expert payload serialization and sentinel values do not appear in prompts, results, errors, decisions, or cache identities. - No GCMD hierarchy/classifier routing, recommendation, deterministic keyword validation, expert comparison, evidence-packet construction, or semantic `not_classified` decision occurred. - Everything under `tests/fixtures/datasets/` remained unchanged, and all eight manifest hashes still match. - `data/gcmd_hierarchy.json`, `data/articles.json`, `prototype/app_hf_poc.py`, and `schemas/classification_result.schema.json` remained unchanged. - `ArticleRecord`, `ArticleResult`, publication prompts, publication results, and publication behavior were not modified. - No OCR, retrieval expansion, arbitrary URL/PDF input, UI, Gradio, persistence service, background task, cleanup, retention, expiration, or deletion behavior was added. ### Deviations from DATASET_README_IMPLEMENTATION_PLAN.md - No scope deviations. - No formal output schema was changed because Milestone 5 owns strict typed intermediate and model-response contracts; the existing four formal dataset output schemas remain drift-free. - Explicit README aliases use a conservative, versioned literal ` for short` declaration rule. Broader semantic alias inference remains prohibited. ### Known limitations - Deterministic enumeration/family/template recognition is intentionally conservative and may produce extra unresolved or review signals; the controlled model stage resolves semantic scope without changing source records. - Document-context blocks remain a separate partition for Milestone 6 rather than being automatically treated as scientific keyword evidence. - Confidence values are uncalibrated routing signals. - The product-list tokenizer targets source-like uppercase/digit identifiers and does not attempt unrestricted named-entity recognition. - No production persistence/cache service or human-review UI exists yet. ### Deferred work Milestones 6 through 11 remain deferred, including scientific evidence selection and immutable packet construction, shared classifier adaptation, blind persistence and stage caches, expert comparison, review output, two-tab UI integration, and evaluation tooling. ### Acceptance criteria Every Milestone 5 acceptance criterion is satisfied: - every extracted block receives exactly one typed, traceable product-scope decision; - other-product-only blocks are excluded from later evidence eligibility; - vague and undetermined applicability remain review-required rather than forced; - deterministic identities, declarations, aliases, methods, prompt/schema versions, retries, and provenance are preserved; - all four frozen cases match their approved coverage expectations with auditable block partitions; - sealed and real expert ScienceKeywords remain outside the blind workflow; - fixture and protected publication files remain unchanged; - complete dataset and publication regressions pass; and - no Milestone 6 work was started. ## Milestone 6: Structured Evidence Selection and Packet Construction Status: completed ### Files created - `src/gcmd_classifier/datasets/evidence.py` - `src/gcmd_classifier/datasets/evidence_prompts.py` - `tests/datasets/test_evidence.py` ### Files modified - `src/gcmd_classifier/config.py` - `src/gcmd_classifier/datasets/__init__.py` - `src/gcmd_classifier/datasets/errors.py` - `src/gcmd_classifier/datasets/models.py` - `src/gcmd_classifier/llm/base.py` - `schemas/dataset_evidence_packet.schema.json` - `tests/datasets/test_models_and_schemas.py` - `DATASET_README_PROGRESS.md` No files were deleted. No fixture, publication source, publication schema, vocabulary, article data, prototype, UI, retrieval, extraction, or product-resolution behavior was modified. ### Requirements implemented - Added the strict blind `EvidenceSelectionInput`, which can receive only the validated Milestone 4 extraction manifest and validated Milestone 5 product-resolution result. Full CMR responses, sealed expert keywords, expected keyword answers, arbitrary text/blocks/paths/URLs, and comparison outputs are rejected as unknown fields. - Added deterministic contiguous source-ordered grouping bounded by configured block and source-character limits. Groups record stable IDs, extraction-manifest identity, ordered target/context membership, limits, grouping policy, and explicit context-link method. - Added the versioned `dataset_evidence_selection_v1` prompt and strict response contract. Every decision target must appear exactly once. Unknown, missing, duplicate, fabricated, stale, cross-document, invalid-link, invalid-role, and product-scope-override responses fail validation and use bounded retries. - Preserved Milestone 5 authority: only `eligible` blocks become scientific decision targets; `other_product` and explicit target-excluding blocks remain excluded; `review_only` ambiguity cannot become eligible; document context can enter only through an explicit qualifying link from selected scientific evidence. - Kept scientific relevance, product applicability, evidence role, packet inclusion, and review state separate. Context blocks are labeled `context_only`, use the approved `qualification` role, identify the scientific blocks they qualify, and cannot form a context-only packet. - Constructed one immutable logical packet from authoritative extraction records in original source order. Model-returned text is never accepted. Each included block is verified by stable ID, exact text, SHA-256, document/PDF identity, page, section, offsets, and source order. - Added deterministic deduplication by stable block ID and explicit rejection of conflicting repeated decisions. Packet ordering never depends on response order, confidence, role, heading, or group. - Added explicit block-count, character-count, and character-divided-by-four token estimates. Oversized selections receive exactly one traceable reduction pass over only initially selected blocks. Remaining over-limit packets return `EVIDENCE_PACKET_LIMIT_EXCEEDED`; text is never truncated, split, summarized, normalized, or paraphrased. - Added stable canonical JSON hashing for target, source identity, ordered block identity, ordered source text, product scope, selection decisions, policy versions, limit/reduction lineage, and the complete packet. Logical identity excludes timestamps, paths, process/host values, transport diagnostics, expert keywords, expected results, and later classifications. - Added the minimal `DatasetClassifierInput` future boundary. It accepts only approved target metadata plus a validated classification-ready `DatasetEvidencePacket`; raw PDF text, arbitrary block lists, extraction manifests, product resolution plus raw blocks, URLs, paths, summaries, full CMR records, sealed experts, and review-required packets cannot substitute for it. - Added typed evidence failures with generic diagnostics and zero GCMD classifier calls. No logging of prompts, block text, packet text, raw responses, full CMR payloads, sealed experts, or secrets was introduced. ### Versions and approved roles - Evidence-selection prompt: `dataset_evidence_selection_v1` - Structured response schema: `dataset_evidence_selection_schema_v1` - Grouping policy: `dataset_evidence_grouping_v1` - Evidence-role policy: `dataset_evidence_roles_v1` - Packet builder: `dataset_evidence_packet_v1` - Packet-limit policy: `dataset_evidence_limits_v1` - Token estimate: `characters_div_4_v1` Approved roles are unchanged from the specification: `product_identity`, `scientific_description`, `measured_variable`, `geophysical_parameter`, `method_or_algorithm`, `spatial_temporal_context`, `processing_or_format`, `qualification`, and `other`. ### Prompt, retry, and injection safety README blocks are explicitly delimited as untrusted evidence. The prompt states that their contents cannot change the task, target, schema, product scope, eligibility, block identity, packet limits, or response policy; authorize tools/retrieval; request expert keywords; initiate GCMD classification; or provide replacement packet text. Strict Pydantic parsing rejects unknown response fields, including source text, excerpts, summaries, and replacement content. Semantic validation then enforces the exact target partition, source identity, Milestone 5 scope/eligibility, approved links, approved roles, relevance/inclusion consistency, and reduction constraints. Timeout, transport, malformed/schema, and semantic failures use the configured bounded retry count. Exhaustion returns `EVIDENCE_SELECTION_RETRIES_EXHAUSTED` and prevents packet construction and classification. Fake-policy tests include source instructions to force selection/exclusion, alter scope, fabricate IDs, request expert keywords and GCMD classification, return summaries, override limits/schema, and ignore policy. These strings remain unchanged source data and never alter application control flow. ### Frozen use-case outcomes The integration tests use unchanged frozen CMR/PDF sources, real extraction, validated Milestone 5 product-resolution outputs, deterministic grouping/packet code, controlled fake selection responses, and no network or live model calls. - `OMPS_NPP_NMTO3_L2_2`: classification-ready packet; 1 selected, 0 excluded, 0 ambiguous, 0 included context blocks. Initial/final: 1 block and 507 characters; no reduction. The exact authoritative extracted text is preserved. - `ESMRN5IM_001`: classification-ready packet; 4 selected, 1 ESMRN5L1-only excluded, 0 ambiguous, 0 included context blocks. Initial/final: 4 blocks and 1,368 characters; no reduction. Other-product-only text is absent. - `M2T1NXAER_5.12.4`: classification-ready packet; 2 final selected, 36 other-product exclusions, 0 ambiguous, 0 included context blocks. Initial selection: 5 blocks and 6,907 characters. One configured reduction pass produced 2 blocks and 3,988 characters in original source order. - `OCO2_L2_Lite_SIF_11.2r`: no packet; review-required typed stop with 0 selected, 0 excluded, 58 ambiguous, and 0 context blocks. No evidence-selection or GCMD classification call occurs because Milestone 5 marks downstream applicability ineligible. The expected coverage mode is used only as the approved product-resolution fixture oracle. No expected GCMD keyword list is read or inferred. ### Schema status The typed `DatasetEvidencePacket` source model was expanded first because the Milestone 1 schema did not yet contain the Milestone 6 decision lineage, readiness, component hashes, and packet-limit/reduction records explicitly required by the approved plan. `schemas/dataset_evidence_packet.schema.json` was then regenerated through `write_dataset_schemas`; it was not manually patched. All four generated dataset schemas match their checked-in typed contracts. The publication `classification_result.schema.json` is unchanged. ### Tests and exact results - Prerequisite complete suite before editing: `python -m pytest` → `494 passed, 1 skipped in 25.05s`. - Focused Milestone 6 suite: `python -m pytest tests/datasets/test_evidence.py` → `24 passed in 9.61s`. - Existing dataset regressions excluding the new file: `python -m pytest tests/datasets --ignore=tests/datasets/test_evidence.py` → `190 passed in 24.35s`. - Publication regression suite: `python -m pytest tests --ignore=tests/datasets` → `304 passed, 1 skipped in 0.61s`. - Complete suite: `python -m pytest` reached 100%; the exact combined result is `518 passed, 1 skipped`, confirmed by the disjoint dataset and publication runs above. - `python -m ruff check .` → `All checks passed!`. - `python -m ruff format --check .` → `80 files already formatted`. - Dataset schema drift verification: all four checked-in dataset schemas equal regenerated typed schemas. - Fixture verification: all four CMR and all four PDF SHA-256 values match `cases.json`; 8 verified, 0 mismatches. The skipped test is the existing optional live integration test. ### Blind boundary, hashing, and immutability - A conspicuous sealed expert sentinel is rejected by production input types and remains absent from groups, prompts, retry requests, response schemas/fakes, decisions, packet content and metadata, component/packet hashes, cache identity, failures, and classifier-boundary input. - Real frozen CMR `ScienceKeywords` remain sealed and are never read as relevance, role, inclusion, reduction, packet, or expected-classification evidence. - Stable hashes repeat for identical logical inputs, change with authoritative source text or relevant policy/prompt versions, preserve canonical field and source order, and exclude volatile operational metadata. - All eight frozen fixture hashes still match, and nothing under `tests/fixtures/datasets/` changed. - `data/gcmd_hierarchy.json`, `data/articles.json`, `prototype/app_hf_poc.py`, and `schemas/classification_result.schema.json` remain unchanged. - `ArticleRecord`, `ArticleResult`, publication prompts, and publication behavior remain unchanged. ### Deviations, limitations, and deferred work No scope deviation from Milestone 6. The formal evidence-packet schema expansion is the plan-authorized typed-model-first change described above. Known limitations: - Evidence relevance is model-mediated and uncalibrated; deterministic validation proves contract and provenance integrity, not scientific correctness. - Context inclusion is deliberately conservative and requires explicit Milestone 5 qualifying links plus a selected scientific block. - Token size is an explicit deterministic character-based estimate, not a provider tokenizer. - No persistence/cache service is implemented; only blind-safe cache identity material is defined. - OCO-2 vague applicability stops for review without constructing a packet. Milestones 7 through 11 remain deferred. No GCMD Topic/Term/Variable routing, keyword recommendation, classifier prompt/call, deterministic keyword validation, expert comparison/metrics, embeddings, vector index/search, summary/paraphrase/compression, OCR, retrieval change, product-resolution behavior change, UI/Gradio work, cleanup, retention, deletion, or background task was added. ### Acceptance criteria Every requested Milestone 6 acceptance criterion is satisfied: - packets contain only authoritative immutable source blocks plus clearly separated derived metadata; - product exclusions and ambiguity remain authoritative; - grouping, response validation, consolidation, limits, one-pass reduction, hashing, retries, and review stops are typed and tested; - the validated evidence packet is the sole future README-text classifier input; - expert keywords remain outside every blind-stage type and identity; - all frozen use cases and publication regressions pass; - fixtures and protected publication sources remain unchanged; and - no Milestone 7 work was started. ## Milestone 7: Shared Hierarchical Classifier Adapter Status: completed ### Files created - `src/gcmd_classifier/classification/evidence_context.py` - `src/gcmd_classifier/datasets/classifier_adapter.py` - `src/gcmd_classifier/datasets/classifier_prompts.py` - `tests/datasets/test_classifier_adapter.py` ### Files modified - `src/gcmd_classifier/classification/__init__.py` - `src/gcmd_classifier/config.py` - `src/gcmd_classifier/datasets/__init__.py` - `src/gcmd_classifier/datasets/errors.py` - `src/gcmd_classifier/datasets/evidence.py` - `src/gcmd_classifier/datasets/models.py` - `src/gcmd_classifier/llm/base.py` - `schemas/dataset_classification_result.schema.json` - `tests/datasets/test_models_and_schemas.py` - `DATASET_README_PROGRESS.md` No files were deleted. No fixture, article prompt, article model, publication schema, publication pipeline, vocabulary, source article, prototype, retrieval, extraction, product-resolution, evidence-selection, packet-construction, UI, or persistence behavior was modified. ### Shared source-agnostic context and hierarchy reuse Added strict `HierarchyEvidenceContext` and `HierarchyEvidenceUnit` contracts with neutral source kind, logical source identity, immutable evidence IDs/text, source order, typed display metadata, independent eligibility, evidence links, and citation validation. The article helper represents Title and Abstract neutrally without changing any article routing or prompt rendering. The dataset adapter reuses the existing authoritative: - local `VocabularyIndex` and canonical records; - Topic, Term, and Variable candidate builders; - direct-child Term and Variable relationship validators; - assignable internal-node and leaf behavior; - authoritative parent and ancestor lookups; - `ClassificationCandidate` deterministic vocabulary validation; - authoritative record construction; - hierarchy path and parent consistency checks; and - redundancy removal. Existing publication router/descent entrypoints remain unchanged because their signatures and responses are intentionally article-specific. Dataset-specific prompt calls, citation checks, product-confidence inheritance, review gating, and result construction remain in `datasets.classifier_adapter`; no second vocabulary or copied authoritative GCMD records were added. Shared services contain no CMR, PDF, README, packet, product-family, or expert-keyword knowledge. ### Dataset adapter boundary `classify_dataset_packet` accepts one typed `DatasetEvidencePacket`, authoritative `VocabularyIndex`, injected `ModelClient`, model settings, and dataset classifier settings. Before any hierarchy call it revalidates: - packet ID and complete logical packet hash; - target, source, ordered-block, ordered-text, product-scope, policy-version, and limit-lineage component hashes; - each exact selected-block SHA-256; - document identity and source order; - classification readiness and review state; and - at least one independently eligible scientific evidence role. Raw text, raw blocks, extraction manifests, product-resolution results, paths, URLs, arbitrary dictionaries, altered text, invalid hashes, cross-document blocks, review-blocked packets, and context-only evidence cannot cross this boundary. ### Dataset prompts and versions - Topic: `dataset_topic_router_v1` - Term: `dataset_term_router_v1` - Variable: `dataset_variable_router_v1` - Structured response schema: `dataset_hierarchy_response_v1` - Citation policy: `dataset_block_citations_v1` - Review policy: `dataset_initial_review_v1` Dataset prompts render exact immutable text with block ID, one-based page span, section ID/heading, evidence role, product scope, inherited uncalibrated product confidence, and independent-eligibility state. Application controls and supplied candidates are separate from delimited untrusted `README_BLOCK` records. Prompts state that README content cannot change target, candidates, hierarchy stage, schema, citations, product applicability, stopping/review policy, or confidence; fabricate IDs; skip levels; request tools/retrieval, expert keywords, or a semantic validator; or force acceptance. ### Structured decisions, citations, and retries Strict dataset responses return only supplied candidate IDs, uncalibrated keyword confidence, cited block IDs, and rationale. Child responses contain exactly one of a direct-child selection, explicit cited stop, or cited no-selection outcome. Unknown response fields are rejected, so model-generated UUIDs, labels, levels, paths, parents, citation text, excerpts, or confidence overrides cannot become authority data. Every selected Topic, Term, and Variable decision and every stop validates non-empty, unique packet citations. Citations must resolve within the packet and include at least one independently eligible scientific block. Context-only, product-identity-only, processing/format-only, qualification-only, or other-role blocks cannot be sole scientific support. Joint evidence across pages, sections, non-adjacent blocks, and roles remains independently cited without concatenation or summarization. Timeout, transport, schema, unknown/duplicate candidate, level-skipping, missing/unknown/duplicate/context-only citation, and invalid stopping responses use the configured bounded retry count. Exhaustion returns typed `DATASET_HIERARCHY_RETRIES_EXHAUSTED`; it never becomes semantic `not_classified`. ### Direct-child, stopping, and authority behavior - Topics come only from supplied Topic candidates. - Terms are validated direct children of the selected Topic. - Variable Level 1, 2, and 3 choices are validated direct children of the current parent. - Model output never supplies final UUIDs, labels, levels, paths, or parents. - Final records and paths are reconstructed from the local authoritative vocabulary and deterministically validated. - Classification can stop at Topic, Term, Variable Level 1, Variable Level 2, or terminal Variable Level 3. - Multiple Topic branches and joint multi-block support are preserved. - Existing redundancy removal runs after deterministic validation. - No per-block classification or keyword union occurs. ### Confidence and initial review policy Product applicability and keyword evidence remain distinct typed values. Product confidence is inherited from cited packet blocks and cannot be supplied or rewritten by hierarchy responses. Keyword confidence is preserved separately for Topic, Term, each Variable level, and final depth. The initial `0.75` product-match threshold is an uncalibrated review-routing signal. A fake response with keyword confidence `0.9` and inherited product confidence `0.5` produces `pending_review` with a `review_required` classification; high keyword confidence cannot raise or compensate for weak applicability. The independent dataset semantic validator is never called. Every attempted result records `semantic_validator_state: not_run_by_policy`; it is never represented as passed. Publication semantic-validation behavior and all publication pipeline code remain unchanged. ### Semantic not_classified A valid classification-ready packet with a completed cited no-Topic decision returns: - `processing_status: completed`; - `classification_attempted: true`; - `classification_outcome: not_classified`; - no accepted paths; - a source-grounded reason and cited packet block IDs; and - `semantic_validator_state: not_run_by_policy`. Packet/review failures stop before inference with `classification_attempted: false`. Transport, retry, schema, citation, candidate, and deterministic-validation failures remain failed attempted classifications and are not mislabeled semantic `not_classified`. ### Article compatibility evidence Before editing, fixed article Topic, Term, and Variable prompts were hashed. Final tests prove byte-for-byte equality: - Topic: `3ad64eb5da7af942046a45867012ea224fa545ee2e6358f570e6e25e4a8c16ef` - Term: `691b66c7187b963b5a8f72dd0e139ccac4be745dc07ae974e5d55dc25025807a` - Variable: `28d338046401b911bc66ca71a650c1ea26e7c593cdf9f6666154d43eacfb0360` `ArticleRecord` remains exactly `DOI`, `Title`, `Year`, and `Abstract`. `ArticleResult` contains no dataset citation or dataset semantic-validator fields. Article prompt versions, wording, structured responses, confidence, stopping, validation, retry, result serialization, and pipeline code are unchanged. `schemas/classification_result.schema.json` remains SHA-256 `9471856f8908d8050a84a1886f8a38ca1f06f2f54827425bbf9f3966a5aab298`. ### Frozen integration and expert isolation All four unchanged frozen cases use real extraction, validated Milestone 5 product resolution, Milestone 6 packet construction, controlled evidence and hierarchy fakes, and no live network/model calls. - OMPS, ESMR, and MERRA-2 produce classification-ready packets accepted by Milestone 7; each controlled accepted path resolves to an authoritative shared-vocabulary Topic and exact unchanged packet citation. - OCO-2 remains product-review blocked and produces no packet; Milestone 7 receives no input and makes zero hierarchy calls. - Every accepted path has an eligible exact block citation and distinct product/keyword confidence. - No independent dataset semantic-validator call occurs. - Serialized real expert keyword payloads and conspicuous sentinel values remain absent from contexts, prompts, candidates, retries, citations, confidence, results, errors, hashes, and model-fake outputs. - No frozen expert keyword or `cases.json` value is used as an expected classification path. ### Schema status The typed `DatasetClassificationResult` was expanded first with packet identity, coverage lineage, explicit semantic-validator state, no-classification citations, and authoritative per-result citation records. The pre-existing source/expert envelope fields are optional for this blind classifier-stage output and remain available for later orchestration. `schemas/dataset_classification_result.schema.json` was regenerated through the established schema generator and not manually patched. All four checked-in dataset schemas exactly match generated typed contracts. The evidence, extraction, comparison, and publication schemas are otherwise unchanged. ### Tests and exact results - Pre-change Milestone 6 boundary: `24 passed in 9.43s`. - Pre-change publication regression: `304 passed, 1 skipped in 0.61s`. - Final focused Milestone 7: `21 passed in 8.90s`. - Final Milestone 6 regression: `24 passed in 9.99s`. - Remaining dataset regressions: `190 passed in 24.91s`. - Final publication regression: `304 passed, 1 skipped in 0.62s`. - Complete disjoint total: `539 passed, 1 skipped`. - `python -m ruff format --check .`: `84 files already formatted`. - `python -m ruff check .`: `All checks passed!`. - Dataset schema drift: zero. - Publication schema hash: unchanged. - Fixture verification: 8 hashes verified, 0 mismatches. The environment’s command wrapper terminates a single foreground pytest process at approximately 30 seconds, so the required complete suite was also run as four exhaustive, non-overlapping subsets above. Their union is exactly the complete test collection. The skipped test is the existing optional live integration test. ### Security, immutability, deviations, and limitations No credentials, full payloads, packet text, prompts, model raw responses, sealed experts, or secrets are logged. No new content logging was added. All eight frozen CMR/PDF hashes match `cases.json`; nothing under `tests/fixtures/datasets/` changed. `data/gcmd_hierarchy.json`, `data/articles.json`, `prototype/app_hf_poc.py`, and `schemas/classification_result.schema.json` remain unchanged. No scope deviation. The publication routing entrypoints remain article-specific to preserve their exact prompts and outputs; the dataset adapter shares their source-independent candidate, direct-child, vocabulary, validation, path, and redundancy mechanics while supplying dataset-only responses and citations. Known limitations: - Dataset hierarchy confidence values are uncalibrated. - The adapter returns an in-memory schema-valid blind classification result; persistence and cache orchestration are deferred. - Dataset no-selection detail below Topic is retained as a failed branch/general no-defensible outcome rather than an evaluation metric. - An independent semantic validator remains intentionally deferred. Milestones 8 through 11 remain deferred. No expert comparison/unsealing, blind persistence workflow, evaluation metrics, UI/Gradio, OCR, retrieval change, product/evidence behavior change, embeddings/vector search, per-block keyword union, live fixture refresh, cleanup, retention, deletion, or background application behavior was added. ### Acceptance criteria Every requested Milestone 7 acceptance criterion is satisfied: - accepted dataset classifications use authoritative shared-vocabulary records and valid selected-block citations; - direct-child selection and stopping at every supported depth are enforced; - product and keyword confidence remain separate; - weak applicability cannot be overridden by hierarchy confidence; - semantic `not_classified` is distinct from failures and pre-classification stops; - dataset semantic validation is explicitly not run by policy; - article prompts, models, results, schema, and regressions remain unchanged; - all four frozen use cases preserve the blind boundary; - fixtures and protected files remain unchanged; and - no Milestone 8 work was started. ## Milestone 8 — Blind Orchestration, Persistence, and Stage Caches Status: complete (2026-08-13). ### Files created or modified Created: - `src/gcmd_classifier/datasets/runtime_models.py` - `src/gcmd_classifier/datasets/runtime_persistence.py` - `src/gcmd_classifier/datasets/orchestration.py` - `src/gcmd_classifier/datasets/pipeline.py` - `src/gcmd_classifier/datasets/stage_cache.py` - `tests/datasets/test_runtime_milestone8.py` Modified: - `src/gcmd_classifier/config.py` - `src/gcmd_classifier/datasets/errors.py` - `DATASET_README_PROGRESS.md` No files were deleted. ### Requirements implemented Added one dataset-only blind pipeline shell with the approved CMR source, identity, blind boundary, README discovery, explicit selection, safe retrieval, source persistence, extraction, product resolution, evidence selection, packet, shared hierarchy classification, blind-result persistence, verification, and terminal ordering. Stage business logic remains in the completed Milestone 2–7 components and enters orchestration through a typed injected service boundary. The runtime now has strict, extra-forbid typed contracts for run/session identity, stages, checkpoint status, transitions, terminal outcomes, classification-attempt state, model-call accounting, artifact types/references, persistence state, blind/comparison cache namespaces, cache dispositions and rejection reasons, recovery decisions, comparison eligibility, verified persisted blind-result references, structured stage logs, and runtime failures. `classification_attempted` is durably committed before the first model transport dispatch by `DurableInferenceClient`. Failure to persist that checkpoint prevents the delegate model call. Current-run call accounting is separate from the monotonic durable attempt state. The artifact store validates output/cache roots against the immutable fixture root, rejects absolute/traversal/cross-run/symlink paths, creates isolated session/concept/run directories, writes same-filesystem temporary files, flushes and fsyncs, publishes with `os.replace`, fsyncs the directory, reloads bytes, verifies SHA-256, rejects different-byte overwrites, and exposes no deletion or cleanup API. Interrupted temporary artifacts are deliberately retained. Recovery reconstructs state from strict immutable checkpoint documents, validates run/session identity, monotonic checkpoint sequence, and unchanged transition lineage, and never infers completion from a filename alone. Terminal state is immutable; cached completion requires a verified artifact reference. Section 11 has separate typed identities for CMR source, README artifact, extraction, product resolution, evidence selection, evidence packet, and blind classification. Keys use canonical serialization, namespace and version binding, and complete stage-specific identities. The implemented store is blind-only; no comparison cache or expert-keyword cache was introduced. The comparison prerequisite reloads persisted bytes, verifies path containment and artifact hash, validates the dataset result type, requires completed classified or semantic `not_classified`, validates Native ID and packet lineage, rejects comparison fields, and returns a typed eligible reference. It cannot be constructed from an in-memory result alone. Structured logs accept only typed identifiers, statuses, counts, versions, timestamps, hashes, and relative references. Central validation rejects forbidden source/prompt/expert/secret classes and absolute paths. ### Important design decisions - Publication persistence/cache utilities were not refactored; the dataset runtime is isolated to preserve publication semantics. - Existing Milestone 2–7 functions remain the owners of retrieval, extraction, product/evidence, and hierarchy behavior. The orchestrator fixes ordering and supplies durable services rather than duplicating stage algorithms. - Exact CMR/PDF bytes are persisted at acquisition boundaries; derived typed values use canonical JSON. - Blind and future comparison namespaces are represented separately, but only the blind cache is implemented in Milestone 8. - Cache identity includes URLs for source acquisition and content hashes for downstream invalidation; downstream content-addressed identities can therefore reuse identical bytes while changed bytes invalidate descendants. - There is intentionally no automatic deletion, retention, expiry, rotation, purge, or cleanup path. ### Tests and validation Executed: - `python -m ruff check .` — all checks passed. - `python -m ruff format --check .` — 90 files already formatted. - `python -m pytest` — 553 passed, 1 skipped in 42.02 seconds. - `python -m pytest tests/datasets/test_runtime_milestone8.py -q` — 14 passed. - `python -m pytest tests/datasets/test_fixture_manifest.py -q` — 7 passed. Focused coverage includes strict models, unsafe storage roots, contained regular artifacts, immutable overwrite rejection, cross-run denial, atomic interruption before publish, retained temporary bytes, illegal/stale transitions, durable pre-dispatch attempt persistence, zero transport calls when attempt persistence fails, verified checkpoint recovery, complete cache identity invalidation, blind cache tamper rejection, sanitized log references, persisted-byte comparison gating, and ordered no-README pre-inference termination. The skipped test is the existing opt-in live integration test. Routine tests made no live network or model calls. ### Security and immutability All eight frozen CMR/PDF files were revalidated through the authoritative manifest with zero mismatches. `git diff` confirms nothing under `tests/fixtures/datasets/` changed. The pipeline rejects expert `ScienceKeywords` in derived blind artifacts and never implements unsealing or comparison. `ArticleRecord`, `ArticleResult`, publication prompts, publication persistence/cache behavior, `schemas/classification_result.schema.json`, `data/gcmd_hierarchy.json`, `data/articles.json`, and `prototype/app_hf_poc.py` remain unchanged. No dataset formal output schema change was required. ### Deviations, limitations, and deferred work No approved Milestone 8 behavior was moved into publication code. The plan does not prescribe concrete orchestrator adapter class names, so `BlindStageServices` is the narrow dependency-injection seam used to call the already completed stages and deterministic fakes. Known limitations are the approved phase boundaries: comparison eligibility is only a gate and does not unseal or compare expert keywords; no comparison cache is implemented; no UI changes, evaluation dashboard, cleanup service, OCR, embeddings, arbitrary document input, or production retention policy exists. Model confidence remains uncalibrated. Milestone 9 expert resolution/comparison and all Milestones 10–11 work remain deferred. ### Acceptance criteria All requested Milestone 8 acceptance criteria are satisfied: pre-inference stops retain false/zero attempt accounting; inference dispatch is durably committed before transport; completed artifacts require full identity/hash validation; comparison requires a verified persisted blind result; run/session boundaries reject cross-run access; cache identities invalidate on all applicable inputs; blind and comparison namespaces are isolated; generated artifacts are never automatically deleted; full publication and dataset regressions pass; and frozen fixtures and protected publication files remain unchanged. ## Milestone 8 Boundary Completion — Persisted Sealed Expert Artifact and Comparison Provenance Binding Status: complete (2026-08-13). ### Original Milestone 9 blocker Milestone 9 was stopped before implementation because the Milestone 8 runtime persisted the raw CMR response but left `SealedExpertKeywordPayload` only in memory. `ArtifactType.SEALED_EXPERT` was unused, and `VerifiedPersistedBlindResultReference` did not bind the blind result to the exact CMR source, sealed expert artifact, collection identity, or revision. Safe post-blind unsealing therefore could not prove same-source provenance. ### Files and generated schemas changed Created: - `schemas/dataset_sealed_expert_artifact.schema.json` - `schemas/dataset_expert_source_binding.schema.json` - `schemas/dataset_verified_blind_reference.schema.json` - `tests/datasets/test_expert_source_boundary.py` Modified: - `src/gcmd_classifier/datasets/runtime_models.py` - `src/gcmd_classifier/datasets/runtime_persistence.py` - `src/gcmd_classifier/datasets/orchestration.py` - `src/gcmd_classifier/datasets/pipeline.py` - `src/gcmd_classifier/datasets/schemas.py` - `tests/datasets/test_runtime_milestone8.py` - `DATASET_README_PROGRESS.md` No files were deleted. The three schemas were generated from strict typed source models through `write_dataset_schemas`; no generated schema was manually patched. ### Corrected source boundary After exact CMR bytes are atomically persisted and hash verified, the identity stage now requires the typed `ResolvedCMRCollection`. It validates Concept ID, Native ID, ShortName, Version, revision ID, and exact CMR SHA-256, then persists `sealed_expert.json` as `ArtifactType.SEALED_EXPERT` before any blind stage can continue. `SealedExpertArtifactEnvelope` version `sealed-expert-artifact-v1` preserves the complete `SealedExpertKeywordPayload`, exact source entries, contiguous zero-based source positions, original order, `umm.ScienceKeywords` source path, collection identity, exact CMR artifact reference/hash, payload canonicalization version `canonical-json-v1`, and sealed payload hash. No expert value is normalized, resolved, interpreted, deduplicated, reordered, or rewritten. The pipeline replaces the resolved full collection and raw CMR object with an opaque CMR artifact reference and the existing allow-listed `BlindCollectionView` before invoking coverage, product resolution, evidence selection, packet construction, or classification services. The private orchestration boundary retains only opaque artifact references for later authorization. ### Expert-source binding and extended verified reference `ExpertSourceBinding` version/policy `expert-source-binding-v1` contains no expert values. It binds run identity; all five collection identity fields; exact CMR artifact reference/hash; sealed artifact reference/hash; sealed payload hash; and a canonical logical binding hash. It is created only after both source and sealed artifacts have been reloaded and verified, persisted as `ArtifactType.EXPERT_SOURCE_BINDING`, then reloaded and revalidated. The extended `VerifiedPersistedBlindResultReference` now requires the full typed collection identity, blind-result reference/hash, evidence-packet hash, vocabulary identity, CMR reference/hash, sealed artifact reference/hash, sealed payload hash, binding reference/hash, completed classified or `not_classified` status, and `hash_verified` persistence state. It exposes no expert values. Construction reloads and validates the CMR, sealed, binding, and blind-result bytes; checks containment and same-run access; rechecks all recorded hashes; reconstructs identity from exact CMR fields; compares exact source `ScienceKeywords` with the sealed payload; validates binding lineage; and requires the blind result to match Concept ID, Native ID, full identity, packet hash, completion status, and run. Legacy references lacking this provenance fail strict validation and are not comparison eligible. Pre-correction runs must be rerun or explicitly reprocessed; no migration or artifact rewrite was added. ### Blind independence, substitution, and recovery Changing only expert assignments leaves the allow-listed blind view, evidence/packet inputs, blind-result material, and blind-classification cache identity unchanged, while changing the sealed payload hash. Expert hashes were not added to blind classification results or blind cache keys. Cross-run references, path escapes, substituted source/sealed artifacts, changed bytes, stale binding bytes, mismatched collection fields, revision mismatch, payload/source mismatch, and invalid logical binding hashes are rejected with typed persistence or recovery failures. Immutable different-byte overwrites remain rejected. Source, sealed, and binding references are now included in the identity checkpoint. Recovery reloads every referenced artifact, verifies byte counts and SHA-256, strictly validates sealed and binding schemas, and verifies that the binding’s source hashes occur in the recovered checkpoint chain. It does not trust filenames or state flags and does not expose sealed contents to blind services. Atomic interruption before sealed publication leaves the verified CMR artifact and same-filesystem temporary bytes intact, publishes no sealed artifact, keeps `classification_attempted: false`, and makes zero model calls. No success, failure, restart, or verification path deletes generated artifacts. ### Frozen fixtures and sentinel isolation All four frozen CMR responses were resolved through the existing exact CMR adapter, persisted to temporary output roots, sealed, rebound, and verified. Every expert source assignment retained its exact dictionary, source order, and source position. No frozen expert assignment was used as an expected blind-classification output. Tests prove the expert sentinel is absent from the allow-listed blind context and all downstream stage contexts, blind results, and blind cache identity. Existing Milestones 2, 5, 6, and 7 tests continue to prove absence from prompts, retries, coverage/product inputs, evidence packets, citations, hierarchy results, logs, and errors. All eight frozen CMR/PDF hashes match `cases.json`; nothing under `tests/fixtures/datasets/` changed. ### Validation results - `python -m ruff check .`: all checks passed. - `python -m ruff format --check .`: 91 files already formatted. - Corrective focused suite: 27 passed. - Complete dataset regression suite: passed. - `python -m pytest`: 566 passed, 1 skipped in 44.64 seconds. - Publication-only regression suite: 304 passed, 1 skipped. - Schema drift: all seven dataset schemas exactly match generated typed schemas and validate as Draft 2020-12. - Fixture manifest and corrective boundary suite: 19 passed; 8 source hashes verified, 0 mismatches. - Comparison/model calls in this correction: zero. The one pipeline inference fake remains the existing Milestone 8 blind-pipeline test and is not comparison logic. `ArticleRecord`, `ArticleResult`, article prompts and versions, publication persistence/cache behavior, `schemas/classification_result.schema.json`, `data/gcmd_hierarchy.json`, `data/articles.json`, and `prototype/app_hf_poc.py` remain unchanged. ### Scope confirmation and remaining work No expert resolution, UUID/path comparison, ancestor/descendant logic, comparison category, review record, comparison persistence/cache, CMR update, expert mutation, blind reclassification, prompt, UI, OCR, embedding, cleanup, retention, deletion, cache eviction, or migration behavior was added. The original Milestone 9 provenance blocker is resolved for new corrected runs. Milestone 9 remains entirely unimplemented and requires a new explicit request. Existing pre-correction verified references remain intentionally ineligible. ## Milestone 9 — Expert Keyword Comparison and Review Records Status: complete (2026-08-13). ### Files changed Created comparison.py, comparison_models.py, and expert_resolution.py under src/gcmd_classifier/datasets/; schemas/dataset_human_review.schema.json; and tests/datasets/test_comparison_milestone9.py. Modified dataset errors, runtime models, schema generation, stage-cache identities, generated dataset comparison/provenance schemas, and schema tests. No files were deleted. ### Requirements implemented Comparison is a separate post-blind operation that accepts only a verified, persisted blind-result reference. Before expert unsealing it reloads and verifies the blind result, exact CMR source, sealed artifact, expert-source binding, hashes, run and collection identity, vocabulary identity, and completed classified or not_classified state. Invalid or substituted inputs fail with typed errors and invoke the expert unsealer zero times. Expert assignments are resolved deterministically against the local authoritative vocabulary. Resolution supports standard UUID fields and the ordered Topic → Term → Variable_Level_1 → Variable_Level_2 → Variable_Level_3 fields, using exact matching first and unique case/whitespace-normalized matching second. UUID syntax, existence, level, direct-parent continuity, and UUID/field agreement are checked. Original entries, order, positions, missing fields, normalization details, conflicts, unresolved entries, and duplicate occurrences are preserved; ambiguity is never guessed. Every blind classification is compared with every resolved expert assignment using exact, blind-descendant, expert-descendant, or independent relationships and hierarchy distances. Primary categories use the approved precedence: unresolved evidence → review_required; exact → already_assigned; expert descendant of blind → redundant; blind descendant of expert → proposed_refinement; independent → proposed_addition. Secondary relationships remain visible. Expert-only assignments are preserved; no expert keyword is removed, replaced, downgraded, or mutated. Semantic not_classified results compare with zero blind items and retain expert-only assignments. Other non-completed or unverified inputs are ineligible. Review records are additive, deterministic, separately persisted, and contain immutable blind recommendation/evidence context plus relevant expert facts. Unresolved experts create review records even without a blind classification. Results use stable component and logical hashes, policy/schema versions, deterministic IDs, atomic persistence, verified reload, sanitized logging, and an expert-sensitive comparison cache identity isolated from blind caching. Comparison has no model client and makes zero LLM calls. ### Frozen-fixture evaluation All four frozen cases used independently authored persisted blind recommendations; expert assignments were not blind expected answers. OMPS_NPP_NMTO3_L2_2, ESMR_SWE_NIMBUS7_1, MERRA2_400, and OCO2_L2_Lite_FP_11.2 resolved 1, 1, 21, and 2 expert entries without conflict. With the fixed independent blind recommendation, each produced one proposed_addition and one review record. These are structural integration outcomes, not accuracy claims. ### Validation results - python -m ruff check .: all checks passed. - python -m ruff format --check .: 95 files already formatted. - python -m pytest -q: complete suite passed; one existing opt-in integration test skipped. - Publication-only suite: passed; one existing opt-in integration test skipped. - Milestone 9 focused suite: 38 passed. - Schema, fixture-manifest, and expert-boundary suite: 35 passed. - git diff --check: passed. Coverage includes all hierarchy levels, resolution failures and ambiguity, duplicates, all categories and precedence, multiple branches, semantic no-classification, additive reviews, idempotent and atomic persistence, cache isolation, sanitized logs, invalid-boundary zero-unseal behavior, frozen fixtures, and zero comparison model calls. ### Security, immutability, deviations, and limitations All frozen files remain contained regular files with recorded hashes; nothing under tests/fixtures/datasets/ changed. ArticleRecord, ArticleResult, publication prompts and persistence/cache behavior, schemas/classification_result.schema.json, data/gcmd_hierarchy.json, data/articles.json, and prototype/app_hf_poc.py remain unchanged. There were no functional deviations. Generated provenance schemas changed only because the shared runtime stage enum now includes comparison. There is no CMR update, network access, blind reclassification, UI work, cleanup, deletion, or cache eviction. Review records are future human-review inputs, not review decisions. Comparison does not update CMR or experts, and no UI or evaluation dashboard is included. All Milestone 9 acceptance criteria are satisfied. Milestone 10 and later work remain deferred. ## Pre-Milestone 10 Corrective Prerequisite — Production Dataset Application-Service Composition Status: complete (2026-08-13). Milestone 10 is unblocked but was not started. ### Blocker and corrective scope Milestone 10 was blocked because BlindStageServices had only test fakes and no production composition or two-operation application façade. This correction adds that missing boundary without adding Gradio code or changing any completed stage policy. ### Files added and changed Created: - src/gcmd_classifier/datasets/application_service.py - schemas/dataset_awaiting_readme_selection.schema.json - tests/datasets/test_application_service_corrective.py Modified: - src/gcmd_classifier/datasets/schemas.py - DATASET_README_PROGRESS.md No files were deleted. Root app.py, all current UI modules, the prototype, publication models/prompts/schema, vocabulary and article data, and frozen dataset fixtures were not changed. ### Production composition ProductionBlindStageServices is the concrete adapter over the existing CMR retrieval/resolution, blind-source separation, exact README discovery/selection, selected public PDF retrieval, extraction, product resolution, evidence selection/packet construction, shared dataset hierarchy classifier, persistence/verifier, and Milestone 9 comparison services. The adapter contains stage wiring only; every URL, document, extraction, evidence, classification, expert-resolution, comparison, and review rule remains owned by its completed module. build_dataset_application_service constructs the artifact store, configured HTTP clients, DNS resolver, vocabulary, provider-neutral model client, settings, concrete stage adapter, and stateless DatasetApplicationService. Construction performs no HTTP request, model call, run creation, expert unsealing, or fixture-content read. Tests inject transports, resolver, vocabulary, and model client. ### Two application operations retrieve_collection(RetrieveCollectionRequest) retrieves and resolves one CMR collection, persists and verifies exact CMR bytes plus the sealed expert artifact and binding, constructs the blind view, discovers exact READ-ME candidates, persists checkpoints, and returns sanitized identity/candidate data. It never retrieves a PDF or calls a model. classify_selected_readme(ClassifySelectedReadmeRequest) reloads and verifies the persisted waiting checkpoint and source artifacts, validates one exact source-position/URL pair through the existing selection service, then delegates the continuation stages. Comparison receives only VerifiedPersistedBlindResultReference after atomic blind persistence and hash verification. Comparison failure preserves the verified blind result and returns a separate sanitized comparison failure. ### Waiting state, selection, and isolation AwaitingReadmeSelection is a strict persisted checkpoint with run/session and collection identity, ordered discovery result, opaque source-boundary references, discovery artifact, and the authoritative persisted BlindRunState. Its generated schema is dataset-awaiting-readme-selection-v1. It is neither terminal nor semantic not-classified, and it preserves false classification-attempt state and zero model calls. One candidate is marked preselectable but still requires the second operation. Multiple candidates have no preselection. Zero candidates return the existing skipped outcome without a waiting checkpoint. The second operation reloads the checkpoint through the run-isolated store and re-verifies exact CMR, sealed, binding, and discovery artifacts. It rejects mismatched run/session/checkpoint references, stale or terminal state, unknown source positions, URL substitutions, position/URL mismatches, local paths, fixture paths, and cross-session references before PDF retrieval or inference. No module-level session, run, selection, or result state exists. ### Recovery, idempotency, and attempt semantics Waiting checkpoints survive service reconstruction and can resume without CMR re-retrieval. A verified completed application result is persisted separately and safely reused for an identical repeated selection with zero current-call model calls. Persisted model-call provenance remains distinct. The existing DurableInferenceClient remains the sole inference boundary: dispatch intent is persisted before the first actual model request, deterministic stages do not mark an attempt, and the application façade only reports the persisted state. Continuation failures retain all already published immutable artifacts. The façade does not delete or overwrite them. Fine-grained automatic resume from every individual in-progress continuation stage remains governed by the existing runtime checkpoint/recovery primitives; this correction does not add a competing recovery engine. ### Application-facing safety Retrieve results contain no raw CMR bytes, sealed payload, ScienceKeywords, absolute paths, credentials, or authorization data. Processing results expose sanitized extraction/product/evidence diagnostics rather than extracted README blocks or packet text. Optional stage events contain only identities, stage/status, attempt state, model-call count, and safe event code. Blind results remain unchanged. Expert values appear only inside the authorized Milestone 9 comparison result after the verified persisted blind gate. ### Corrective tests and frozen cases Focused tests cover production construction with injected dependencies; one, multiple, and zero candidates; invalid CMR URLs; exact candidate order; persisted waiting recovery; arbitrary URL/local/fixture substitution; wrong position/URL; cross-session denial; valid selection reaching PDF retrieval without premature inference; stage-event and result expert isolation; protected hashes; and all four frozen CMR cases through the first production operation. All four frozen cases return their recorded Concept ID, Native ID, and selected README URL in source order, persist a waiting checkpoint, and make zero model calls. Frozen expert keywords are not blind expected answers. ### Validation and immutability - python -m ruff check .: passed. - python -m ruff format --check .: passed. - Complete pytest suite: passed with the existing opt-in integration test skipped. - Corrective focused suite: 21 passed. - Publication-only regression suite: passed with the existing opt-in integration test skipped. - Schema, fixture, expert-boundary, and corrective focused suite: passed. - Generated dataset schemas match their typed source models. All eight frozen CMR/PDF hashes remain manifest-exact. app.py, src/gcmd_classifier/ui/gradio_app.py, prototype/app_hf_poc.py, schemas/classification_result.schema.json, data/gcmd_hierarchy.json, and data/articles.json retain their pre-correction hashes. No Gradio component, tab, callback, or presenter was added. No stage business rule, expert-resolution rule, comparison category, or review policy was duplicated. No automatic cleanup, retention, deletion, cache eviction, or artifact mutation was added. ## Milestone 10 — Two-Tab Gradio and Hugging Face Integration Status: complete (2026-08-13). Post-MVP work was not started. ### Files added and changed Created: - src/gcmd_classifier/ui/dataset_tab.py - tests/datasets/test_dataset_ui_milestone10.py Modified: - src/gcmd_classifier/ui/gradio_app.py - tests/test_ui.py - tests/datasets/test_application_service_corrective.py - requirements.txt - .env.example - README.md - DATASET_README_PROGRESS.md No files were deleted. The corrective boundary test changed only to permit the now-authorized active UI composition change; its root launcher, prototype, and production-service protections remain. ### Application composition and publication compatibility The active factory now builds one gr.Blocks application containing Publication Classifier first and Dataset Classifier second. build_publication_tab retains the existing publication Title, Abstract, optional DOI, optional Year, Classify action, compact summary, accepted-classification table, detailed ArticleResult JSON, diagnostics, handler argument order, serialization, model client behavior, prompt behavior, and error handling. create_demo remains a backward-compatible alias for build_app. Publication compatibility is protected by the existing 17 UI tests plus the complete publication suite. ArticleRecord, ArticleResult, article prompts and prompt versions, publication semantic validation, and schemas/classification_result.schema.json remain unchanged. ### Thin launcher and Hugging Face integration Root app.py is unchanged and remains the guarded Space launcher. It imports and constructs the packaged application, queues and launches only through launch(), and contains no Dataset layout or business logic. Import does not launch a server. requirements.txt now explicitly retains the dataset runtime dependencies and declares the supported Gradio runtime. .env.example documents DATASET_OUTPUT_ROOT and DATASET_CACHE_ROOT. README.md documents the two tabs, two explicit actions, public unauthenticated sources, model configuration, output/cache roots, expert-review boundary, and absence of automatic cleanup. Construction under installed Gradio 5.50 succeeds without retrieval, inference, or run creation. ### Dataset UI service and state boundary The Dataset tab accepts the concrete DatasetApplicationService through dependency injection and invokes only retrieve_collection and classify_selected_readme. It does not parse or authorize URLs, inspect files, retrieve HTTP resources, extract PDFs, resolve products, select evidence, construct packets, classify GCMD paths, persist artifacts, unseal experts, compare results, or apply review policy. DatasetSessionState is strict, frozen, and created through a callable gr.State initializer, producing a new opaque session identity per browser session. It stores only submitted URL, opaque run/waiting reference, sanitized collection identity, approved candidate views, selection identity, typed stage views, sanitized diagnostics, verified blind presentation, and authorized comparison presentation. No raw CMR/PDF bytes, extracted README text, packet text, prompt, model response, sealed payload, credential, header, cookie, absolute path, or mutable module-level run state is stored. ### Two-action workflow and enablement Initial construction leaves Retrieve collection enabled and classification disabled, with empty diagnostics and hidden blind/comparison outputs. Retrieve collection forwards the submitted string unchanged to the production façade. It displays Concept ID, Native ID, ShortName, Version, optional EntryTitle, and every exact candidate in source order with source position, exact URL, Description, and Subtype. One candidate is preselected but never classified automatically. Multiple candidates begin without selection. Zero candidates display NO_README_AVAILABLE and remain unattempted. Download selected README and classify is enabled only for a current successful retrieval with an explicit current candidate, waiting reference, non-stale URL, and no active classification request. The request uses the candidate identity held in current server-side state; arbitrary URLs, local paths, stale IDs, and cross-session candidate IDs cannot reach the service. URL edits stale the current retrieval, disable classification, and hide prior results without deleting artifacts. concurrency_limit=1 plus application-service idempotency prevents concurrent duplicate inference. ### Status, diagnostics, blind results, and comparison Presenters expose typed stage/status/cache fields only. Extraction, product-resolution, and evidence diagnostics come from sanitized application-service dictionaries. They preserve coverage mode, packet identity/counts, warnings, and the separate product-match and keyword-evidence confidence dimensions without displaying source blocks or packet text. A blind result is rendered only when a VerifiedPersistedBlindResultReference exists. The allow-listed presenter shows collection identity, processing/classification outcome, authoritative classification paths/UUIDs, citations, stopping decisions, confidence dimensions, review status, and semantic not-classified reason. Pre-inference skips and failures are never rendered as semantic not-classified. Comparison remains hidden until authorized Milestone 9 output exists after verified blind persistence. Its presenter excludes sealed/source artifacts and displays only comparison status/items, authorized resolved/unresolved/expert-only context, category counts, review status/reasons, and advisory-only notice. Comparison failure preserves the verified blind result while hiding partial expert data. ### Session isolation and rendering safety Pure controller/presenter tests prove fresh session objects, independent candidates and results, cross-session selection rejection, stale-selection rejection, no result crossing, and absence of sealed/raw-source fields from serialized state. Service results with mismatched run or session identity are rejected. Errors render only fixed sanitized UI messages and safe error codes; raw exceptions, service error text, absolute paths, HTML, expert sentinels, and prompt-injection content are not rendered. No raw HTML is built from source data. ### Tests and validation - python -m ruff check .: all checks passed. - python -m ruff format --check .: 99 files already formatted. - python -m pytest -q: 642 passed, 1 existing opt-in integration test skipped. - Test collection: 643 tests. - Publication-only regression: 304 passed, 1 skipped. - Focused publication plus Dataset UI suite: 34 passed. - UI, schema, fixture-manifest, and expert-boundary focused suite: 69 passed. - Real Gradio construction: one Blocks application, exact two ordered tabs, one session State, zero service calls, zero launches. - Fake Gradio construction and callback wiring: passed. - Schema drift: all generated dataset schemas match typed source models. - git diff --check: passed. The only warning is an upstream Gradio 5.50 deprecation notice concerning Dataframe row_count tuple handling in the preserved publication component; behavior remains compatible and no publication default was changed. ### Security, immutability, and scope confirmation All eight frozen fixture hashes remain manifest-exact and nothing under tests/fixtures/datasets/ changed. Root app.py, prototype/app_hf_poc.py, schemas/classification_result.schema.json, data/gcmd_hierarchy.json, data/articles.json, ArticleRecord, ArticleResult, and article prompt files retain their protected baselines. No source content or expert value can cross sessions through UI state. No retrieval, extraction, product, evidence, packet, classification, comparison, persistence, or review-policy business logic was added to the Dataset tab. No session/global run dictionary, automatic cleanup, retention, deletion, cache eviction, disconnect cleanup, or artifact mutation was added. Known limitations remain the approved MVP boundaries: the interface is a development demonstration rather than production tenancy or adjudication; confidence values are uncalibrated; reviews are advisory; CMR is never updated; OCR and non-PDF inputs remain unsupported. All requested Milestone 10 acceptance criteria are satisfied. ## Pre-Milestone 11 Fixture-Manifest Contract Correction Status: complete (2026-08-13). The test-only fixture contract now requires strict, non-negative, zero-based `selected_readme_source_index`, indexing the original unfiltered CMR `RelatedUrls` array with the same semantics as production `READMECandidate.source_index`. The four manifest values are OMPS `5`, ESMR `4`, MERRA-2 `9`, and OCO-2 `5`. The test-only loader resolves the frozen collection through production CMR resolution and README discovery, then requires exact index, case-sensitive `READ-ME` subtype, and URL agreement. Missing, coerced, aliased, negative, out-of-range, non-README, and mismatched values fail without repair. Changed files were `tests/fixtures/datasets/cases.json`, `tests/datasets/fixture_manifest.py`, and `tests/datasets/test_fixture_manifest.py`. Focused correction tests passed (62 tests); the then-current complete suite passed 656 tests with one existing opt-in live test skipped. Frozen CMR/PDF and fixture-guide hashes remained unchanged. No production behavior changed. ## Milestone 11 — Smoke/Evaluation Harness and Documentation Status: complete (2026-08-13). No post-MVP enhancement was started. ### Files added and changed Added: - `src/gcmd_classifier/datasets/smoke.py` - `tests/datasets/test_smoke_milestone11.py` Modified: - `README.md` - `pyproject.toml` - `DATASET_README_PROGRESS.md` The preceding manifest correction remains a separately authorized prerequisite. No production retrieval, extraction, product-resolution, evidence, classifier, comparison, persistence, cache, application-service, UI, article, or schema file changed during Milestone 11. ### Requirements implemented The typed `DatasetSmokeHarness` is callable from Python and through `python -m gcmd_classifier.datasets.smoke --output-root `. It validates the strict manifest, file containment, regular-file status, manifest hashes, identities, revisions, timestamps/status, exact selected source index, exact URL, and exact `READ-ME` subtype before processing. Production CMR resolution and README discovery are used in preflight; mismatches return a typed failure before constructing a service or fake client. Each valid case runs the real production two-operation application service. `FrozenWorkflowTransport` accepts only the configured public CMR URL, serves exact frozen CMR bytes, and refuses PDF mapping until the candidate has been returned by the first application operation and explicitly selected by original source index plus exact URL. The ordered trace is `cmr_response_served`, `candidate_discovered_by_application`, `explicit_selection_authorized`, then `selected_pdf_served`. Local paths, unconfigured URLs, wrong indexes, and URL substitutions are rejected. The adaptive structural fake responds only to blind runtime prompt contracts. It has no access to sealed expert artifacts and is authored independently of frozen expert assignments. It returns deterministic simulated duration/token metadata and uses runtime block/candidate identifiers. Reports label fake limitations, unavailable cost, uncalibrated confidence, and advisory review policy. Stable logical hashes exclude run IDs, absolute paths, wall-clock timestamps, and output roots. Reports contain typed provenance outcomes, stage states, document/extraction diagnostics, separate product and keyword confidence, product/evidence/packet counts, vocabulary validity, comparison category counts, cache/checkpoint counts, simulated latency/tokens, unavailable cost reason, warnings, and structural outcome. They exclude source payloads/text, PDF bytes, packet text, prompts, raw responses, expert payloads, credentials, secrets, and absolute paths. Reports may be written only beneath an explicit output root and never beneath frozen fixtures. A dedicated fixture-guide test reads `tests/fixtures/datasets/README.md` only as human documentation. Harness configuration comes exclusively from `cases.json`. The `live_dataset` pytest marker identifies a bounded, credential-free, public CMR/README test enabled only by `GCMD_RUN_LIVE_DATASET_TESTS=1`; it is skipped by default and cannot update fixtures. Existing focused suites supply the adverse-path evaluation for no README, non-public source, unsupported format, malformed and image-only PDF, malformed extraction, injection text, evidence/packet failure, invalid classifier output, semantic `not_classified`, malformed/invalid/ambiguous/conflicting expert assignment, comparison-before-verification denial, and comparison persistence failure. Milestone 11 did not duplicate or alter those production policies. ### Frozen structural evaluation - `omps_single_product`: expected/actual `single_product`; 24 pages, 37,218 extracted characters, 102 included/packet blocks, 9 current-run model calls, 227,587 simulated tokens, 0.09 simulated model seconds, one valid accepted path out of one, one advisory `proposed_addition`. - `esmr_explicit_multi_product`: expected/actual `explicit_multi_product`; 21 pages, 25,200 characters, 222 included blocks, one other-product block excluded, 222 packet blocks, 15 calls, 461,466 simulated tokens, 0.15 seconds, one valid path out of one, one advisory `proposed_addition`. - `merra2_product_family`: expected/actual `product_family`; 66 pages, 77,539 characters, 115 included blocks, bounded reduction from 115 to 4 packet blocks (111 removed), 15 calls, 443,199 simulated tokens, 0.15 seconds, one valid path out of one, one advisory `proposed_addition`. - `oco2_vague_scope`: expected/actual `vague_product_scope`; 14 pages, 22,146 characters, zero eligible/packet blocks, 3 product/evidence model calls, 77,080 simulated tokens, 0.03 seconds, review required, no blind result, no comparison, and vocabulary validity `not_applicable` with denominator zero. Aggregate structural status is three passed and one expected review-required case; coverage agreement is 4/4. Accepted vocabulary validity is 3/3 (100%). The stable logical report hash from the recorded run is `c8f258162bbe4ad318983e7633c6d10b18520cada172377bc98da7bb258a6e09`. Repeated isolated runs produced the same logical hash. Comparison categories are mechanics-only and are not scientific accuracy measurements. ### Documentation Root `README.md` now preserves publication batch guidance and documents the Phase 2 source boundary, setup, output/cache configuration, local/Hugging Face launch, two tabs and two explicit actions, one/multiple/no-README behavior, public/PDF/OCR limitations, privacy and model-provider data handling, retention responsibility, status distinctions, separate confidence meanings, advisory comparison categories, human-review requirements, smoke limitations, opt-in live command, and manual fixture policy. ### Validation commands and actual results - `python -m ruff check .`: passed. - `python -m ruff format --check .`: 101 files already formatted. - Schema/drift suite (`tests/datasets/test_models_and_schemas.py`, `tests/datasets/test_expert_source_boundary.py`, `tests/test_output_schema.py`): 49 passed. - Focused Milestone 11 suite: 7 passed, 1 `live_dataset` test skipped by default. - Complete focused frozen/adverse dataset evaluation: passed with the `live_dataset` test skipped. - Complete dataset suite: 359 passed, 1 skipped (360 collected). - Publication regression suite: 304 passed, 1 existing opt-in live-model smoke skipped (305 collected). - Complete suite: 663 passed, 2 expected opt-in skips (665 collected). - `git diff --check`: passed in final audit. The optional live dataset test was not enabled or run. The only warning is the existing upstream Gradio 5.50 Dataframe `row_count` deprecation warning. ### Security, immutability, deviations, limitations, and deferred work All four CMR and four PDF hashes remained manifest-exact, and the fixture guide remained unchanged. Smoke execution wrote only to explicit temporary/output roots. No expected keyword list was read from `cases.json`, and fake answers were not derived from frozen `ScienceKeywords`. Expert values were opened only by the existing post-verification comparison service. No network or live model call occurred in routine evaluation; no credentials were constructed or read for retrieval; no CMR write path exists. No fixture or generated artifact was refreshed, normalized, repaired, replaced, expired, evicted, or deleted. No application policy was changed to make a case pass. Available cache metrics are reported; the first-run harness intentionally exercises misses/bypass rather than fabricating cache hits. Actual provider cost is unavailable without an approved versioned pricing configuration. Confidence remains uncalibrated. Frozen fake runs prove structural behavior only, not scientific precision, recall, or accuracy. Scientific evaluation against a separately expert-reviewed expected-classification corpus remains deferred. All requested Milestone 11 acceptance criteria are satisfied. ## Final Phase 2 Status Phase 2 is complete through all 11 approved milestones plus the sealed-expert, production-composition, and fixture-manifest corrective prerequisites. The production workflow preserves exact CMR identity, discovers and explicitly selects public READ-ME PDFs, extracts provenance-bearing blocks, resolves product applicability, constructs immutable joint evidence packets, reuses constrained hierarchical classification, persists and verifies blind results, and performs advisory expert comparison only across the verified post-persistence gate. The development Space presents separate Publication and Dataset tabs and preserves publication schemas/prompts. The four-case immutable corpus covers single-product, explicit multi-product, product-family, and vague-scope behavior. Routine evaluation is deterministic and network/model isolated, with 100% vocabulary validity for accepted fake outputs and explicit review blocking for vague scope. Proposed additions/refinements remain advisory; expert-only assignments are never removal candidates; nothing is written to CMR. Known limitations are unchanged: public unauthenticated PDF sources only, no OCR, no production authentication/tenancy, no automatic retention cleanup, uncalibrated confidence, and no scientific-accuracy claim from fake runs. A separately governed expert-reviewed scientific evaluation dataset and methodology remain deferred.