CROSSWIRE / docs /build-ledger.md
Kai9987kai's picture
Release crosswire-formal-synthesis-2026-11-03 from GitHub 91e7ac87c4e4
450ac2f verified
|
Raw History Blame Contribute Delete
21.6 kB

CROSSWIRE build ledger

Approved design: 22 October 2026. Baseline commit 3a7385b. User explicitly approved building in the new standalone folder.

Ruling: execute the implementation plan without another approval round — the direct “Approve—build CROSSWIRE” instruction authorizes implementation — routine implementation choices remain reviewable in Git.

Ruling: retain the standalone checkout rather than create another worktree — the user's named new project folder is the intended work location — no existing NEUROGRAFT source/checkpoint is part of this build.

Ruling: parallelize disjoint core, target-protocol and browser files — active multi-agent instructions favor independent work and each worker has exclusive paths — root owns integration and commits; shared interfaces are fixed in implementation-contract.md.

Preflight interface scan

Tasks Produced/consumed boundary Finding
1 / 2 Machine serialization used by fixture references/audit models fixed tuple cells and integer opaque symbols
1 / 3 CandidateSpace predictions/plans/ambiguity and exact product comparison fixed methods and counters, root does not introspect private target
2 / 3 Case public/private and TargetProcess query/step/reset/costs public-only learner input, audit after freeze
3 / 4 Run results, receipt exports and async job status fixed HTTP and result schema
4 / 5 CLI/package/web assets root verifies wheel and actual browser workflows
1 Algebra tests versus core implementation excludes empty-family vacuous admission and prefix hazards
2 Fixture independence versus runtime procedural branch cannot read reference transition table
3 Exact guarantees versus finite evidence conditional soundness; misspecification/semantic corruption explicit
4 UI versus core visualizes Python output, no duplicated scientific learner
5 Research conclusion versus software status negative pilot permitted; no breakthrough/publication claim

Integration Review Rulings & Resolutions

  1. Deployment Prefix Auditing: Fixed in audit.py. Only executed prefixes are audited; aborted suffixes are never simulated as if executed.
  2. Unified System-Call Cap: Fixed in study.py and baselines.py. Resets and steps share a common call budget $B$.
  3. Deployment Failure Accounting: Fixed in study.py. Deployment interface crashes are counted in summary metrics without mutating frozen decisions.
  4. Forced-Fit Residual Diagnostics: Fixed in study.py. Preserved chosen adapter and residual mismatch breakdown in baseline diagnostics.
  5. AALpy Transition Work: Fixed in baselines.py. Model comparison steps are counted explicitly.
  6. Stratified Paired Contrasts: Fixed in study.py. Matched cell-by-cell comparisons remain within identical call-cap levels.
  7. Asynchronous Import/Replay: Fixed in server.py and app.js. Receipt import and verification run as cancellable jobs on the scientific worker.

Milestone Status

  • Task 1 (Core Algebra & Mappings): Completed and verified with 17 tests.
  • Task 2 (Targets & Subprocess Protocol): Completed and verified with 34 tests.
  • Task 3 (Study Orchestration, AALpy, Audit, Receipts): Completed and verified with 20 tests.
  • Task 4 (CLI, Async Job Server, Browser Workbench): Completed and verified with 9 tests, Playwright e2e automation, and mobile responsive check.
  • Task 5 (Reproducible Pilot Benchmarks, Verification, Documentation):
    • Smoke benchmark (27 runs): verified.
    • In-family benchmark (350 runs): verified.
    • Stress benchmark (216 runs): verified.
    • Wheel build & clean environment parity: verified.
    • docs/paper.md and docs/verification.md: authored.
    • README.md and Run-CROSSWIRE.ps1: operational.
  • Task 6 (Closed-Loop Dynamic Runtime Shield, Temporal Adapter Synthesis, & CEGP):
    • Closed-Loop Dynamic Runtime Shield (crosswire/shield.py): monitors deployment step responses; halts on empty survivor set or pre-emptively intercepts latent hazard suffixes; verified with 3 tests in tests/test_shield.py.
    • Temporal Latency-Aware Adapter Algebra (crosswire/core.py): expands candidate space to 720 models supporting 1-step pipeline delay and initial buffer search; 3-tuple synchronized product comparison; verified with 3 tests in tests/test_temporal.py.
    • Counterexample-Guided Active Probing (crosswire/core.py): optimized $O(W \cdot N)$ marginal frequency counting for pair disagreement; verified with 2 tests in tests/test_cegp.py.
    • Integrated full test suite passing (88/88 passed in 44.55s).
    • Open-Loop vs. Shielded Comparative Benchmark (reports/pilot-shield-openloop-receipt.json & reports/pilot-shield-shielded-receipt.json, 72 runs): empirically verified 8 hazards averted on misspecified targets.
    • Temporal Adapter Benchmark (reports/pilot-temporal-receipt.json, 4 runs): achieved 100% useful coverage with 0% selective harm on pipeline-delayed interfaces.
    • Total verified empirical evaluations: 669 runs.
    • Research paper and verification report updated with empirical tables and mathematical formulations.
  • Task 7 (Decision-Focused CEGP Witnesses, Pre-Step Guarding, Causal Auditing, & Portfolio Survey):
    • Decision-focused CEGP probe synthesis (crosswire/core.py): dynamically synthesizes exact distinguishing counterexample witnesses via product BFS beyond the fixed probe pool. Evaluated on long_trace, resolving candidates that stalled standard policies and achieving 100% useful coverage (+1.000 paired difference).
    • Pre-step preemptive hazard interception (crosswire/shield.py): evaluates pending action before dispatch; halts without executing target.step if the action is unanimously hazardous under surviving adapters.
    • Full Test Suite Passing (tests/test_escalation.py + full suite): 116/116 tests passing in 117.88s.
    • Empirical Benchmark (reports/pilot-escalation-receipt.json & reports/pilot-escalation.md, 36 runs): demonstrated 4.5 mean calls for entropy probing on latch (vs 8.5 for task and 9.0 for fixed), verified autonomous escalation on delay, certified structural contradiction on incompatible, and confirmed Simplex hazard aversion. Cryptographic replay verified bit-exact reproduction. Total verified empirical evaluations: 757 runs.
  • Task 10 (Independent Public-Evidence Certificates, Conservative Recovery, & Condition-Stratified Analysis):
    • Independent Public-Evidence Checker (crosswire/certificates.py): independently re-enumerates adapters from first principles without importing CandidateSpace or internal caches; verifies surviving belief sets against public transcripts and anchors; validates robust words and adaptive policy trees; extracts deletion-minimal contradiction cores upon refutation; binds evidence_certificate before freezing decisions.
    • Conservative Simplex Recovery Fallback (crosswire/shield.py): guarantees recovery fallback dispatch occurs only when live belief is consistent, non-empty, within horizon, and possesses an action with zero modeled hazards across all surviving interpretations. Contradicted beliefs trigger immediate abstention (INTERVENED_ANOMALY) rather than speculative action.
    • Condition-Stratified Analysis & Exact-Cell Pairing (crosswire/study.py & crosswire/audit.py): enforces strict stratification across execution mode, escalation setting, split, anchors, and horizon to prevent cross-condition leakage; audits only executed prefix actions rather than unexecuted suffixes.
    • Full Workbench Verification & Browser Automation (crosswire/web/app.js, crosswire/web/index.html, tools/check_workbench.cjs): fixed JS floating-point serialization divergence via raw text streaming; rendered public model evidence panel with certificate status; verified end-to-end with headless Chromium via Playwright.
    • Full Test Suite Passing: 158/158 tests passing in 145.07s across 15 test modules. Wheel build verified.
  • Task 11 (Submodular EC2 Active Probing, Receding-Horizon Replanning Shield, & Risk-Controlled Admission):
    • Submodular Equivalence Class Edge Cutting Probing (crosswire/core.py, crosswire/study.py): partitions candidate adapters into contract decision equivalence classes; selects probe sequences cutting cross-class edges weighted by Jaccard distance; filters out queries that differentiate contract-equivalent adapters without altering admission. Verified with 6 tests in tests/test_ec2.py.
    • Receding-Horizon Dynamic Replanning Shield (crosswire/shield.py, crosswire/study.py): implements "replan" execution mode; dynamically tracks belief state during deployment; upon encountering intermediate observations that prune surviving models and expose suffix hazards or dead-ends, executes online bounded BFS (_find_safe_suffix) to discover verified safe detours (REPLANNED_ONLINE_SAFE), salvaging execution without hazard. Verified with 5 tests in tests/test_replan.py.
    • Risk-Controlled Version-Space Admission (crosswire/core.py, crosswire/study.py, crosswire/certificates.py): implements "risk_controlled" admission mode inspired by conformal risk control; admits plans satisfying certified risk tolerance $\hat{R}{\text{hazard}} \le \epsilon$ and goal tolerance $\hat{G}{\text{goal}} \ge 1 - \delta$; strictly recovers robust consensus when $\epsilon = 0.0$; validated via independent public certificates. Verified with 4 tests in tests/test_risk_controlled.py.
    • Empirical Benchmark (reports/pilot-replanning-ec2-receipt.json, 96 runs): evaluated across delay, handshake, and latch under replanning execution; confirmed 100% useful coverage for ec2 on delay via autonomous escalation; verified zero selective harm; cryptographically validated with crosswire verify and crosswire check. Brings total verified empirical evaluations to 853 runs across 10 benchmarks.
    • Workbench & UI End-to-End Automation (crosswire/web/app.js, crosswire/web/index.html, tools/check_workbench.cjs): wired ec2, replan, and risk_controlled into controls, status badges, replanning detour indicators, and contradiction core inspectors; verified with Playwright Chromium test.
    • Full Test Suite Passing: 173/173 tests passing in 127.39s across 18 test modules. Wheel build verified.
  • Task 12 (Model-Safe Active Calibration & Configurable Risk Certification - Priorities 1 & 2):
    • Model-Safe Calibration Guards (crosswire/core.py, crosswire/study.py): pre-evaluates pending probe queries against surviving adapter beliefs; guarantees no admitted probe sequence triggers a modeled hazard under any surviving interpretation (probe_safety="model_safe" vs "unrestricted"); halts safely when no safe probe exists.
    • Configurable Risk Tolerance Certification (crosswire/certificates.py, crosswire/study.py): exposes --risk-tolerance across CLI, study runner, and UI; verifies empirical risk fractions in independent evidence certificates.
    • Empirical Studies (reports/pilot-calibration-safe-receipt.json & reports/pilot-calibration-unrestricted-receipt.json, 16 runs): demonstrated zero calibration hazards under model-safe probing vs 2 calibration hazards under unrestricted probing.
  • Task 13 (Uncertainty Diagnostics & Independent Statistical Reliability - Priorities 3 & 4):
    • Public Model-Mismatch Diagnostics (crosswire/certificates.py): evaluates sequential forecasts before observing each output; identifies earliest model class refutations and reset-prefix inconsistencies; reports explicit semantic blind spots (semantic corruption, bypass, unobserved transitions); preserves UNKNOWN inclusion status for compatible models without over-claiming.
    • Independent Target Dynamics & Versioned Generator (crosswire/fixtures.py): implements generated_protocol with separate structural instance_seed and label seed; generates independently sampled stages, progress/fallback actions, and absorbing terminals; procedural targets do not introspect reference tables.
    • Target-Cluster Statistical Reliability (crosswire/reliability.py): aggregates nuisance label permutations and procedural/table implementations within parent clusters; computes exact binomial hazard bounds and conservative Hoeffding coverage intervals for equal-target means; withholds intervals for legacy fixtures or unequal designs.
    • Empirical Benchmark (reports/pilot-independent-protocols-receipt.json, 128 runs over 32 parent clusters): verified 81.25% useful coverage for entropy/robust and 75% for fixed/robust with 0% selective harm; verified exact bit-level replay across all 128 runs. Total verified empirical evaluations reached 997 runs across 13 benchmarks.
    • Full Test Suite Passing: 321/321 tests passing in 93s across 25 test modules. Browser workbench check verified.
  • Task 14 (Adaptive Branching Calibration & Dual-OS CI Workflow - Priorities 5 & 10):
    • Adaptive Within-Query Branching Calibration (crosswire/core.py, crosswire/study.py): implemented choose_branching_step and policy="branching"; selects next action conditionally based on live within-query observations; evaluates predictive output entropy and pre-probe model safety at every dispatch; terminates early when contract consensus is achieved or ambiguity is fully resolved; reduces target call overhead. Verified with 10 unit tests in tests/test_branching_calibration.py.
    • Continuous Integration & Release Automation (.github/workflows/ci.yml): added dual-OS matrix workflow across Ubuntu and Windows for Python 3.11 and 3.12; automates testing, clean wheel packaging and installation, scientific receipt replay verification, and headless Playwright Chromium workbench end-to-end checks.
    • Full Test Suite Passing: 331/331 tests passing across 26 test modules. Browser workbench check verified.
  • Task 15 (Online Replanning Fault Schedules & Factored Candidate Spaces - Priorities 6 & 7):
    • Explicit Change-Point Fault Schedules & Latency Measurement (crosswire/fixtures.py, crosswire/shield.py, crosswire/study.py, crosswire/audit.py, crosswire/cli.py): added fault_schedule to RunConfig, _Engine, and CLI; supports 5 runtime fault transformation modes (sensor_drift, actuator_swap, delay_spike, stuck_output, semantic_swap); measures detection latency $\Delta t = t_{\text{detect}} - t_{\text{fault}}$; tracks causal hazard aversion under shielded abort and online replanned safe detours; verifies epistemological negative control (semantic_swap undetectable by black-box monitor, certified hazardous by private audit). Verified with 7 tests in tests/test_change_points.py.
    • Factored Hypothesis Space & Scaling Beyond Enumeration (crosswire/core.py): implemented FactoredCandidateSpace with orthogonal factor decomposition $\mathcal{P}_I \times \mathcal{P}_O$ and _LazyAdapters proxy; evaluates semantic traces once per input factor ($O(N_I)$); prunes output factor space independently via anchors ($O(N_O \cdot |A|)$); achieves 100% mathematical parity against exhaustive CandidateSpace on 144 and 720 models; scales to 17,280 models with sub-second pruning (0.0005s) and complete-survivor semantics. Verified with 6 tests in tests/test_factored_space.py.
    • Full Test Suite Passing: 344/344 tests passing across 28 test modules.
  • Task 16 (Symbolic Factor-Level Safety/Planning, Realistic Robotics & Protocol Benchmarks, & Workbench Telemetry - Priorities 7, 8 & 9):
    • Symbolic Factor-Level Safety & Full Method Parity (crosswire/core.py): upgraded FactoredCandidateSpace to subclass CandidateSpace with full method parity (losses, level_ids, distinguishing_word, choose_probe, choose_branching_step, ambiguity, and all 7 plan admission modes) plus $O(|\mathcal{I}_{\text{surv}}|)$ symbolic input-factor probe_is_safe, robust, and risk_controlled planning over 17,280 models.
    • Realistic Protocol & Robotics Benchmarks (crosswire/fixtures.py, tests/test_realistic_benchmarks.py): added robotics_airlock (5-state dual-door rover airlock interlock controller) and protocol_session (6-state SYN -> AUTH -> OPEN -> COMMIT session conformance benchmark) with independently authored table and procedural engines, verified zero-hazard model_safe calibration without anchors, out-of-horizon 4-step synthesis, and change-point fault injection. Verified with 7 tests in tests/test_realistic_benchmarks.py.
    • Experimental Workbench Upgrades (crosswire/web/app.js, tools/check_workbench.cjs): exposed branching policy, robotics_airlock and protocol_session benchmarks, factored hypothesis space compression (active_input_factors × active_output_factors), and change-point fault detection latency ($\Delta t$) telemetry in the UI; verified end-to-end in headless Chromium.
    • Full Test Suite Passing: 351/351 tests passing across 29 test modules.
  • Task 17 (BALD Information Profiles, Contingent Policy Tree Analytics, Adaptive Simplex Recovery, & Certified Soft Return-to-Home):
    • BALD Mutual Information & Minimax Probe Ranking (crosswire/core.py): added information_profile and rank_probes to CandidateSpace and FactoredCandidateSpace computing exact predictive Shannon entropy / BALD mutual information $I(\alpha; O \mid w, \mathcal{V})$, cost-efficiency (bits/call), expected survivor reduction, and minimax worst-case survivor bounds.
    • Contingent Policy Tree Analytics & Simplex Recovery (crosswire/adaptive.py, crosswire/study.py): added policy_metrics computing internal decision node counts, leaf counts, best/worst/expected path lengths, max branching factor, and leaf partition information gain; wired Simplex fail-safe recovery (mode="recovery") into execute_policy upon mid-tree anomaly or model risk.
    • Certified Soft Return-to-Home Stabilization (crosswire/shield.py): added synthesize_safe_return to synthesize shortest universally hazard-free suffixes that return the reference state machine to its initial standby state $q_0$ across all surviving input factors.
    • Workbench UI Analytics (crosswire/web/app.js): added contingent policy tree structural summaries and empirical version-space information gain (bits resolved across queries). Verified with 6 tests in tests/test_advanced_features.py.
    • Full Test Suite Passing: 357/357 tests passing across 30 test modules.

2026-10-30 — Resilience workflows and evidence integrity

  • Completed fault controls and study schedule grids, canonical condition isolation, phase-correct private audit/export, and composable scheduled sensor/semantic transformations.
  • Corrected observation-leaf information metrics, soft-return horizon validation, and larger output-alphabet contract evaluation.
  • Reports validate historical receipt digests and retain explicit integrity-only scope; direct-script execution works from a fresh checkout. Existing strict source-bound replay/import remains intact.
  • Final pre-publication full suite: 539 passed in 300.10 seconds, including direct-script CLI regressions. Browser desktop/mobile, fresh eight-run public checking/exact replay, all 14 study integrity/report checks, wheel/sdist archive inspection and clean outside-checkout installed-wheel checks passed.
  • Independent integrated code review found no remaining Important findings. Historical receipts are unchanged; new evidence lives in reports/resilience-workflows-receipt.json and reports/resilience-workflows.md.
  • This entry records local validation. Publication and hosted CI status are tracked separately in Git and GitHub Actions; factored runtime integration and automatic soft-return dispatch are not claimed. See docs/verification-resilience-workflows.md for exact scope and negative-control outcomes.

2026-11-02 — Checked budgeted experiments and factored runtime

  • Added public-only depth-two lookahead calibration, independently reconstructed experiment certificates, actual flat/factored selection across runtime workflows, bounded factor-product robust planning and exact class-histogram task scoring. Preserved dependency-free runtime defaults and finite-model limits.
  • Resolved independent review findings in the global branch objective and UI state. Optimized response partitions/readiness/descriptor construction while preserving complete model weights, prototype behavior and independent objective checks. Separate worker startup deadlines preserve reset/action accounting.
  • Frozen-source full suite: 666 passed in 245.51 seconds. Chromium desktop/mobile, 44 public checks/exact replays, 22 matched backend pairs, actual escalation/AALpy paths, isolated installed-wheel checks, wheel/source archive inspection and whitespace checks passed. Historical tracked receipts remain unchanged.
  • The two-parent generated pilot found no coverage advantage; lookahead used one fewer mean calibration call at cap eight and half a call more at cap four. Unrestricted controls retained calibration hazards and all four semantic-corruption deployment hazards. No general efficiency or safety conclusion is claimed.
  • Source: 1004a81c28c2969b59ced6684b6178f474d3d12787a73614c2a6a0ccc7ea76f2. Exact receipt identities, commands, outcomes and limitations are in docs/verification-budgeted-experiments.md. This records local evidence; publication and hosted CI remain separate.