CROSSWIRE / docs /verification-resilience-workflows.md
Kai9987kai's picture
Release crosswire-formal-synthesis-2026-11-03 from GitHub 91e7ac87c4e4
450ac2f verified
|
Raw History Blame Contribute Delete
5.13 kB

Resilience workflow verification — 2026-10-30

The upgrade completes controlled fault experiments across runs, studies, browser configuration, condition-stratified analysis and saved reports. It also repairs demonstrated finite certification and private audit errors. All evidence here is local and synthetic.

Implemented behavior

  • Optional sensor drift, actuator swap, injected delay, stuck output and semantic swap settings include a zero-based onset, explicit calibration/deployment/both phase, and validated stuck symbol. Unknown fields and ineffective duplicate schedule defaults are rejected.
  • Studies accept fault_schedules, including null nominal controls. Shared condition identity prevents different types, phases, onset steps or stuck symbols from pooling or matching. Saved summaries, reliability records, reports and workbench tables retain those settings.
  • Faults compose with existing sensor/semantic mechanisms. Independent injected-delay memory is reset and included in the private finite target export. Equivalence checking uses deployment-phase dynamics and a saturated onset counter. Onsets beyond 512 withhold an exact exported model rather than allocating an unbounded state graph; actual trajectory auditing still executes the schedule.
  • Calibration hazard reconstruction uses calibration phase. Calibration-only schedules do not masquerade as detected deployment change points.
  • Policy information metrics distinguish complete observation paths, soft-return synthesis obeys the remaining horizon, and contract evaluation supports larger nonnegative output alphabets.
  • Reports check saved envelope and scientific digests, including historical source identities. This is integrity checking, not target replay or authentication. Import and replay still require the exact source revision.
  • CI now checks the wheel in a clean temporary environment outside the checkout, with isolated Python, actual deployment and replay. It compares installed Python source identity and every web asset against the checkout.

Validation

The final pre-publication full suite passed: 539 tests in 300.10 seconds, including the direct-script CLI regressions. The report-rendering check also passed all 23 report tests. Expected failures were reproduced for the corrected scientific behaviors.

  • Real Chromium workbench check passed: native input validation, nominal defaults, fault configuration, actual faulted run and paired study, export, import/replay, row inspection, model diagnostics and 390-pixel mobile layout. Desktop and mobile screenshots were inspected.
  • Fresh eight-run study: configs/resilience-workflows.json -> reports/resilience-workflows-receipt.json. Independent public checking passed all eight runs. CLI exact replay matched scientific hash a4d9a0fa2aec1b776bc04f64535d8252c6761c4586230ca1ce131111e4a72e1f.
  • All 14 saved study receipts passed source-independent digest validation and report rendering. The original 13 receipt files remain unchanged; those require their original source revision for replay.
  • Wheel built successfully. Its clean outside-checkout installation matched Python source hash f0c070bc77cebfbd711ec6be9282ed20779d1e6aa91769619905d345f4ee20a1 and all three web assets. A real routing/adaptive/shielded smoke used zero acquisition calls and five deployment calls, then replayed exactly. Empty/multiple-wheel inputs and stale-source wheels were rejected.
  • The standalone report command was exercised from outside the checkout under isolated Python, for valid historical evidence and corrupted evidence.
  • Wheel and source distribution builds passed. Archive inspection confirmed the web assets, shared conditions module, new configuration/receipt, verification document and distribution checker are included where required. git diff --check passed.

Fresh smoke-study outcomes and scope

Both table and procedural engines were nuisance repeats of one latch parent, with label seed 42 and unrestricted calibration at budget 16. This is a workflow regression demonstration, not an independently sampled reliability study.

Deployment condition Runs Goal completions Monitor interventions Audited deployment hazards
Nominal 2 2 0 0
Sensor drift at step 2 2 0 2 No hazard in observed prefix; incomplete outcome remains unknown
Stuck output at step 2 2 0 2 No hazard in observed prefix; incomplete outcome remains unknown
Semantic swap at step 1 2 0 0 2

Unrestricted calibration incurred one hazardous probe per run and is reported separately from deployment. Semantic swaps remained invisible in raw outputs and hazardous in the private audit. No deployment-safety or semantic-grounding claim follows from these results.

The factored solver and soft-return synthesizer remain direct library capabilities. Current CLI/run/study acquisition uses CandidateSpace. This report records local verification; publication and hosted Windows/Linux, Python 3.11/3.12 CI results are tracked separately in Git and GitHub Actions.