| event_id,run,cause,ruling,resolution,evidence |
| smoke_initial_zero_step,Review17 smoke attempt,Single malformed canary was treated as an immediate startup stop; 0 optimizer steps.,Review17/18: preserve and exclude; canary failures are floor-rewarded and counted in aggregate rate.,Guard corrected and regression-tested; fresh smoke separately authorized.,results/phase6/review17_smoke_20261006T042616Z/; tests/test_review18_smoke.py |
| smoke_step8_failure_rate,"Smoke run, step 8","2.0178% failure rate exceeded old 2% threshold by one rollout, all 93 failures were policy malformed outputs; zero verifier/infra failures.","Review18 prospective D-GRPO-6a: >3% stop, or >2% and non-declining for three consecutive step windows.",Weights-only resume with Adam reset authorized for smoke; completed steps 9–20; no scale implication.,docs/reports/historical/results_phase6_review18_interim_report.md; results/phase6/review19_resume_20261006T105346Z/ |
| scale_v3_sampler_invariant,First scale sampler attempt,Stopped manually after 1 optimizer step when dataloader epoch progress advanced incorrectly; sampler omitted group repetition.,Excluded from scale results; no resume. Fresh launch only after fix was authorized.,Sampler/dataloader invariant and fractional-epoch assertions added; later fresh run passed preflight.,results/phase6/review20_launch_incident.md; results/phase6/scale_20261006_v3/scale_final_status.json |
| scale_v4_failure_rate,Fresh scale attempt v4,D-GRPO-6a failure_above_2_percent_non_declining_three_steps at step 14; 1.7451 allocated GPU hours.,Stopped; diagnosis and subsequent explicit authorizations required.,Preserved run evidence; later scale run v5 launched under updated prospective failure policy.,results/phase6/scale_20261006_v4/scale_final_status.json; DECISIONS.md Review20 entries |
| production_step400_reward_tripwire,Production GRPO continuation,R2/R3 rising while R1 flat/falling for registered window; 400 updates completed.,Step-400 deviation accepted; no replay/resume; step399 is inference-only.,Callback ordering defect fixed and regression-tested; no step400 checkpoint or VALID eval exists.,results/phase6/review23/step400_stop_report.md; results/phase6/review23/requirement_audit.md |
| e3_registered_early_stop,E3 evidence-first detector,Seed42 TRAIN-holdout AUC 0.49797 below matched classical+0.01 gate.,Registered stop after seed42; seeds43/44 and ablations skipped.,Negative result retained; no tuning or follow-on training.,results/phase6/review23/report_B_e3.md; results/phase6/review23/e3/holdout_training_status.json |
| e4_infrastructure_interruption,E4 balanced reward,Infrastructure interruption at step49; 5.880 GPU hours; no registered endpoint.,Archived and closed; no resume from partial checkpoint. E4b fresh re-execution explicitly authorized.,E4b completed frozen step200 endpoint.,results/phase6/review24/e4_interruption_report.md; results/phase6/review26/e4b_balanced_reward/ |
| e4b_step100_software_failure,E4b first step100 matched eval attempt,Redundant adapter publication/move raised a software exception after checkpoint save.,Stop and preserve; no auto-resume for software failure. Review28 authorized fix and standalone retry.,"Publication made idempotent, test added, standalone matched eval completed, then full-state restore resumed.",DECISIONS.md Review28; tests/test_e4_matched_retry.py; results/phase6/review26/e4b_balanced_reward/step100_matched_eval_retry/ |
| test_aggregation_failure,Review35 final TEST aggregation,Post-inference KeyError: classical ranking field stored as decision_score; pair accumulator also overwrote a member.,Persist partials; no split reread or inference rerun. Fix aggregation and recompute from saved outputs only.,Final aggregate completed_with_posthoc_aggregation; raw outputs immutable; corrected code regression-tested.,results/phase10/final_test_20261012/test_final_report.md; tests/test_review35_final_test_metrics.py |
|
|
