Sep 21 14:58 p7 probe: prove --ckpt-repo chooses the publisher's weight source 14:56 mirror kernels/ so the kept code is self-contained 14:27 add p6 launcher with a Hub heartbeat: a kernel that cannot report is a kernel you cannot debug (E-056) Sep 20 13:02 p6_cli_probe v3: build metric keys from the YAML's own metric/aggregation fields and rank candidate config files -- the first pass guessed metric_uri and picked a mmlu subject file 12:56 p6_cli_probe: locate lm_eval via a child process -- a user-site dir created mid-run is not on the parent's sys.path 12:50 p6_cli_probe: check run_benchmarks' flag list and PRIMARY_METRIC against the installed 0.4.13 CLI and its shipped YAMLs 12:15 p6_harness_probe v2: measure what a resolving lm-eval install moves, instead of assuming --no-deps is safe (it is not: sacrebleu import failure) 12:08 p6_harness_probe: does lm-eval 0.4.13 resolve on transformers 5.0.0, and what do the 9 task configs declare (configs only, no eval data) 11:12 bootstrap: QUOTA_LEFT_HOURS 24.9 from the 11:10Z live read -- session 1 launch card 11:00 phase4_bootstrap: REV must be at or after the launcher it sha-pins -- point at c237b478 11:00 phase4_prep: REV must be at or after the launcher it sha-pins -- point at c237b478 10:58 p4_plan_sweep: repin to the D-018 launcher at c237b478 10:58 phase4_bootstrap: repin to the D-018 launcher at c237b478 10:58 phase4_prep: repin to the D-018 launcher at c237b478 10:58 launcher: adopt D-018 -- no gradient checkpointing, push every 127 steps, plan at the sustained 12,300 tok/s 10:16 p4_plan_sweep: drive the shipped plan() at each candidate rate to settle the session shape before D-018 09:43 p4_probe2_soak_runner: the wrapper as submitted for probe chain v8 / soak / rehearsal v4 09:43 p4_prep_runner: the wrapper as submitted for probe chain v8 / soak / rehearsal v4 09:43 p4_probe_runner: the wrapper as submitted for probe chain v8 / soak / rehearsal v4 09:26 phase4_bootstrap: E-044 repin (rank-0 stop assertions; memory gate moved onto the training-only leg) 09:26 phase4_prep: E-044 repin (rank-0 stop assertions; memory gate moved onto the training-only leg) 09:26 p4_stop_probe: E-044 repin (rank-0 stop assertions; memory gate moved onto the training-only leg) 09:25 launcher: repin the E-044 trainer 09:09 p4_prep_runner: re-pin to the body actually submitted 09:09 p4_probe_runner: re-pin to the body actually submitted 09:07 phase4_bootstrap: repin to b6210a29 (per-rank post-train marks + readable probe failures) 09:07 phase4_prep: repin to b6210a29 (per-rank post-train marks + readable probe failures) 09:07 p4_stop_probe: repin to b6210a29 (per-rank post-train marks + readable probe failures) 09:06 launcher: repin the instrumented trainer 08:59 phase4_bootstrap: E-042 repin + rehearsal of the launcher's real argv and the prune guard 08:59 phase4_prep: E-042 repin + rehearsal of the launcher's real argv and the prune guard 08:59 p4_stop_probe: E-042 repin + rehearsal of the launcher's real argv and the prune guard 08:55 launcher: repin train files; print TRAIN_ARGV under PREP_ONLY so the rehearsal can parse the real flag list 08:38 prep runner: re-pin to the corrected stage-4 assertion 08:38 prep stage 4: assert on n_hub, not on the log-capped missing list 08:32 kaggle wrappers: pinned runner for the stop/resume probe 08:32 kaggle wrappers: pinned runner for the CPU rehearsal 08:26 phase4_bootstrap: pins at a813e5c7 + bounded-readback rehearsal stage / tee-prefix fix 08:26 phase4_prep: pins at a813e5c7 + bounded-readback rehearsal stage / tee-prefix fix 08:26 p4_stop_probe: pins at a813e5c7 + bounded-readback rehearsal stage / tee-prefix fix 08:19 launcher: repin hubckpt to the bounded read-back 07:56 pin 7570b7b2 : forced-stop legs skip the collective validation pass; probe expectations split leg1 (skip by design) from leg2 (must run it) 07:56 pin 7570b7b2 : forced-stop legs skip the collective validation pass; probe expectations split leg1 (skip by design) from leg2 (must run it) 07:56 pin 7570b7b2 : forced-stop legs skip the collective validation pass; probe expectations split leg1 (skip by design) from leg2 (must run it) 07:55 phase4_session: repin the trainer 07:49 p4_stop_probe: repair the soak branch's timeout tuple (an earlier edit dropped it, which would have been a NameError only in soak mode) and right-size the leg ceilings under the notebook timeout 07:47 pin 850729d3 (review round 3) 07:47 pin 850729d3 (review round 3) 07:47 pin 850729d3 (review round 3) 07:46 phase4_session: repin the trainer (rank-visible eval failure, cross-rank peak memory)