ProprietaryLegal's picture
PLI Labs public snapshot 2026-10-28 (history squashed)
e1acc15
|
Raw History Blame Contribute Delete
6.79 kB

PP3 pre-registered coordinator A/B window — NOT RUN

Decision now: STOP. This is an execution plan after missing qualification gates close, not authorization to deploy. ESTIMATED total69minutes; hard ceiling75. No PP3 lane access to Spark/sparkworker, serving endpoints, production pair/units or any window directory is authorized or exercised. Coordinator owns every live operation and the exact restore receipt. No automatic promotion.

Admission gates

Freeze source commits/hashes and obtain independent review. Qualify the entire production K2 TP2 sampled path under its actual interpreter/Torch/Triton and weights; verify B1–B4 at depths4/5, T0 identity, q counts and conditioning, slot reuse, request permutation, cancel/reset, reconnect, timeout and late reply. Measure actual exposed added cost including eager/graph differences, Markov, candidate exchange or full-vocab peer staging, CRC/validation and target work. The present two-launch selector is .155–.209ms/depth5; it misses the absolute .05ms objective and is not a speed-qualified implementation.

Reconcile both pinned image sources with live source/image hashes. Prepare both overlays offline with prepare_verifier, review their diff/receipt, then use a separate coordinator-approved experimental read-only mount profile. Ordinary production staging intentionally does not deploy these files. Prove both ABI markers and engine failure invalidation; trial missing/corrupt q on both ranks must become target-only. Validate actual grammar/penalty/stop/EOS, tool JSON and reasoning parser state on the exact engine. Solo histograms and pure callback oracles do not establish those stateful properties.

Require a conservative cost estimate with positive margin at the family lower bounds and32K proxy; fill missing terms rather than pricing them at zero. Measure150K acceptance/cost separately in the bounded window. If any admission gate fails or evidence is incomplete, do not open the window. The current lane therefore hands over a NO-GO, plus this reproducible plan for a later revision.

Frozen arms and schedule

A: current production K2 greedy implementation, worker/engine sampled switches OFF, standard rejection, fixed depth5. B: same weights, backbone, Markov/cache, Engram backend, target processors and source build; both explicit sampled switches ON, top32/tau.7/dyadic24-floor-first-v1, same fixed depth5. Both disable confidence/adaptive admission, prompt lookup, dynamic depth and depth-file reload. A remains the existing implementation under the same fixed-depth experimental setting; do not compare adaptive A against fixed5 B. If depth5 is unsupported, STOP and register a new fixed4 experiment for both arms.

Keep target T1/top_p1, high reasoning effort, priority10, output budget and context/KV profile identical. B1 is primary, short/32K/150K occupied contexts with reasoning/output reserve. Two frozen public task families each provide one short, one32K and one150K request (six pairs total). Hash input sources, request/seed/target policy and ordered family assignment before any A/B data; no use of the development/PP2 fit examples and no prompt/output text in git. Stochastic texts need not match. T0 uses paired identical prompts/seeds; compare emitted tokens, not undefined padding past num_sampled.

Interleave A1/B1/B2/A2 with a separate engine session for each arm block. Each block runs all six frozen requests in the same predeclared rotating order with matched seeds/budgets. This uses real restarts with recorded cold/warm TTFT; changing environment variables in a running engine is not an arm switch. Do not claim per-round randomized interleaving. Begin every block only after its worker/engine identity, real warmup rounds and effective target parameters are attested; no fabricated switch or launcher flags.

ESTIMATED budget69minutes:

Stage Minutes Stop condition
Drain/ownership, exact snapshots and rollback preflight 7 No idle/restore ownership receipt: abort
A1/B1/B2/A2, each start+warm4, prefix priming4, six bounded decode runs3 44 Any block over11min: truncate the experiment, preserve raw partials
Paired greedy and failure/structured smoke within qualified test processes 3 Any semantic mismatch: reject B
Restore exact baseline and verify usable service 12 Use existing coordinator rollback escalation
Receipt/statistics/close 3 Incomplete data: no promotion

Decode runs are at most30s each after prefix priming; delivered tokens and complete target rounds are counted without requiring a fixed token count to finish after a deadline. Record any truncated full-task observation as censored. At elapsed50min stop admitting experiment requests and begin restore if any stage has overrun; preserve12min for restore. Hard75min includes restoration, not an excuse to leave B running. Longer full-task/extra-family/B4 work requires a separately registered window; do not expand this window opportunistically.

Endpoints, measurements and frozen decision

Only coordinator-reviewed experimental endpoints/processes are used, after production drain/ownership and HOLD checks from COMMON. Exact endpoint and launcher commands come from the coordinator's live registry; this plan invents none. Prefix priming is identical across arm blocks, with cold start/prefill reported separately and exact occupied token counts verified. No new context limit, GPU power/clock or concurrency setting is permitted.

Collect per-request/source-family/block and per-round IDs: actual delivered tokens, accepted drafts/round, complete ms/round p50/p95/p99, target verifier and q costs, TTFT, first visible answer, answer/reasoning counts, bytes, watchdogs, queue/staging and both-rank synchronization. Include canceled/failed/timeout requests in the disposition; never drop slow samples. Save unprofiled headline rates separately from CUDA-event profiling. Check memory/serving health and request backlog throughout; abort on unexplained failures/pressure.

Pre-registered rule: REJECT for any q/target-law, greedy-identity, grammar, stop, lifecycle, corruption or recovery failure, any lost request, or target policy/weight mismatch. NO PROMOTION / INCONCLUSIVE if incomplete, measured throughput gain<5%, source-cluster paired90% bootstrap lower bound<=0, or>5% TTFT/p95-round regression in any context/family. Bootstrap2000 source-cluster resamples with seed4103; report both task families and the small-family limit, never treat thousands of rounds as independent sources. Report accepted/round as a diagnostic; it cannot override delivered throughput or TTFT. Candidate only if every gate passes, point gain>=5%, lower bound>0 and per-context guards pass. Even then, independent coordinator approval and promotion are separate.