Download docs/MP_OPD.md from codemaivanngu/ctopd: direct link, hf CLI and curl.
- Browser
- Download file 8.71 kB
-
https://huggingface.co/codemaivanngu/ctopd/resolve/main/docs/MP_OPD.md
- Command line
-
hf download hf://codemaivanngu/ctopd/docs/MP_OPD.md
-
curl -L -o MP_OPD.md https://huggingface.co/codemaivanngu/ctopd/resolve/main/docs/MP_OPD.md
Isolated MP-OPD port: Atomic, Fixed, Random, GRASS-Chunk and ALIGN
These are experimental on-policy modes, ported from Tung's SimCT/MP-OPD checkout.
They are not aliases for Strict full/top-16, and their results must not be reported
as reproduction of the CTOPD paper's Strict objective or the full SimCT recipe.
Models, data, teacher warm start, optimizer, rollout and evaluation still come from
the CTOPD launcher. The initial adapter requires one student rank, ring size 1,
kd_ratio=1, kd_temperature=1, and kd_loss_fn=rkl.
Available presets
tools/launch_single_b200.py --algorithm |
Registered algorithm | Intervention |
|---|---|---|
mp-atomic |
mp_opd_atomic |
Original atom credits |
mp-fixed1 through mp-fixed5 |
mp_opd_fixed |
Hard pooling over 1 through 5 atoms |
mp-random1-2 |
mp_opd_random |
Hard pooling, group length sampled uniformly from [1,2] |
mp-random2-5 |
mp_opd_random |
Hard pooling, group length sampled uniformly from [2,5] |
mp-grass-chunk |
mp_opd_grass_chunk |
Risk-adaptive shrinkage within supplied chunks; no DP |
mp-align |
mp_opd_align |
Donor ALIGN v1 cyclic coefficient-space PCGrad within the same chunks |
mp-align-new |
mp_opd_align_new |
Local ALIGN plus full-response head-descent guard; see ALIGN_NEW.md |
mp-trust-r |
mp_opd_trust_r |
Strict plus mismatch scaled once per response |
mp-trust-b |
mp_opd_trust_b |
Strict plus mismatch scaled once per complete optimizer batch |
mp-trust-p |
mp_opd_trust_p |
Project opposing mismatch component through a complete-batch strict scale |
See TRUST_PAPER.md for the two-pass B/P actor contract and the
separate simct_paper score reconstruction. TRUST uses strict/mismatch atom types,
not the xtoken chunk partition used by ALIGN/GRASS.
Fixed/random groups tile each response independently. The final tail may be shorter than the requested length/minimum. Randomness uses the configured seed and response token IDs, independent of microbatch row/rank and global Python RNG. This intentionally improves grouping stability over the donor's row-index seed; it is a documented integration difference, not identical donor RNG replay.
Run and dry-run
From this checkout, print the full workload without importing ML libraries, creating a run, or using a GPU:
python3 tools/launch_single_b200.py --algorithm mp-align \
--runtime /opt/ctopd-b200-u22 --dry-run
python3 tools/launch_single_b200.py --algorithm mp-grass-chunk \
--runtime /opt/ctopd-b200-u22 --dry-run
python3 tools/launch_single_b200.py --algorithm mp-fixed5 --dry-run
python3 tools/launch_single_b200.py --algorithm mp-random2-5 --dry-run
Remove --dry-run only in the actual allocated B200 environment, exposing the one
GPU assigned to the run through CUDA_VISIBLE_DEVICES. Models/data must already
exist at the launcher's paths. Each run needs a fresh run ID. These presets retain
512 rollout/optimizer responses, max generation 8192, 100 iterations, and step-0
plus every-10-step MATH500 evaluation with 32 responses/problem. Defaults for the
two Strict presets remain unchanged. Evaluation is still MATH500, not the full
paper benchmark suite.
GRASS-Chunk and ALIGN default to --mp-chunk-source xtoken, using the same
vendored tokenizer/string aligner as the donor. This use does not need a learned
vocabulary projection. --mp-chunk-source run --mp-chunk-run-length 2 explicitly
chooses the fixed-run ablation instead. --mp-geometry diag is a separate diagonal
geometry ablation; the default is exact_head. ALIGN here is v1 align, not
align_new or a full-Transformer/global-descent guard. The separately ported
mp-align-new has a full-response head guard; it still does not measure the
full Transformer gradient. The donor does not name these modes v2/v3.
Scientific and numerical contract
Each response is byte-checked and partitioned into the donor's minimal synchronized atoms. Terminal EOS is excluded from this mode family's credits (Strict's EOS handling is unchanged). Atom credit is the detached teacher-minus-current-student log-score, divided by student token count. The student loss is detached effective credit times differentiable atom NLL, divided by the existing shared optimizer-window token normalizer. It is a sampled path-credit surrogate and may be negative; it is not the directly enumerated shared-vocabulary KL used by Strict.
GRASS reuses the corrected donor span-wide distortion, noise estimator and exact
LM-head Gram. The noise estimator updates per response and excludes adjacency
across chunk boundaries. Weighted signed credit is checked after dtype conversion;
evaluation uses the stored noise estimate without updating it. An invalid result
fails rather than silently switching to Atomic. ALIGN does not
claim credit conservation: its intervention is deterministic direction-conflict
projection. All geometry is detached. Geometry flags restore process-global FP32
settings; disabling MP_OPD_GRASS_EXACT_FP32 is rejected for these two modes.
The adapter applies the configured student/teacher final-logit softcap to raw
linear heads. Token tiles use checkpointed log-prob computation; changing
chunked_loss_size does not set research chunk boundaries. Exact-head geometry
still materializes per-response logits and an atom Gram, so long-response GPU
memory/throughput remain unqualified. No new wheels or native extension are added.
Complete responses with replacement_character, normalization_mismatch or
empty_response contribute zero. The adapter never sanitizes/relabels their text
or substitutes an Atomic loss. All other atomization failures remain fatal.
The shared optimizer-window token denominator is retained, including excluded
tokens; remaining responses are not reweighted per microbatch. Exclusion counts,
reason fractions and excluded token counts are logged under distill/mp_opd_*.
This policy changes the effective training sample distribution and can reduce
gradient magnitude; inspect exclusions before comparing results. It prevents
unsafe cross-text scoring but does not repair the source of retokenization mismatch.
See INVALID_RESPONSES.md for the policy and qualification.
Singleton alignment chunks
leave GRASS/ALIGN unchanged; inspect singleton fractions and coefficient-change
metrics before claiming an intervention occurred.
state_dict/load_state_dict expose algorithm state and reject a changed mode or
MP-OPD contract. They do not add full training resume: existing HF model exports do
not save optimizer/RNG/GRASS noise state. The launcher continues to refuse existing
run directories. Fresh-run comparisons only are supported by this integration.
Isolation, provenance and verification
kdflow/algorithms/mp_opd.py is the new adapter. Donor kernels live only in
kdflow/algorithms/_mp_opd_port/ and are imported lazily after selecting an MP mode.
The three existing algorithm files, byte alignment, RKL and tiled loss are
byte-identical to the pre-port checkpoint. Shared edits only register names,
validate the new flags, allow them through the on-policy CLI, attach teacher
softcap metadata for MP modes, and provide launcher presets.
MP_OPD_SOURCE.json records the donor commit, every source/ported file hash,
line-ending/import changes, and frozen Strict source hashes. The donor repository's MIT
notice and the NVIDIA aligner's Apache-2.0 notice/license are retained alongside
the vendored files. The donor checkout's unrelated staged diagnostic files were
not modified or committed here.
Run CPU verification using an existing compatible PyTorch environment; do not install dependencies merely to run this command:
TORCH_COMPILE_DISABLE=1 OMP_NUM_THREADS=1 python -m unittest discover -s tests -v
The tests cover donor Atomic loss/gradient parity, Fixed1 identity, fixed/random coverage, RNG independence, per-response/microbatch normalization, numerical tile invariance, GRASS identities/conservation, explicit-autograd head-Gram parity with softcapping, nontrivial ALIGN projection, native/run chunk backward, source hashes, and existing Strict loss/gradient regression. A Python environment without torch skips tensor tests explicitly; it is not a full verification pass.
The checked CPU environment and commands are recorded in MP_OPD_VALIDATION.json. CPU tests establish implementation checks only. B200 execution, FSDP runtime integration, full-size memory/throughput, training resume and scientific efficacy have not been qualified for the new modes. No training or GPU job is launched by the port or its tests.