codemaivanngu commited on
Commit
2bf0f47
·
unverified ·
1 Parent(s): 0eeed4c

feat(h1b): add span factorial arms and offline probability replay tools

Browse files
README.md CHANGED
@@ -32,6 +32,11 @@ Separate [TRUST-R/B/P and SimCT paper-score modes](docs/TRUST_PAPER.md) provide
32
  optimizer window; `simct_paper` corrects log-probability scores while `simct` stays
33
  the released author-code baseline.
34
 
 
 
 
 
 
35
  ## Original project documentation
36
 
37
  This repository provides the implementation and example training scripts for `byte_align_ctkd` and `topk_byte_align_ctkd`.
 
32
  optimizer window; `simct_paper` corrects log-probability scores while `simct` stays
33
  the released author-code baseline.
34
 
35
+ An opt-in [H1b span experiment](docs/H1B.md) adds five factorial presets,
36
+ `h1b-a`, `h1b-b`, `h1b-b-bl`, `h1b-c-str`, `h1b-c`, plus offline trunk/exact
37
+ scoring and gradient replay. Real-input and B200 gates remain separate from CPU
38
+ checks; see the contract's approximation and baseline caveats before training.
39
+
40
  ## Original project documentation
41
 
42
  This repository provides the implementation and example training scripts for `byte_align_ctkd` and `topk_byte_align_ctkd`.
docs/H1B.md ADDED
@@ -0,0 +1,220 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # H1b: path versus trunk span supervision
2
+
3
+ Implementation inputs: Tung's `H1b Implementation Plan Bias vs Variance in Span
4
+ Supervision.md` (19 October 2026), and the companion Experiment Design (18 October
5
+ 2026). The implementation plan takes precedence where the two differ. Baseline
6
+ source: `0eeed4ca2b0d7b07dd50076e35d81f030a3b9f2d`.
7
+
8
+ ## Contract
9
+
10
+ `h1b_span` subclasses ByteAlignCrossTokenizerKD, with five isolated launcher
11
+ presets. The shared Strict files, byte codec and loss kernels remain byte-identical
12
+ to the baseline; see `H1B_SOURCE.json`. Existing presets keep their defaults.
13
+
14
+ | Preset | Arm | Detached span coefficient multiplying log q_student_path |
15
+ |---|---|---|
16
+ | h1b-a | a | zero |
17
+ | h1b-b | b | 2 lambda delta_path |
18
+ | h1b-b-bl | b_bl | 2 lambda (delta_path - previous_bucket_mean_path) |
19
+ | h1b-c-str | c_str | 2 lambda delta_trunk |
20
+ | h1b-c | c | 2 lambda (delta_trunk - previous_bucket_mean_trunk) |
21
+
22
+ Lambda is 0.5. Both strict and span sums use the existing common optimizer-window
23
+ token denominator. The baseline is frozen during each complete fit window, then
24
+ updated after the optimizer step. Evaluation/no-grad does not update it or window
25
+ statistics. State exports contain previous bucket means, arm/lambda and window
26
+ index, **not** a complete optimizer/RNG/training resume checkpoint.
27
+
28
+ The workload is Qwen2.5-7B-Instruct -> Llama-3.2-3B-Instruct, math_code_20k,
29
+ 512-response rollout/train batches, microbatch 4 with 128 accumulation microsteps,
30
+ 8,192 generated tokens, 100 updates, and MATH500 mean@32 every 10 updates. H1b
31
+ rollouts use top-p 1.0; MATH evaluation independently retains top-p 0.95. There is
32
+ one student rank with ring size 1, no final-logit softcap, and no multirank claim.
33
+ The optimizer and other flags are inherited unchanged from strict-full.
34
+
35
+ ## Byte groups and probability semantics
36
+
37
+ Groups use Strict's token-piece bytes and EOS segmentation. Equal complete bytes
38
+ are partitioned at shared cumulative byte ends. Content 1:1 groups and paired EOS
39
+ feed the original overlap-vocabulary RKL/chunked_loss. Nonempty 1:N/N:1/N:M groups
40
+ feed the span term; EOS is excluded. Responses with byte/segment mismatch, empty
41
+ pieces or unsupported special IDs are excluded as a whole from both terms and
42
+ counted. Offsets and the original denominator include excluded tokens.
43
+
44
+ **Parity boundary:** arm a has tested loss/all student-gradient parity with Strict
45
+ on accepted fixture rows. Strict can still keep a paired EOS when a content segment
46
+ has mismatched bytes; H1b's planned whole-response exclusion intentionally removes
47
+ that EOS too. Empty-piece/segment rejection is also stricter. Universal equality
48
+ on rejected responses is not claimed. Real-response differences are reported by
49
+ the alignment validator rather than silently passed.
50
+
51
+ The trunk probability is the sum of disjoint events: follow the canonical path
52
+ for j tokens, then finish or overhang the chunk in one token. It includes the full
53
+ canonical path and is a lower bound on the conditional byte-prefix event. All
54
+ duplicate byte-equivalent token IDs are retained in covers and exact branches.
55
+ Content paths must spell the group bytes exactly. Full-vocabulary head logits are
56
+ tiled; only selected path logps retain gradients. Detached cover/overhang masses
57
+ are reduced inside checkpointed student tiles; teacher scoring is no-grad.
58
+
59
+ This does **not** establish unbiased string-KL training. Remaining confounds are:
60
+
61
+ - Canonical prefix conditioning and outcome-dependent synchronization chunks.
62
+ - Re-encoding decoded rollout text instead of preserving the sampled token path;
63
+ this is the implementation plan's choice, shared across arms. H1b logs the raw
64
+ response re-encoding match rate. Top-p 1.0 alone does not eliminate this issue.
65
+ - A baseline indexed by the sampled chunk's type/length is outcome-dependent even
66
+ when its means come from the previous window. Previous-window separation removes
67
+ same-sample mean leakage, not all possible score-function bias.
68
+ - Whole-response exclusion changes the effective training distribution.
69
+
70
+ These are ablations of the specified estimators, not proof of the Design's
71
+ unbiasedness claims or a complete reproduction of Hou's unreleased span ablation.
72
+
73
+ ## Logging and replay provenance
74
+
75
+ Every arm computes both targets, both baselines and the same diagnostics. Window
76
+ moments weight chunks directly, not averages of microbatch means. JSON summaries
77
+ include all 24 buckets, counts and flags for undefined correlations/bias-floor
78
+ subsets. Scalar logs use finite zero plus counts/defined flags for empty subsets.
79
+ Per-chunk parquet uses null for absent overhang log-mass (log(0)); it never stores
80
+ raw prompts, only response hash, chunk hex and diagnostics.
81
+
82
+ Files live at `checkpoints/h1b/step_K.json` and `chunks_step_K.parquet`. K is the
83
+ zero-based pre-update model/window index: 100 updates produce K=0..99. Full dumps
84
+ are requested at 0/20/60/100; index 100 is an **offline final-checkpoint scoring**
85
+ operation, not a fictitious 101st training window.
86
+ The trainer's first training archive is `rollout_data/1.jsonl` (pre-update model
87
+ index 0); it does not write a training `rollout_data/0.jsonl`.
88
+ Sampling is deterministic from
89
+ window/response hash/span and does not advance the training RNG. H1b rollout JSONL
90
+ adds the teacher prompt and sampled output IDs so offline tokenization can reuse
91
+ the production method. Older archives lack that prompt; tools require explicit
92
+ `--assume-shared-prompt` if the caller has independently established equality.
93
+
94
+ Offline tools require existing local snapshots and `local_files_only=True`; they
95
+ never download models or regenerate responses. Reports include archive SHA-256
96
+ and model/tokenizer metadata hashes; these metadata hashes do not authenticate
97
+ every weight shard. Keep the model/checkpoint provenance alongside each report.
98
+
99
+ ## Commands (company node; existing runtime)
100
+
101
+ From the committed source root, with the allocated GPU exposed explicitly:
102
+
103
+ ```bash
104
+ export PYTHONPATH="$PWD"
105
+ RUNTIME=/opt/ctopd-b200-u22
106
+ ROOT=/workspace/storage-shared/nlp/tungks/ctopd_all
107
+ S="$ROOT/models/Llama-3.2-3B-Instruct"
108
+ T="$ROOT/models/Qwen2.5-7B-Instruct"
109
+ python3 tools/launch_single_b200.py --algorithm h1b-c --runtime "$RUNTIME" --dry-run
110
+ ```
111
+
112
+ If an old Strict archive lacks `tea_prompt`, recover it without regenerating any
113
+ responses. This uses the production PromptDataset template builder on the original
114
+ parquet pool and demands an exact, unambiguous student-prompt match. It preserves
115
+ finish reasons and never fabricates missing sampled IDs:
116
+
117
+ ```bash
118
+ bash "$RUNTIME/python-b200.sh" -m tools.h1b.enrich_archive \
119
+ --archive "$OLD_ARCHIVE" --dataset "$ROOT/data/Train/math_code_20k/train.parquet" \
120
+ --student "$S" --teacher "$T" --output "$ROOT/h1b-enriched.jsonl"
121
+ ARCHIVE="$ROOT/h1b-enriched.jsonl"
122
+ ```
123
+
124
+ The five-step canary retains the original launcher's initial MATH500 evaluation;
125
+ previewing a plan consumes no GPU, whereas executing it includes that evaluation.
126
+
127
+ T1/T3/T9 real-input validation (TEXTS must contain at least 10,000 texts, ARCHIVE
128
+ at least 500 real responses, with the production prompt/truncation metadata):
129
+
130
+ ```bash
131
+ bash "$RUNTIME/python-b200.sh" -m tools.h1b.validate_alignment \
132
+ --student "$S" --teacher "$T" --texts "$TEXTS" --archive "$ARCHIVE" \
133
+ --output "$ROOT/h1b-alignment.json"
134
+ ```
135
+
136
+ M2 trunk table, followed by M3 bounded exhaustive checking on 200 responses:
137
+
138
+ ```bash
139
+ bash "$RUNTIME/python-b200.sh" -m tools.h1b.exact_strprob \
140
+ --student "$S" --teacher "$T" --archive "$ARCHIVE" --limit 512 \
141
+ --trunk-only --output "$ROOT/h1b-trunk.jsonl"
142
+ bash "$RUNTIME/python-b200.sh" -m tools.h1b.exact_strprob \
143
+ --student "$S" --teacher "$T" --archive "$ARCHIVE" --limit 200 \
144
+ --prune 1e-6 --max-nodes 10000 --output "$ROOT/h1b-exact.jsonl"
145
+ ```
146
+
147
+ The checker uses depth-first search with a branch-safe, depth-bounded KV cache
148
+ stack. Siblings get independent cache copies, including when an HF cache mutates
149
+ in place. `--full-prefix` selects an independent reference path; branches deeper
150
+ than `--cache-depth` also fall back to full-prefix forwards. It is not yet batched
151
+ across frontier nodes; real-model cache compatibility and B200 throughput/memory
152
+ remain unmeasured. The finite search preserves duplicates and records an upper bound on
153
+ all pruned/unexpanded probability; only zero omitted mass is called exact.
154
+ The conservative gate uses mean *upper-bound* log-probability error < 0.1 mean
155
+ absolute path/trunk delta in every populated bucket, for both models. Empty
156
+ buckets are `insufficient`, not PASS; a zero gap cannot pass a strict relative
157
+ threshold. Both gate policy and raw counts must be inspected before training.
158
+ Gate status is currently **NOT MEASURED**. Depth-2 extension is conditional on a
159
+ failed measured gate and is not implemented speculatively.
160
+
161
+ T10 has a separate diagnostic launcher so the standard 100-update presets remain
162
+ unchanged. Preview first; `--execute` actually consumes the allocated company GPU:
163
+
164
+ ```bash
165
+ python3 -m tools.h1b.canary --algorithm h1b-c --runtime "$RUNTIME" \
166
+ --run-id canary-h1b-c-s42 --root "$ROOT"
167
+ # Repeat for each of the five presets, adding --execute only after allocation.
168
+ ```
169
+
170
+ Gradient replay uses a fixed 4,096-response archive; it never creates that archive.
171
+ Start with 128 responses (two groups of 64) to measure memory/time and sketch error:
172
+
173
+ ```bash
174
+ bash "$RUNTIME/python-b200.sh" -m tools.h1b.grad_replay \
175
+ --student "$S" --teacher "$T" --archive "$REPLAY_ARCHIVE" \
176
+ --limit 128 --output "$ROOT/h1b-replay-probe"
177
+ bash "$RUNTIME/python-b200.sh" -m tools.h1b.validate_sketch \
178
+ --replay "$ROOT/h1b-replay-probe" --output "$ROOT/h1b-sketch-gate.json"
179
+ # After sizing, use --limit 4096, --buckets 0,3 and a fresh output directory.
180
+ bash "$RUNTIME/python-b200.sh" -m tools.h1b.fit_cb \
181
+ --replay "$REPLAY_DIR" --bootstrap 200 --output "$ROOT/h1b-curves.json"
182
+ ```
183
+
184
+ Replay reuses the algorithm's targets/gradients, with leave-group-out bucket
185
+ means. Seven full FP32 buffers are allocated by default (five terms plus two
186
+ disjoint strict-position controls), more than the plan's five-buffer estimate.
187
+ `--buckets 0,3` adds eight buffers and may exceed B200 memory; `--buffer-device cpu`
188
+ is an explicit alternative requiring host RAM and transfers. Exact vectors for
189
+ the first two groups can be very large on disk (`--exact-groups 0` disables them,
190
+ but then the real T11 gate is not measured). Raw gradients only are implemented;
191
+ optional Adam-preconditioned replay and optimizer-state exports are not enabled.
192
+ The qualified architecture contract requires zero dropout; checkpointed replay
193
+ does not accumulate the training logger/baseline.
194
+
195
+ Analysis reports c(B), strict-position control cosine, split-half agreement,
196
+ finite-pool SNR, norm ratios, group noise scale, fitted c_infinity/B_noise and
197
+ group-bootstrap intervals. Token-weighted pooling retains common batch
198
+ normalization across differently sized responses. At B=64, two 32-response halves
199
+ cannot be recovered from 64-response group sketches: split-half is null, not a
200
+ fabricated measurement. At the entire finite pool SNR can be undefined; it is
201
+ reported null. Boundary fits are marked not identified. This is not a causal
202
+ bias-versus-variance verdict or an automatic decision-table result.
203
+
204
+ ## Validation and milestone status
205
+
206
+ Run CPU tests only in an existing environment:
207
+
208
+ ```bash
209
+ TORCH_COMPILE_DISABLE=1 OMP_NUM_THREADS=1 python -m unittest discover -s tests -v
210
+ ```
211
+
212
+ See `H1B_VALIDATION.json` for the executed checks. Fixtures verify algorithmic
213
+ identities; they do not replace T1/T3/T9 real-tokenizer/500-response gates, T10
214
+ B200 five-preset canaries, or T11 on full model gradient groups.
215
+
216
+ M1/M2/M4/M5 **code** is implemented with the limitations above. No real trunk
217
+ table, M3 gate decision, B200 qualification, gradient curves, trained checkpoint,
218
+ new model/data transfer, or M6/M7 scientific result is claimed. No new GPU job was
219
+ launched for this implementation. Do not update checkouts used by active jobs;
220
+ use fresh source/run IDs and preserve the existing Strict/SimCT/MP runs.
docs/H1B_SOURCE.json ADDED
@@ -0,0 +1,121 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "input_plans": [
3
+ {
4
+ "name": "Implementation Plan",
5
+ "sha256": "d99aad05f79545931f051601414eaa8b736a871a226cb898f8d4ef1f3ee24d32"
6
+ },
7
+ {
8
+ "name": "Experiment Design",
9
+ "sha256": "b13831f4182dd87fa286851b7617144ec3edce030ed148428676cf00699cba58"
10
+ }
11
+ ],
12
+ "files": [
13
+ {
14
+ "path": "kdflow/h1b/__init__.py",
15
+ "sha256": "bb8fa0f3ccd318368d986568b5423b522b51100210b877b3844835b191f6279a"
16
+ },
17
+ {
18
+ "path": "kdflow/h1b/baseline.py",
19
+ "sha256": "a1b46847ae84906bdad4068a9a6bae3fca47b07748f72e20e0ac6a699d7c393d"
20
+ },
21
+ {
22
+ "path": "kdflow/h1b/buckets.py",
23
+ "sha256": "964745a54ad61fe3d7148efdabbd8af8ea6c193885db5fe5a8e72330d55a5c2f"
24
+ },
25
+ {
26
+ "path": "kdflow/h1b/chunklog.py",
27
+ "sha256": "37e53384993b3d19fe533e58f873953a0c04cc8c730a6ee5ed1ef07b954c778b"
28
+ },
29
+ {
30
+ "path": "kdflow/h1b/exact.py",
31
+ "sha256": "b97eabf9a5b6f6435311c6d3b0321bfe63723b2fd23299684078f1d08dec0f4b"
32
+ },
33
+ {
34
+ "path": "kdflow/h1b/groups.py",
35
+ "sha256": "129788526be7c11d6a4493b25ec6ab5843dc090981a5342c482ff68b7c16f20a"
36
+ },
37
+ {
38
+ "path": "kdflow/h1b/strprob.py",
39
+ "sha256": "ea7a893c15fcaa96046293af1b37a50fa663b47b13e1bd20a0fbb9bbf49ce736"
40
+ },
41
+ {
42
+ "path": "kdflow/h1b/vocab_index.py",
43
+ "sha256": "55ac8c76ced13aa98ead01375de41906be9764276569af25908683d6a5a3abf3"
44
+ },
45
+ {
46
+ "path": "tools/h1b/__init__.py",
47
+ "sha256": "bbc4b569caaf873fc370d1dbe601de3b41ea107725c328997cb6534c0cc93f4a"
48
+ },
49
+ {
50
+ "path": "tools/h1b/canary.py",
51
+ "sha256": "f106d45d41024386df56e8ccc8da74c56247edfeb9a2c35879f9fe8712512e93"
52
+ },
53
+ {
54
+ "path": "tools/h1b/common.py",
55
+ "sha256": "8adf680d9e747e40f2781613a2b53b77f44b58bfe58ac0606c08432de4e05d06"
56
+ },
57
+ {
58
+ "path": "tools/h1b/enrich_archive.py",
59
+ "sha256": "db21f7bdd5d038b5cb8fc9888bd70a7e13265107edd8bf34fe32f0d6475eb770"
60
+ },
61
+ {
62
+ "path": "tools/h1b/exact_strprob.py",
63
+ "sha256": "99b9630d4c6b7cc6d5685cd47014a5dfcb80fce0ae6f133a529a9799ce25be35"
64
+ },
65
+ {
66
+ "path": "tools/h1b/fit_cb.py",
67
+ "sha256": "077f8e0c832110052711cc81e71f3b71cc74c606197f0bbc9049ea021a21247b"
68
+ },
69
+ {
70
+ "path": "tools/h1b/grad_replay.py",
71
+ "sha256": "a1af8a33d105c448ee5a79e69ad7bf9b5b077757578dc28f61b4fc3e5c6e33fb"
72
+ },
73
+ {
74
+ "path": "tools/h1b/sketch.py",
75
+ "sha256": "69b6093ffd3a096686aaa47936bb0b8785426ca8ef88ccc58e0daf77aa214186"
76
+ },
77
+ {
78
+ "path": "tools/h1b/validate_alignment.py",
79
+ "sha256": "39bcfd71736f5f1ddf79bb4eee9abf85152d77e4b158d84cdc452fcc7217e3e5"
80
+ },
81
+ {
82
+ "path": "tools/h1b/validate_sketch.py",
83
+ "sha256": "84e47110eda3f7b1599fdeef4f1be0cd4bf4eb2bd5e78ac974f619f000c01de1"
84
+ },
85
+ {
86
+ "path": "kdflow/algorithms/h1b_span.py",
87
+ "sha256": "8c332d91549ea7e549b2f771d16b7b31dcc263cf3d77000bfe4d5eb941489acb"
88
+ },
89
+ {
90
+ "path": "tests/test_h1b.py",
91
+ "sha256": "61e61fff4bdf4a8b1dea1c6e9fd97974a34d75b06a1516ce562dfe971b9a2c40"
92
+ }
93
+ ],
94
+ "frozen_strict": [
95
+ {
96
+ "sha256": "fdbd99c55aeac4d061527158bdfc82a93333a42bf347eceaa134d34f5ef05952",
97
+ "path": "kdflow/algorithms/simple_ctkd.py"
98
+ },
99
+ {
100
+ "sha256": "5c6647c6fb8064ffd5f7b1ed971168cd1f3392caee62e2d75916d41931828e18",
101
+ "path": "kdflow/algorithms/byte_align_ctkd.py"
102
+ },
103
+ {
104
+ "sha256": "298925da4ce973b0f2917e5ba5ad2adbc4862d7c1a73515c5c86514ac0d22ddb",
105
+ "path": "kdflow/algorithms/topk_byte_align_ctkd.py"
106
+ },
107
+ {
108
+ "sha256": "c77cd962cb0430cd97ace5312077e847398a03da3ce1473062fd2c5c9808498a",
109
+ "path": "kdflow/token_alignment.py"
110
+ },
111
+ {
112
+ "sha256": "abca69fdbe4f7d90a7ca990300189147230cce8f20d589c934d88f0b63c4edf6",
113
+ "path": "kdflow/loss/reverse_kl_div.py"
114
+ },
115
+ {
116
+ "sha256": "151c81614f97b908465a952f595abe1f4dc5c0d17cea86431a7068c381125cc5",
117
+ "path": "kdflow/loss/chunked_loss.py"
118
+ }
119
+ ],
120
+ "baseline_commit": "0eeed4ca2b0d7b07dd50076e35d81f030a3b9f2d"
121
+ }
docs/H1B_VALIDATION.json ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "date": "2026-10-19",
3
+ "baseline_commit": "0eeed4ca2b0d7b07dd50076e35d81f030a3b9f2d",
4
+ "platform": "WSL Linux, existing CPU environment; no package installation",
5
+ "python": "3.12.3",
6
+ "torch": "2.14.0+cpu",
7
+ "full_suite": {
8
+ "command": "TORCH_COMPILE_DISABLE=1 OMP_NUM_THREADS=1 python -m unittest discover -s tests -v",
9
+ "passed": 104,
10
+ "skipped": 0,
11
+ "elapsed_seconds": 72.46
12
+ },
13
+ "verified": [
14
+ "2000 cover-prefix/brute-force queries, duplicate byte IDs and 0xff bounds",
15
+ "Strict-group fixture parity including EOS and flattened response offsets",
16
+ "30-token toy LM on 50 strings: one-exit enumeration equals trunk; exhaustive probability >= trunk >= canonical path",
17
+ "Exact search duplicate branches, pruning/node-budget omitted-mass bounds",
18
+ "Same-tokenizer fixture: all five arms have identical loss and zero span term",
19
+ "Previous-window baseline, state restore and no evaluation/no-grad mutation",
20
+ "Arm b student gradients equal autograd span MSE; arm a loss/all student gradients equal Strict on accepted fixtures",
21
+ "Tile/checkpoint gradient invariance; whole-response mismatch exclusion and connected zero backward",
22
+ "Actual actor fit/evaluate bodies with CPU SGD: window hook after one optimizer update and no hook during evaluation",
23
+ "Actual rollout tokenizer method: EOS policy and raw re-encoding match field",
24
+ "Actual math evaluation sampling expression: independent eval top-p and unchanged rollout kwargs",
25
+ "Window moments weight chunks directly rather than microbatch means",
26
+ "CountSketch chunk invariance, fixture norm error <1 percent and cosine error <0.01",
27
+ "End-to-end saved sketch analysis CLI, known noise curve and undefined zero-vector handling",
28
+ "KV oracle versus full-prefix fixture with deliberately mutating cache and sibling branches",
29
+ "Five full-workload launcher dry-run plans; separate five-step diagnostic preview",
30
+ "New source hashes verified; six frozen Strict sources match baseline Git blobs",
31
+ "All existing Strict/SimCT/MP/ALIGN/TRUST regression tests",
32
+ "git diff --check",
33
+ "Old archive teacher-prompt recovery: exact unambiguous dataset/template match, no sampled-ID fabrication"
34
+ ],
35
+ "test_isolation": "CPU suite bypasses utility package tokenizer-factory initialization using a test-only namespace; real algorithm/loss/logging numerical source is executed",
36
+ "not_measured": [
37
+ "T1 on 10000 real texts with both real tokenizers",
38
+ "T3/T9 on 500 real archived responses",
39
+ "M2 real model trunk table and M3 exact/trunk gate at checkpoints 0 and 60",
40
+ "T10 five-preset B200/FSDP canaries and parquet validation",
41
+ "Real HF KV cache compatibility, head-tile throughput and memory",
42
+ "T11 on two full-model gradient groups; real replay curves",
43
+ "Full training, task accuracy, bias-versus-variance decision or paper reproduction"
44
+ ],
45
+ "deliberately_not_implemented": [
46
+ "Batched KV frontier expansion; current checker uses depth-bounded DFS cache with full-prefix fallback",
47
+ "Conditional depth-2 training estimator before a failed measured trunk gate",
48
+ "Optional Adam-preconditioned replay and optimizer-state exports",
49
+ "Fresh 4096-response generation; replay requires a supplied fixed archive",
50
+ "Automatic causal decision-table classification or optional c-plus experiments"
51
+ ],
52
+ "compute": "No new GPU job, model/dataset download, remote build or training launched",
53
+ "h1b_tests_in_full_suite": 21
54
+ }
kdflow/algorithms/__init__.py CHANGED
@@ -12,6 +12,7 @@ def register_algorithm(name):
12
 
13
  from kdflow.algorithms.simple_ctkd import SimpleCrossTokenizerKD
14
  from kdflow.algorithms.byte_align_ctkd import ByteAlignCrossTokenizerKD
 
15
  from kdflow.algorithms.topk_byte_align_ctkd import TopKByteAlignCrossTokenizerKD
16
  from kdflow.algorithms.simct import SimCT
17
  from kdflow.algorithms.simct_paper import SimCTPaper
 
12
 
13
  from kdflow.algorithms.simple_ctkd import SimpleCrossTokenizerKD
14
  from kdflow.algorithms.byte_align_ctkd import ByteAlignCrossTokenizerKD
15
+ from kdflow.algorithms.h1b_span import H1bSpanCTKD
16
  from kdflow.algorithms.topk_byte_align_ctkd import TopKByteAlignCrossTokenizerKD
17
  from kdflow.algorithms.simct import SimCT
18
  from kdflow.algorithms.simct_paper import SimCTPaper
kdflow/algorithms/h1b_span.py ADDED
@@ -0,0 +1,165 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """H1b Strict + detached path/string span coefficients, opt-in only."""
2
+ import hashlib
3
+ from dataclasses import replace
4
+ import math
5
+ import torch
6
+ from kdflow.algorithms import register_algorithm
7
+ from kdflow.algorithms.byte_align_ctkd import ByteAlignCrossTokenizerKD
8
+ from kdflow.loss.chunked_loss import chunked_loss
9
+ from kdflow.h1b.groups import align_byte_span_groups_batch
10
+ from kdflow.h1b.vocab_index import VocabIndex
11
+ from kdflow.h1b.strprob import score_hidden_chunks
12
+ from kdflow.h1b.baseline import BucketBaseline
13
+ from kdflow.h1b.buckets import bucket_of
14
+ from kdflow.h1b.chunklog import ChunkLogger
15
+
16
+
17
+ @register_algorithm('h1b_span')
18
+ class H1bSpanCTKD(ByteAlignCrossTokenizerKD):
19
+ def __init__(self,*args,**kwargs):
20
+ super().__init__(*args,**kwargs)
21
+ kd=self.args.kd
22
+ self.arm=kd.h1b_arm; self.lam=kd.h1b_span_lambda
23
+ if self.arm not in {'a','b','b_bl','c_str','c'} or not math.isfinite(self.lam) or self.lam<0:
24
+ raise ValueError('Invalid H1b arm or span lambda')
25
+ if kd.kd_ratio!=1 or kd.kd_temperature!=1 or kd.kd_loss_fn!='rkl':
26
+ raise ValueError('H1b requires kd_ratio=1, temperature=1 and rkl')
27
+ if (self.args.train.num_nodes*self.args.train.num_gpus_per_node!=1
28
+ or self.args.model.ring_attn_size!=1):
29
+ raise ValueError('H1b supports one student rank and ring size 1')
30
+ config=getattr(self.student.model,'config',None)
31
+ config=getattr(config,'text_config',None) or config
32
+ if getattr(config,'final_logit_softcapping',None) or getattr(self.teacher_lm_head,'_mp_opd_softcap',None):
33
+ raise ValueError('H1b softcapped heads are not qualified')
34
+ self.svi=VocabIndex(self.student_tokenizer,self.student_byte_codec)
35
+ self.tvi=VocabIndex(self.teacher_tokenizer,self.teacher_byte_codec)
36
+ self.baseline=BucketBaseline()
37
+ self.window=0
38
+ self.chunk_logger=ChunkLogger(getattr(self.args.train,'save_path',None),
39
+ kd.h1b_log_sample_rate,tuple(int(s) for s in kd.h1b_dump_every.split(',') if s.strip()))
40
+
41
+ def validate_optimizer_window(self,batches):
42
+ if self.strategy.step!=0 or len(batches)!=self.strategy.accumulated_gradient:
43
+ raise ValueError('H1b requires one complete optimizer window per fit')
44
+
45
+ def state_dict(self):
46
+ return dict(version=1,arm=self.arm,span_lambda=self.lam,window=self.window,baseline=self.baseline.state_dict())
47
+
48
+ def load_state_dict(self,state):
49
+ if state['version']!=1 or state['arm']!=self.arm or state['span_lambda']!=self.lam:
50
+ raise ValueError('Changed H1b state contract')
51
+ self.baseline.load_state_dict(state['baseline']); self.window=int(state['window'])
52
+ self.chunk_logger.reset(); self.chunk_logger.step=self.window
53
+
54
+ def on_optimizer_window_end(self):
55
+ if self.strategy.step!=0: raise ValueError('Cannot close partial H1b optimizer window')
56
+ metrics=self.chunk_logger.end_window(self.window)
57
+ self.baseline.end_window(); self.window+=1
58
+ return metrics
59
+
60
+ def training_step(self,batch):
61
+ replay=getattr(self,'replay_enabled',False)
62
+ # Offline replay intentionally retains graphs; normal training must not
63
+ # keep the previous microbatch graph alive during the next forward.
64
+ self.replay_terms={}; self.replay_records=[]
65
+ sm,tm=batch['stu_loss_mask'].bool(),batch['tea_loss_mask'].bool()
66
+ sy,ty=batch['stu_input_ids'].roll(-1,1),batch['tea_input_ids'].roll(-1,1)
67
+ st=[self.student_tokenizer.convert_ids_to_tokens(sy[i][sm[i]].cpu().tolist()) for i in range(len(sm))]
68
+ tt=[self.teacher_tokenizer.convert_ids_to_tokens(ty[i][tm[i]].cpu().tolist()) for i in range(len(tm))]
69
+ rows=align_byte_span_groups_batch(tt,st,self.teacher_tokenizer.eos_token,self.student_tokenizer.eos_token,
70
+ self.teacher_byte_codec.piece_to_bytes,self.student_byte_codec.piece_to_bytes)
71
+ for i,row in enumerate(rows):
72
+ sids=sy[i][sm[i]].cpu().tolist(); tids=ty[i][tm[i]].cpu().tolist()
73
+ if (any(t in self.svi.special and t!=self.student_tokenizer.eos_token_id for t in sids)
74
+ or any(t in self.tvi.special and t!=self.teacher_tokenizer.eos_token_id for t in tids)):
75
+ rows[i]=replace(row,groups=(),valid=False,reason='unsupported_special')
76
+ sh=self.student(batch['stu_input_ids'],attention_mask=batch['stu_attn_mask'],allgather_logits=True,
77
+ ring_attn_group=self.strategy.ring_attn_group,**(batch.get('stu_multi_modal_inputs') or {}))['hidden_states'][-1][sm]
78
+ th=batch['teacher_hiddens']
79
+ if isinstance(th,list): th=torch.cat(th,0)
80
+ if len(th)!=int(tm.sum()): raise ValueError('H1b teacher hidden/mask mismatch')
81
+ th=th.detach().to(self.teacher_lm_head.weight)
82
+ sl,tl=sy[sm],ty[tm]
83
+ groups=[g for row in rows for g in row.groups]
84
+ strict=[g for g in groups if g.kind=='strict']
85
+ mis=[g for g in groups if g.kind=='mis']
86
+ si=[g.s_idx[0] for g in strict]; ti=[g.t_idx[0] for g in strict]
87
+ size=self.args.train.chunked_loss_size or max(len(sh),1)
88
+ denom=float(batch['avg_micro_batch_token_num'])
89
+ if not math.isfinite(denom) or denom<=0: raise ValueError('Positive finite common normalizer required')
90
+ zero=sh.sum()*0.
91
+ strict_sum=zero; metric_sums={}
92
+ if si:
93
+ ash,ath=sh[si],th[ti]
94
+ def slog(h,skip=False): return self.student.model.lm_head(h,skip=skip)[:,self.student_overlap_token_ids]
95
+ def tlog(a,b):
96
+ with torch.no_grad():
97
+ return self.teacher_lm_head(ath[a:b])[:,self.teacher_overlap_token_ids]
98
+ strict_sum,metric_sums=chunked_loss(ash,self.student.model.lm_head,self.loss_fn,
99
+ student_logits_fn=slog,teacher_logits_fn=tlog,chunk_size=size,reduction='sum',
100
+ metric_fns=self.metric_fns,return_metrics=True)
101
+ sprob=score_hidden_chunks(self.student.model.lm_head,sh,sl,[(g.s_idx,g.content) for g in mis],self.svi,size,student=True)
102
+ with torch.no_grad():
103
+ tprob=score_hidden_chunks(self.teacher_lm_head,th,tl,[(g.t_idx,g.content) for g in mis],self.tvi,size,student=False)
104
+ terms={k:zero for k in ('path','str','base_path','base_str')}
105
+ training=self.student.training and torch.is_grad_enabled()
106
+ log=self.chunk_logger
107
+ if training:
108
+ log.rows+=len(rows); log.dropped+=sum(not r.valid for r in rows)
109
+ log.byte_dropped+=sum(r.reason=='byte_mismatch' for r in rows)
110
+ log.stu+=sum(r.student_tokens for r in rows); log.tea+=sum(r.teacher_tokens for r in rows)
111
+ log.strict_stu+=sum(len(g.s_idx) for g in strict if not g.eos)
112
+ log.strict_tea+=sum(len(g.t_idx) for g in strict if not g.eos)
113
+ log.mis_stu+=sum(len(g.s_idx) for g in mis)
114
+ if 'stu_reencode_match' in batch:
115
+ values=torch.as_tensor(batch['stu_reencode_match']).detach()
116
+ log.reencode_sum+=float(values.sum()); log.reencode_n+=values.numel()
117
+ # Response identity excludes prompt text and does not retain raw content.
118
+ response_hashes={i:hashlib.sha256(bytes(str((st[i],tt[i])),'utf8')).hexdigest() for i in range(len(rows))}
119
+ owner={id(g):i for i,r in enumerate(rows) for g in r.groups}
120
+ for g,s,t in zip(mis,sprob,tprob):
121
+ dp=(s.log_q_path-t.log_q_path).detach(); ds=(s.log_p_trunk-t.log_p_trunk).detach()
122
+ bucket=bucket_of(g.content)
123
+ bp=self.baseline.lookup(bucket,'path'); bs=self.baseline.lookup(bucket,'str')
124
+ if replay:
125
+ self.replay_records.append(dict(bucket=bucket,path=dp,str=ds,log_q=s.log_q_path))
126
+ for k,v in (('path',dp),('str',ds),('base_path',bp),('base_str',bs)):
127
+ terms[k]=terms[k]+2*self.lam*v*s.log_q_path
128
+ if training:
129
+ p,v=float(dp),float(ds)
130
+ coefficient=0. if self.arm=='a' else 2*self.lam*(p-bp if self.arm=='b_bl' else v-bs if self.arm=='c' else p if self.arm=='b' else v)
131
+ log.add(dict(response_hash=response_hashes[owner[id(g)]],step=self.window,bucket=bucket,
132
+ b0=g.b0,b1=g.b1,chunk_hex=g.content.hex(),s_tokens=len(g.s_idx),t_tokens=len(g.t_idx),
133
+ path=p,str=v,path_bl=p-bp,str_bl=v-bs,gap=p-v,baseline_path=bp,baseline_str=bs,coefficient=coefficient,
134
+ stu_log_q_path=float(s.log_q_path.detach()),tea_log_q_path=float(t.log_q_path),
135
+ stu_log_p_trunk=float(s.log_p_trunk),tea_log_p_trunk=float(t.log_p_trunk),
136
+ stu_log_overhang=float(s.log_overhang),tea_log_overhang=float(t.log_overhang),
137
+ slack_stu=float(s.log_p_trunk-s.log_q_path.detach()),slack_tea=float(t.log_p_trunk-t.log_q_path),
138
+ overhang_stu=float((s.log_overhang-s.log_p_trunk).exp()),overhang_tea=float((t.log_overhang-t.log_p_trunk).exp())))
139
+ self.baseline.accumulate(bucket,p,'path'); self.baseline.accumulate(bucket,v,'str')
140
+ span=zero if self.arm=='a' else (terms['path'] if self.arm=='b' else terms['path']-terms['base_path'] if self.arm=='b_bl'
141
+ else terms['str'] if self.arm=='c_str' else terms['str']-terms['base_str'])
142
+ loss=(strict_sum+span)/denom
143
+ if not bool(torch.isfinite(loss)): raise FloatingPointError('Nonfinite H1b loss')
144
+ if replay:
145
+ self.replay_terms={k:v/denom for k,v in terms.items()}
146
+ self.replay_terms['strict']=strict_sum/denom
147
+ if getattr(self,'replay_control',False):
148
+ for parity in (0,1):
149
+ x,y=sh[si[parity::2]],th[ti[parity::2]]
150
+ if len(x):
151
+ def replay_teacher(a,b,y=y):
152
+ with torch.no_grad():
153
+ return self.teacher_lm_head(y[a:b])[:,self.teacher_overlap_token_ids]
154
+ val=chunked_loss(x,self.student.model.lm_head,self.loss_fn,
155
+ student_logits_fn=slog,
156
+ teacher_logits_fn=replay_teacher,
157
+ chunk_size=size,reduction='sum')
158
+ else: val=zero
159
+ self.replay_terms['strict_'+str(parity)]=val/denom
160
+ info={'train/loss':loss,'train/kd_loss':loss,'train/h1b_span_loss':span/denom,
161
+ 'distill/alignment_ratio':torch.tensor(len(si)/max(len(sl),1)),
162
+ 'distill/h1b_mis_groups':loss.detach().new_tensor(float(len(mis))),
163
+ 'distill/h1b_dropped_responses':loss.detach().new_tensor(float(sum(not r.valid for r in rows)))}
164
+ info.update({k:v/denom for k,v in metric_sums.items()})
165
+ return info
kdflow/arguments/distillation_args.py CHANGED
@@ -23,7 +23,7 @@ class DistillationArguments:
23
  default="byte_align_ctkd",
24
  metadata={
25
  "help": "KD algorithm for each training step.",
26
- "choices": ["simple_ctkd", "byte_align_ctkd", "topk_byte_align_ctkd", "simct", "simct_paper", *MP_OPD_ALGORITHMS],
27
  }
28
  )
29
  kd_loss_fn: str = field(
@@ -132,18 +132,37 @@ class DistillationArguments:
132
  mp_opd_trust_eps_g: float = field(default=1e-12)
133
  mp_opd_trust_vocab_chunk: int = field(default=4096)
134
 
 
 
 
 
 
135
  def __post_init__(self):
136
  if self.multi_teacher_config is not None:
137
  raise ValueError(
138
  "This release supports one teacher model per run; "
139
  "use --teacher_name_or_path instead of --multi_teacher_config."
140
  )
141
- supported_algorithms = {"simple_ctkd", "byte_align_ctkd", "topk_byte_align_ctkd", "simct", "simct_paper", *MP_OPD_ALGORITHMS}
142
  if self.kd_algorithm not in supported_algorithms:
143
  raise ValueError(
144
  f"kd_algorithm must be one of {sorted(supported_algorithms)}, "
145
  f"got {self.kd_algorithm!r}."
146
  )
 
 
 
 
 
 
 
 
 
 
 
 
 
 
147
  if self.kd_algorithm in {'simct','simct_paper'}:
148
  if self.kd_ratio != 1 or self.kd_temperature != 1 or self.kd_loss_fn != 'rkl':
149
  raise ValueError('SimCT baseline requires kd_ratio=1, kd_temperature=1, kd_loss_fn=rkl')
 
23
  default="byte_align_ctkd",
24
  metadata={
25
  "help": "KD algorithm for each training step.",
26
+ "choices": ["simple_ctkd", "byte_align_ctkd", "topk_byte_align_ctkd", "simct", "simct_paper", "h1b_span", *MP_OPD_ALGORITHMS],
27
  }
28
  )
29
  kd_loss_fn: str = field(
 
132
  mp_opd_trust_eps_g: float = field(default=1e-12)
133
  mp_opd_trust_vocab_chunk: int = field(default=4096)
134
 
135
+ h1b_arm: str = field(default='a', metadata={'choices':['a','b','b_bl','c_str','c']})
136
+ h1b_span_lambda: float = field(default=0.5)
137
+ h1b_log_sample_rate: float = field(default=0.05)
138
+ h1b_dump_every: str = field(default='0,20,60,100')
139
+
140
  def __post_init__(self):
141
  if self.multi_teacher_config is not None:
142
  raise ValueError(
143
  "This release supports one teacher model per run; "
144
  "use --teacher_name_or_path instead of --multi_teacher_config."
145
  )
146
+ supported_algorithms = {"simple_ctkd", "byte_align_ctkd", "topk_byte_align_ctkd", "simct", "simct_paper", "h1b_span", *MP_OPD_ALGORITHMS}
147
  if self.kd_algorithm not in supported_algorithms:
148
  raise ValueError(
149
  f"kd_algorithm must be one of {sorted(supported_algorithms)}, "
150
  f"got {self.kd_algorithm!r}."
151
  )
152
+ if self.kd_algorithm == 'h1b_span':
153
+ if self.kd_ratio!=1 or self.kd_temperature!=1 or self.kd_loss_fn!='rkl':
154
+ raise ValueError('H1b requires kd_ratio=1, kd_temperature=1, kd_loss_fn=rkl')
155
+ if self.h1b_arm not in {'a','b','b_bl','c_str','c'}:
156
+ raise ValueError('Invalid H1b arm')
157
+ if not math.isfinite(self.h1b_span_lambda) or self.h1b_span_lambda<0:
158
+ raise ValueError('Invalid H1b span lambda')
159
+ if not math.isfinite(self.h1b_log_sample_rate) or not 0<=self.h1b_log_sample_rate<=1:
160
+ raise ValueError('Invalid H1b sample rate')
161
+ try:
162
+ steps=[int(s) for s in self.h1b_dump_every.split(',') if s.strip()]
163
+ except ValueError as error:
164
+ raise ValueError('H1b dump steps must be comma-separated integers') from error
165
+ if any(s<0 for s in steps): raise ValueError('Negative H1b dump step')
166
  if self.kd_algorithm in {'simct','simct_paper'}:
167
  if self.kd_ratio != 1 or self.kd_temperature != 1 or self.kd_loss_fn != 'rkl':
168
  raise ValueError('SimCT baseline requires kd_ratio=1, kd_temperature=1, kd_loss_fn=rkl')
kdflow/arguments/rollout_args.py CHANGED
@@ -5,6 +5,10 @@ from typing import Optional
5
  @dataclass
6
  class RolloutArguments:
7
  """ Arguments for rollout (on-policy distillation)."""
 
 
 
 
8
 
9
  rollout_num_engines: int = field(
10
  default=0,
@@ -30,6 +34,7 @@ class RolloutArguments:
30
  default=1.0,
31
  metadata={"help": "Top-p sampling for rollout."}
32
  )
 
33
  temperature: float = field(
34
  default=1.0,
35
  metadata={"help": "Temperature for rollout."}
 
5
  @dataclass
6
  class RolloutArguments:
7
  """ Arguments for rollout (on-policy distillation)."""
8
+
9
+ def __post_init__(self):
10
+ if self.eval_math_top_p is not None and not 0 < self.eval_math_top_p <= 1:
11
+ raise ValueError('eval_math_top_p must be in (0,1] or None')
12
 
13
  rollout_num_engines: int = field(
14
  default=0,
 
34
  default=1.0,
35
  metadata={"help": "Top-p sampling for rollout."}
36
  )
37
+ eval_math_top_p: float = field(default=None, metadata={'help':'MATH sampling top-p; None inherits rollout top_p.'})
38
  temperature: float = field(
39
  default=1.0,
40
  metadata={"help": "Temperature for rollout."}
kdflow/cli/train_kd_on_policy.py CHANGED
@@ -94,6 +94,7 @@ def train(args):
94
  if not tokenizer_info.vocab_identical and args.kd.kd_algorithm not in {
95
  "simple_ctkd",
96
  "byte_align_ctkd",
 
97
  "topk_byte_align_ctkd",
98
  "simct",
99
  "simct_paper",
@@ -101,7 +102,7 @@ def train(args):
101
  }:
102
  raise ValueError(
103
  "Cross-tokenizer distillation supports simple_ctkd, "
104
- "byte_align_ctkd, topk_byte_align_ctkd, simct, simct_paper, or an opt-in mp_opd_* mode."
105
  )
106
 
107
  environment_rollout = bool(args.rollout.alfworld_config)
 
94
  if not tokenizer_info.vocab_identical and args.kd.kd_algorithm not in {
95
  "simple_ctkd",
96
  "byte_align_ctkd",
97
+ "h1b_span",
98
  "topk_byte_align_ctkd",
99
  "simct",
100
  "simct_paper",
 
102
  }:
103
  raise ValueError(
104
  "Cross-tokenizer distillation supports simple_ctkd, "
105
+ "byte_align_ctkd, topk_byte_align_ctkd, simct, simct_paper, h1b_span, or an opt-in mp_opd_* mode."
106
  )
107
 
108
  environment_rollout = bool(args.rollout.alfworld_config)
kdflow/evaluation/math_evaluator.py CHANGED
@@ -130,10 +130,13 @@ class MathEvaluator:
130
  def evaluate(self, global_step: int) -> Dict[str, float]:
131
  eval_data = self._load_dataset()
132
  prompts, labels = self._build_prompts(eval_data)
 
133
  sampling_params = {
134
  "max_new_tokens": self.max_tokens,
135
  "temperature": self.generate_kwargs.get("temperature", 0.6),
136
- "top_p": self.generate_kwargs.get("top_p", 0.95),
 
 
137
  "n": self.n_samples,
138
  }
139
 
 
130
  def evaluate(self, global_step: int) -> Dict[str, float]:
131
  eval_data = self._load_dataset()
132
  prompts, labels = self._build_prompts(eval_data)
133
+ eval_top_p=getattr(getattr(self.args,'rollout',None),'eval_math_top_p',None)
134
  sampling_params = {
135
  "max_new_tokens": self.max_tokens,
136
  "temperature": self.generate_kwargs.get("temperature", 0.6),
137
+ "top_p": (eval_top_p
138
+ if eval_top_p is not None
139
+ else self.generate_kwargs.get("top_p", 0.95)),
140
  "n": self.n_samples,
141
  }
142
 
kdflow/h1b/__init__.py ADDED
@@ -0,0 +1 @@
 
 
1
+ """Opt-in H1b byte-span supervision; no eager model/runtime imports."""
kdflow/h1b/baseline.py ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Previous completed optimizer-window bucket means; no same-window leakage."""
2
+ import torch
3
+
4
+
5
+ class BucketBaseline:
6
+ def __init__(self):
7
+ self.prev=torch.zeros(2,24,dtype=torch.float64)
8
+ self.sums=torch.zeros_like(self.prev)
9
+ self.counts=torch.zeros_like(self.prev)
10
+
11
+ def lookup(self,bucket,which):
12
+ return float(self.prev[0 if which=='path' else 1,bucket])
13
+
14
+ def accumulate(self,bucket,delta,which):
15
+ i=0 if which=='path' else 1
16
+ value=float(delta)
17
+ if not torch.isfinite(torch.tensor(value)): raise FloatingPointError('Nonfinite bucket delta')
18
+ self.sums[i,bucket]+=value; self.counts[i,bucket]+=1
19
+
20
+ def end_window(self):
21
+ # H1b currently rejects multi-rank training; no collective is hidden here.
22
+ self.prev=torch.where(self.counts>0,self.sums/self.counts.clamp_min(1),self.prev)
23
+ self.sums.zero_(); self.counts.zero_()
24
+
25
+ def state_dict(self): return {'prev':self.prev.clone()}
26
+
27
+ def load_state_dict(self,state):
28
+ value=state['prev'].double().cpu()
29
+ if value.shape!=(2,24) or not bool(torch.isfinite(value).all()): raise ValueError('Bad H1b baseline state')
30
+ self.prev=value.clone(); self.sums.zero_(); self.counts.zero_()
kdflow/h1b/buckets.py ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Six byte classes times four length bins, stable bucket IDs 0..23."""
2
+ import string
3
+
4
+
5
+ def bucket_of(value):
6
+ if not value: raise ValueError('EOS/empty bytes are not mismatch buckets')
7
+ if all(48 <= b <= 57 for b in value): kind=0
8
+ elif value.isspace(): kind=1
9
+ elif any(b>127 for b in value): kind=2
10
+ elif all(65<=b<=90 or 97<=b<=122 for b in value): kind=3
11
+ elif all(chr(b) in string.punctuation for b in value): kind=4
12
+ else: kind=5
13
+ size=0 if len(value)<=2 else 1 if len(value)<=4 else 2 if len(value)<=8 else 3
14
+ return kind*4+size
kdflow/h1b/chunklog.py ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Streaming window moments and bounded, deterministic chunk sampling."""
2
+ import hashlib
3
+ import json
4
+ import math
5
+ from pathlib import Path
6
+
7
+
8
+ class Moments:
9
+ def __init__(self): self.n=0; self.s={}; self.ss={}; self.cross=0.; self.floor_n=0; self.floor_sum=0.
10
+ def add(self,row):
11
+ self.n+=1
12
+ for key in ('path','str','path_bl','str_bl','gap','slack_stu','slack_tea','overhang_stu','overhang_tea'):
13
+ v=row[key]; self.s[key]=self.s.get(key,0.)+v; self.ss[key]=self.ss.get(key,0.)+v*v
14
+ self.cross+=row['path']*row['str']
15
+ if abs(row['str'])<.05: self.floor_n+=1; self.floor_sum+=row['gap']
16
+ def summary(self):
17
+ mean={k:v/max(self.n,1) for k,v in self.s.items()}
18
+ var={k:max(0.,self.ss[k]/max(self.n,1)-mean[k]**2) for k in self.s}
19
+ denom=math.sqrt(var.get('path',0.)*var.get('str',0.))
20
+ corr=(self.cross/max(self.n,1)-mean.get('path',0.)*mean.get('str',0.))/denom if denom else 0.
21
+ return dict(count=self.n,mean=mean,variance=var,correlation=corr,
22
+ correlation_defined=bool(denom),bias_floor=self.floor_sum/max(self.floor_n,1),bias_floor_count=self.floor_n)
23
+
24
+
25
+ class ChunkLogger:
26
+ def __init__(self,save_path=None,sample_rate=.05,dump_steps=(0,20,60,100)):
27
+ self.root=Path(save_path)/'h1b' if save_path else None
28
+ self.sample_rate=sample_rate; self.dump_steps=set(dump_steps); self.step=0
29
+ self.reset()
30
+ def reset(self):
31
+ self.total=Moments(); self.buckets=[Moments() for _ in range(24)]
32
+ self.samples=[]; self.rows=0; self.dropped=0; self.byte_dropped=0
33
+ self.stu=self.tea=self.strict_stu=self.strict_tea=self.mis_stu=0
34
+ self.reencode_n=0; self.reencode_sum=0.
35
+ def add(self,row):
36
+ self.total.add(row); self.buckets[row['bucket']].add(row)
37
+ key=f"{self.step}:{row['response_hash']}:{row['b0']}:{row['b1']}"
38
+ u=int.from_bytes(hashlib.sha256(key.encode()).digest()[:8],'big')/2**64
39
+ if self.step in self.dump_steps or u<self.sample_rate:
40
+ # JSON/parquet store absent overhang as null, never -Infinity.
41
+ self.samples.append({k:(None if isinstance(v,float) and not math.isfinite(v) else v) for k,v in row.items()})
42
+ def end_window(self,step):
43
+ summary=self.total.summary()
44
+ buckets=[b.summary() for b in self.buckets]
45
+ mean=summary['mean']; var=summary['variance']
46
+ metrics={'strict_cov_stu':self.strict_stu/max(self.stu,1),'strict_cov_tea':self.strict_tea/max(self.tea,1),
47
+ 'mis_tok_frac':self.mis_stu/max(self.stu,1),'gap_mean':mean.get('gap',0.),
48
+ 'slack_tea':mean.get('slack_tea',0.),'slack_stu':mean.get('slack_stu',0.),
49
+ 'bias_floor':summary['bias_floor'],'bias_floor_count':summary['bias_floor_count'],
50
+ 'corr_path_str':summary['correlation'],'corr_path_str_defined':float(summary['correlation_defined']),
51
+ 'overhang_share_tea':mean.get('overhang_tea',0.),'overhang_share_stu':mean.get('overhang_stu',0.),
52
+ 'reencode_match':self.reencode_sum/max(self.reencode_n,1),'reencode_count':self.reencode_n,
53
+ 'dropped_byte_mismatch':self.byte_dropped,'window_responses':self.rows,'window_mis_groups':self.total.n}
54
+ for k in ('path','str','path_bl','str_bl'): metrics['var_'+k]=var.get(k,0.)
55
+ for k,b in enumerate(buckets):
56
+ metrics[f'gap_mean_b{k}']=b['mean'].get('gap',0.)
57
+ metrics[f'count_b{k}']=b['count']
58
+ if self.root:
59
+ self.root.mkdir(parents=True,exist_ok=True)
60
+ target=self.root/f'step_{step}.json'
61
+ with target.open('x') as f:
62
+ json.dump(dict(step=step,metrics=metrics,total=summary,buckets=buckets),f,allow_nan=False,indent=2)
63
+ import pyarrow as pa
64
+ import pyarrow.parquet as pq
65
+ table=pa.Table.from_pylist(self.samples) if self.samples else pa.table({'response_hash':pa.array([],type=pa.string())})
66
+ pq.write_table(table,self.root/f'chunks_step_{step}.parquet')
67
+ self.reset(); self.step=step+1
68
+ return {'distill/h1b_'+k:v for k,v in metrics.items()}
kdflow/h1b/exact.py ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Finite exhaustive cover-path enumeration with a certified omitted-mass bound."""
2
+ import math
3
+
4
+
5
+ def logadd(a,b):
6
+ if a == -math.inf: return b
7
+ if b == -math.inf: return a
8
+ m=max(a,b)
9
+ return m+math.log(math.exp(a-m)+math.exp(b-m))
10
+
11
+
12
+ def exact_chunk_logprob(next_logprobs,chunk,vi,prune=0.,max_nodes=10000):
13
+ """next_logprobs(tuple_of_branch_ids) -> normalized logp vector.
14
+
15
+ Empty/special token paths are outside this byte-cover event. Duplicate byte
16
+ token IDs are distinct branches. Returned bounds remain valid on pruning or
17
+ node-budget exhaustion; only zero omitted mass merits the label exact.
18
+ """
19
+ if not chunk or not 0<=prune<1 or max_nodes<1: raise ValueError('Bad exact-search contract')
20
+ frontier=[(0.,0,())]; total=omitted=-math.inf; nodes=0
21
+ while frontier:
22
+ lp,used,path=frontier.pop()
23
+ if nodes>=max_nodes:
24
+ omitted=logadd(omitted,lp)
25
+ continue
26
+ row=next_logprobs(path); nodes+=1
27
+ rem=chunk[used:]
28
+ cover=-math.inf
29
+ for token in vi.cover(rem): cover=logadd(cover,float(row[token]))
30
+ total=logadd(total,lp+cover)
31
+ for k in range(1,len(rem)):
32
+ for token in vi.ids_of(rem[:k]):
33
+ child=lp+float(row[token])
34
+ if prune and child<total+math.log(prune): omitted=logadd(omitted,child)
35
+ else: frontier.append((child,used+k,path+(token,)))
36
+ upper=logadd(total,omitted)
37
+ return dict(log_probability_lower=total,log_probability_upper=upper,
38
+ log_omitted_mass_upper=omitted,nodes=nodes,exact=omitted==-math.inf)
kdflow/h1b/groups.py ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Byte-piece groups using Strict's codecs and EOS segmentation."""
2
+ from dataclasses import dataclass
3
+ from itertools import zip_longest
4
+ from kdflow.token_alignment import _split_on_eos
5
+
6
+
7
+ @dataclass(frozen=True)
8
+ class Group:
9
+ kind: str
10
+ s_idx: tuple
11
+ t_idx: tuple
12
+ b0: int
13
+ b1: int
14
+ content: bytes
15
+ eos: bool = False
16
+
17
+
18
+ @dataclass(frozen=True)
19
+ class RowGroups:
20
+ groups: tuple
21
+ valid: bool
22
+ reason: str = ''
23
+ student_tokens: int = 0
24
+ teacher_tokens: int = 0
25
+
26
+
27
+ def align_byte_span_groups_batch(tea_tokens, stu_tokens, teacher_eos_token,
28
+ student_eos_token, teacher_piece_to_bytes,
29
+ student_piece_to_bytes):
30
+ if len(tea_tokens) != len(stu_tokens):
31
+ raise ValueError('H1b response counts differ')
32
+ rows, so, to = [], 0, 0
33
+ for ts, ss in zip(tea_tokens, stu_tokens):
34
+ groups, reason, sc, tc = [], '', 0, 0
35
+ tsegments, ssegments = _split_on_eos(ts, teacher_eos_token), _split_on_eos(ss, student_eos_token)
36
+ for titem, sitem in zip_longest(tsegments, ssegments):
37
+ if titem is None or sitem is None:
38
+ reason = 'eos_segment_mismatch'
39
+ break
40
+ tseg, teos = titem
41
+ sseg, seos = sitem
42
+ tp = [teacher_piece_to_bytes(p) for _, p in tseg]
43
+ sp = [student_piece_to_bytes(p) for _, p in sseg]
44
+ tc += len(tp)
45
+ sc += len(sp)
46
+ if b''.join(tp) != b''.join(sp):
47
+ reason = 'byte_mismatch'
48
+ break
49
+ if any(not p for p in tp + sp):
50
+ reason = 'empty_piece'
51
+ break
52
+ ti = si = t0 = s0 = b0 = 0
53
+ tend, send = [], []
54
+ for p in tp: tend.append((tend[-1] if tend else 0) + len(p))
55
+ for p in sp: send.append((send[-1] if send else 0) + len(p))
56
+ while ti < len(tp) and si < len(sp):
57
+ if tend[ti] < send[si]: ti += 1; continue
58
+ if tend[ti] > send[si]: si += 1; continue
59
+ tt = tuple(to + tseg[k][0] for k in range(t0, ti + 1))
60
+ st = tuple(so + sseg[k][0] for k in range(s0, si + 1))
61
+ groups.append(Group('strict' if len(tt) == len(st) == 1 else 'mis',
62
+ st, tt, b0, tend[ti], b''.join(tp[t0:ti+1])))
63
+ b0 = tend[ti]
64
+ ti += 1; si += 1; t0, s0 = ti, si
65
+ if teos is not None and seos is not None:
66
+ groups.append(Group('strict', (so+seos,), (to+teos,), b0, b0, b'', True))
67
+ # Content denominators count even a rejected response completely.
68
+ sc = sum(p != student_eos_token for p in ss)
69
+ tc = sum(p != teacher_eos_token for p in ts)
70
+ rows.append(RowGroups(tuple(groups) if not reason else (), not reason, reason, sc, tc))
71
+ so += len(ss); to += len(ts)
72
+ return rows
kdflow/h1b/strprob.py ADDED
@@ -0,0 +1,87 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Canonical path and one-exit trunk prefix probability (detached diagnostics)."""
2
+ from dataclasses import dataclass
3
+ import torch
4
+
5
+
6
+ @dataclass
7
+ class ChunkLP:
8
+ log_q_path: torch.Tensor
9
+ log_p_trunk: torch.Tensor
10
+ log_overhang: torch.Tensor
11
+
12
+
13
+ def chunk_logprobs(logp_rows, path_ids, chunk_bytes, vi):
14
+ if not chunk_bytes or not len(path_ids) or len(path_ids) != len(logp_rows):
15
+ raise ValueError('A chunk needs nonempty bytes and one prediction row per path token')
16
+ if b''.join(vi.bytes(i) for i in path_ids) != chunk_bytes:
17
+ raise ValueError('Canonical path does not spell chunk bytes')
18
+ terms, over, used = [], [], 0
19
+ acc = logp_rows.new_zeros(())
20
+ for row, token in zip(logp_rows, path_ids):
21
+ rem = chunk_bytes[used:]
22
+ ids = vi.cover_ids(rem, row.device)
23
+ terms.append(acc + torch.logsumexp(row[ids], 0))
24
+ oi = [i for i in vi.cover(rem) if len(vi.bytes(i)) > len(rem)]
25
+ if oi:
26
+ over.append(acc + torch.logsumexp(row[oi], 0))
27
+ acc = acc + row[int(token)]
28
+ used += len(vi.bytes(token))
29
+ trunk = torch.logsumexp(torch.stack(terms), 0)
30
+ if not bool(torch.isfinite(acc)) or not bool(torch.isfinite(trunk)):
31
+ raise FloatingPointError('Nonfinite H1b path/trunk probability')
32
+ if bool(trunk < acc - 2e-5): raise ArithmeticError('H1b trunk lower than canonical path')
33
+ return ChunkLP(acc, trunk, torch.logsumexp(torch.stack(over),0) if over else acc.new_tensor(-torch.inf))
34
+
35
+
36
+ def score_hidden_chunks(head, hidden, labels, chunks, vi, tile_size, *, student):
37
+ """Keep selected path gradients; reduce trunk tiles without retaining logits.
38
+
39
+ chunks is a list of (flat indices, bytes). Each checkpoint contains only a
40
+ tile, its selected logps and detached covering-event log-masses.
41
+ """
42
+ from torch.utils.checkpoint import checkpoint
43
+ if tile_size < 1: raise ValueError('Positive head tile size required')
44
+ paths, covers, overs = [], [], []
45
+ label_ids=labels.detach().cpu().tolist()
46
+ flat = []
47
+ for indices,content in chunks:
48
+ used = 0
49
+ ids = [label_ids[i] for i in indices]
50
+ if b''.join(vi.bytes(i) for i in ids) != content:
51
+ raise ValueError('H1b head path/byte mismatch')
52
+ for pos,tid in zip(indices,ids):
53
+ rem=content[used:]
54
+ cov=vi.cover(rem)
55
+ ov=tuple(i for i in cov if len(vi.bytes(i))>len(rem))
56
+ flat.append((pos,tid,cov,ov))
57
+ used += len(vi.bytes(tid))
58
+ for start in range(0,len(flat),tile_size):
59
+ records=flat[start:start+tile_size]
60
+ indices=torch.tensor([r[0] for r in records],device=hidden.device)
61
+ def tile(h, records=records):
62
+ logits=head(h,skip=False) if student else head(h)
63
+ lp=logits.float().log_softmax(-1)
64
+ q=lp[torch.arange(len(records),device=lp.device),[r[1] for r in records]]
65
+ with torch.no_grad():
66
+ c=torch.stack([torch.logsumexp(lp[j,list(r[2])],0) for j,r in enumerate(records)])
67
+ o=torch.stack([torch.logsumexp(lp[j,list(r[3])],0) if r[3] else lp.new_tensor(-torch.inf)
68
+ for j,r in enumerate(records)])
69
+ return q,c,o
70
+ with torch.set_grad_enabled(student and torch.is_grad_enabled()):
71
+ q,c,o=(checkpoint(tile,hidden[indices],use_reentrant=False)
72
+ if student and torch.is_grad_enabled() else tile(hidden[indices]))
73
+ paths.extend(q.unbind()); covers.extend(c.unbind()); overs.extend(o.unbind())
74
+ result=[]
75
+ offset=0
76
+ for indices,_ in chunks:
77
+ n=len(indices)
78
+ qp=torch.stack(paths[offset:offset+n])
79
+ prefix=torch.cat((qp.new_zeros(1),qp.detach().cumsum(0)[:-1]))
80
+ trunk=torch.logsumexp(prefix+torch.stack(covers[offset:offset+n]),0)
81
+ over=torch.logsumexp(prefix+torch.stack(overs[offset:offset+n]),0)
82
+ path=qp.sum()
83
+ if not bool(torch.isfinite(trunk)) or bool(trunk < path.detach()-2e-5):
84
+ raise FloatingPointError('Invalid H1b trunk probability')
85
+ result.append(ChunkLP(path,trunk.detach(),over.detach()))
86
+ offset+=n
87
+ return result
kdflow/h1b/vocab_index.py ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Duplicate-preserving byte-prefix vocabulary index."""
2
+ from bisect import bisect_left
3
+ from functools import lru_cache
4
+ from kdflow.token_alignment import get_tokenizer_byte_codec, _special_token_ids
5
+
6
+
7
+ def next_prefix(value):
8
+ for i in range(len(value)-1, -1, -1):
9
+ if value[i] < 255:
10
+ return value[:i] + bytes([value[i]+1])
11
+ return None
12
+
13
+
14
+ class VocabIndex:
15
+ def __init__(self, tok, codec=None):
16
+ codec = codec or get_tokenizer_byte_codec(tok)
17
+ self.special = _special_token_ids(tok)
18
+ vocab = tok.get_vocab()
19
+ self._bytes = {i: codec.piece_to_bytes(p) for p,i in vocab.items() if i not in self.special}
20
+ pairs = sorted((b,i) for i,b in self._bytes.items() if b)
21
+ self.keys, self.perm = [b for b,_ in pairs], [i for _,i in pairs]
22
+ self._ids = {}
23
+ for b,i in pairs: self._ids.setdefault(b, []).append(i)
24
+ self.duplicate_byte_strings = sum(len(v)>1 for v in self._ids.values())
25
+
26
+ def bytes(self, token_id):
27
+ return self._bytes[int(token_id)]
28
+
29
+ def ids_of(self, value):
30
+ return tuple(self._ids.get(value, ()))
31
+
32
+ def id_of(self, value):
33
+ ids = self.ids_of(value)
34
+ return ids[0] if ids else None
35
+
36
+ @lru_cache(maxsize=16384)
37
+ def cover(self, rem):
38
+ if not rem: raise ValueError('Empty remaining chunk is not a token event')
39
+ high = next_prefix(rem)
40
+ lo, hi = bisect_left(self.keys, rem), (len(self.keys) if high is None else bisect_left(self.keys, high))
41
+ return tuple(self.perm[lo:hi])
42
+
43
+ def cover_ids(self, rem, device=None):
44
+ import torch
45
+ return torch.tensor(self.cover(rem), dtype=torch.long, device=device)
kdflow/ray/train/student_actor.py CHANGED
@@ -239,7 +239,7 @@ class StudentRayActor:
239
  lm_head = lm_head.to(device).eval()
240
  lm_head.requires_grad_(False)
241
  logger.info(f"Loaded lm_head ({weight_key}), shape: {lm_head.weight.shape}")
242
- if self.args.kd.kd_algorithm.startswith('mp_opd_'):
243
  head_config = getattr(config, 'text_config', None) or config
244
  lm_head._mp_opd_softcap = getattr(head_config, 'final_logit_softcapping', None)
245
  return lm_head
@@ -298,6 +298,9 @@ class StudentRayActor:
298
  self.strategy.accumulated_gradient = len(train_data)
299
  self.strategy.step = 0
300
 
 
 
 
301
  for batch in train_data:
302
  micro_batch = self._prepare_micro_batch(batch)
303
 
@@ -333,6 +336,9 @@ class StudentRayActor:
333
 
334
  status["train/lr"] = self.scheduler.get_last_lr()[0]
335
 
 
 
 
336
  for key in status:
337
  status[key] = self.strategy.all_reduce(status[key], op="mean")
338
 
 
239
  lm_head = lm_head.to(device).eval()
240
  lm_head.requires_grad_(False)
241
  logger.info(f"Loaded lm_head ({weight_key}), shape: {lm_head.weight.shape}")
242
+ if self.args.kd.kd_algorithm.startswith('mp_opd_') or self.args.kd.kd_algorithm == 'h1b_span':
243
  head_config = getattr(config, 'text_config', None) or config
244
  lm_head._mp_opd_softcap = getattr(head_config, 'final_logit_softcapping', None)
245
  return lm_head
 
298
  self.strategy.accumulated_gradient = len(train_data)
299
  self.strategy.step = 0
300
 
301
+ if hasattr(self.kd_algorithm,'validate_optimizer_window'):
302
+ self.kd_algorithm.validate_optimizer_window(train_data)
303
+
304
  for batch in train_data:
305
  micro_batch = self._prepare_micro_batch(batch)
306
 
 
336
 
337
  status["train/lr"] = self.scheduler.get_last_lr()[0]
338
 
339
+ if hasattr(self.kd_algorithm,'on_optimizer_window_end'):
340
+ status.update(self.kd_algorithm.on_optimizer_window_end())
341
+
342
  for key in status:
343
  status[key] = self.strategy.all_reduce(status[key], op="mean")
344
 
kdflow/trainer/data_processor/rollout_data_processor.py CHANGED
@@ -14,6 +14,7 @@ class RolloutDataProcessor:
14
  CONDITIONAL_EOS_ALGORITHMS = frozenset(
15
  {
16
  "byte_align_ctkd",
 
17
  "topk_byte_align_ctkd",
18
  }
19
  )
@@ -49,13 +50,17 @@ class RolloutDataProcessor:
49
  """Save, tokenize and collate raw rollout outputs."""
50
  if sample_metadata is not None and len(sample_metadata) != len(outputs):
51
  raise ValueError("sample_metadata and outputs must have the same length")
 
 
 
 
52
  self._save_rollout_data(
53
  stu_prompts,
54
  outputs,
55
  labels,
56
  global_step,
57
  mode,
58
- sample_metadata=sample_metadata,
59
  )
60
 
61
  kd_algorithm = self.args.kd.kd_algorithm
@@ -364,6 +369,10 @@ class RolloutDataProcessor:
364
  "total_length": torch.FloatTensor([[total_length]]),
365
  }
366
  stu_multi_modal_inputs = stu_tokens.get("_stu_multi_modal_inputs")
 
 
 
 
367
  if stu_multi_modal_inputs is not None:
368
  sample["stu_multi_modal_inputs"] = [stu_multi_modal_inputs]
369
  if images:
 
14
  CONDITIONAL_EOS_ALGORITHMS = frozenset(
15
  {
16
  "byte_align_ctkd",
17
+ "h1b_span",
18
  "topk_byte_align_ctkd",
19
  }
20
  )
 
50
  """Save, tokenize and collate raw rollout outputs."""
51
  if sample_metadata is not None and len(sample_metadata) != len(outputs):
52
  raise ValueError("sample_metadata and outputs must have the same length")
53
+ archive_metadata=sample_metadata
54
+ if self.args.kd.kd_algorithm=='h1b_span':
55
+ archive_metadata=[dict(m or {},tea_prompt=t,output_ids=o.get('output_ids',[]))
56
+ for m,t,o in zip(sample_metadata or [{} for _ in outputs],tea_prompts,outputs)]
57
  self._save_rollout_data(
58
  stu_prompts,
59
  outputs,
60
  labels,
61
  global_step,
62
  mode,
63
+ sample_metadata=archive_metadata,
64
  )
65
 
66
  kd_algorithm = self.args.kd.kd_algorithm
 
369
  "total_length": torch.FloatTensor([[total_length]]),
370
  }
371
  stu_multi_modal_inputs = stu_tokens.get("_stu_multi_modal_inputs")
372
+ if self.args.kd.kd_algorithm=='h1b_span':
373
+ tok=getattr(self.student_processor,'tokenizer',self.student_processor)
374
+ encoded=tok(response_text,add_special_tokens=False)['input_ids']
375
+ sample['stu_reencode_match']=torch.FloatTensor([[float(list(encoded)==list(response_ids))]])
376
  if stu_multi_modal_inputs is not None:
377
  sample["stu_multi_modal_inputs"] = [stu_multi_modal_inputs]
378
  if images:
tests/test_h1b.py ADDED
@@ -0,0 +1,347 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Offline H1b contracts. Real tokenizer/rollout/B200 gates are separate tools."""
2
+ import copy
3
+ import ast
4
+ from collections import defaultdict
5
+ import hashlib
6
+ import importlib.util
7
+ import json
8
+ import math
9
+ from pathlib import Path
10
+ import random
11
+ import sys
12
+ import tempfile
13
+ import types
14
+ import unittest
15
+ import test_mp_opd_port as fixtures
16
+
17
+ ROOT=Path(__file__).resolve().parents[1]
18
+ if fixtures.HAS_TORCH:
19
+ import torch
20
+ # Test-only namespace: keep the real numerical/logging modules, bypass the
21
+ # utility package's eager tokenizer factory and optional HF dependencies.
22
+ if 'kdflow.utils' not in sys.modules:
23
+ package=types.ModuleType('kdflow.utils'); package.__path__=[str(ROOT/'kdflow/utils')]
24
+ sys.modules['kdflow.utils']=package
25
+
26
+
27
+ @unittest.skipUnless(fixtures.HAS_TORCH,'requires existing torch; no bootstrap')
28
+ class H1bTests(unittest.TestCase):
29
+ def fixture(self,arm='a'):
30
+ from kdflow.algorithms.h1b_span import H1bSpanCTKD
31
+ helper=fixtures.MPPortTests(); mp,student,teacher=helper.fixture()
32
+ kd=mp.args.kd
33
+ kd.h1b_arm=arm; kd.h1b_span_lambda=.5; kd.h1b_log_sample_rate=.05; kd.h1b_dump_every='0,20,60,100'
34
+ mp.args.train.save_path=None
35
+ mp.strategy.step=0; mp.strategy.accumulated_gradient=1
36
+ algo=H1bSpanCTKD(mp.strategy,student,teacher,fixtures.Tokenizer(),fixtures.Tokenizer())
37
+ return helper,algo,student,teacher
38
+
39
+ def test_T2_cover_matches_bruteforce_and_ff_edges(self):
40
+ from kdflow.h1b.vocab_index import VocabIndex,next_prefix
41
+ vi=VocabIndex(fixtures.Tokenizer())
42
+ rng=random.Random(42)
43
+ for _ in range(2000):
44
+ rem=bytes(rng.choice([97,98,99,100,255]) for _ in range(rng.randint(1,5)))
45
+ self.assertEqual(set(vi.cover(rem)),{i for i,b in vi._bytes.items() if b.startswith(rem) and b})
46
+ self.assertIsNone(next_prefix(b'\xff\xff'))
47
+ self.assertEqual(next_prefix(b'a\xff'),b'b')
48
+ with self.assertRaises(ValueError): vi.cover(b'')
49
+
50
+ def test_T3_strict_groups_match_author_fixture_eos_and_offsets(self):
51
+ from kdflow.h1b.groups import align_byte_span_groups_batch
52
+ from kdflow.token_alignment import align_byte_span_batch_1to1,get_tokenizer_byte_codec
53
+ tok=fixtures.Tokenizer(); codec=get_tokenizer_byte_codec(tok).piece_to_bytes
54
+ for tea,stu in [([['ab','c','<eos>'],['a','b']],[['a','b','c','<eos>'],['ab']]),
55
+ ([['a'],[]],[['a'],[]]),([['<eos>','ab','<eos>']],[['<eos>','a','b','<eos>']])]:
56
+ rows=align_byte_span_groups_batch(tea,stu,tok.eos_token,tok.eos_token,codec,codec)
57
+ groups=[g for r in rows for g in r.groups if g.kind=='strict']
58
+ a=([g.t_idx[0] for g in groups],[g.s_idx[0] for g in groups])
59
+ self.assertEqual(a,align_byte_span_batch_1to1(tea,stu,tok.eos_token,tok.eos_token,codec,codec))
60
+
61
+ def test_T4_trunk_and_exact_enumeration_with_duplicate_byte_ids(self):
62
+ from kdflow.h1b.vocab_index import VocabIndex
63
+ from kdflow.h1b.strprob import chunk_logprobs
64
+ from kdflow.h1b.exact import exact_chunk_logprob
65
+ # 'A' and 'a' map to the same bytes. Exact proper-prefix enumeration
66
+ # must branch on both IDs rather than taking only id_of(...).
67
+ class Tok(fixtures.Tokenizer):
68
+ pieces={0:'<eos>',1:'a',2:'b',3:'ab',4:'A',5:'abc',6:'ba',7:'<pad>'}
69
+ codec=types.SimpleNamespace(piece_to_bytes=lambda s:s.lower().encode())
70
+ vi=VocabIndex(Tok(),codec)
71
+ torch.manual_seed(19)
72
+ table={():torch.randn(8).log_softmax(0),(1,):torch.randn(8).log_softmax(0),(4,):torch.randn(8).log_softmax(0)}
73
+ lp=chunk_logprobs(torch.stack([table[()],table[(1,)]]),[1,2],b'ab',vi)
74
+ expected=torch.logsumexp(torch.stack([table[()][3],table[()][5],table[()][1]+torch.logsumexp(table[(1,)][[2,6]],0)]),0)
75
+ torch.testing.assert_close(lp.log_p_trunk,expected)
76
+ result=exact_chunk_logprob(lambda ids:table[ids],b'ab',vi)
77
+ exact=math.exp(float(expected))+math.exp(float(table[()][4]+torch.logsumexp(table[(4,)][[2,6]],0)))
78
+ self.assertAlmostEqual(result['log_probability_lower'],math.log(exact),places=6)
79
+ self.assertTrue(result['exact'])
80
+ self.assertGreaterEqual(result['log_probability_lower']+1e-6,float(lp.log_p_trunk))
81
+ bounded=exact_chunk_logprob(lambda ids:table[ids],b'ab',vi,max_nodes=1)
82
+ self.assertLessEqual(bounded['log_probability_lower'],math.log(exact)+1e-6)
83
+ self.assertGreaterEqual(bounded['log_probability_upper']+1e-6,math.log(exact))
84
+
85
+ def test_T4_thirty_token_lm_fifty_strings(self):
86
+ import itertools
87
+ from kdflow.h1b.vocab_index import VocabIndex
88
+ from kdflow.h1b.strprob import chunk_logprobs
89
+ from kdflow.h1b.exact import exact_chunk_logprob
90
+ words=[''.join(x) for size in range(1,5) for x in itertools.product('ab',repeat=size)]
91
+ class Tok(fixtures.Tokenizer):
92
+ pieces={0:'<eos>',31:'<pad>',**{i+1:w for i,w in enumerate(words)}}
93
+ all_special_ids=[0,31]; pad_token_id=31
94
+ vi=VocabIndex(Tok()); rng=random.Random(17)
95
+ def oracle(path):
96
+ seed=int.from_bytes(hashlib.sha256(bytes(path)).digest()[:4],'big')
97
+ return torch.randn(32,generator=torch.Generator().manual_seed(seed),dtype=torch.float64).log_softmax(0)
98
+ for _ in range(50):
99
+ chunk=''.join(rng.choice('ab') for _ in range(rng.randint(1,4))).encode()
100
+ path=[vi.id_of(bytes([b])) for b in chunk]
101
+ lp=chunk_logprobs(torch.stack([oracle(tuple(path[:j])) for j in range(len(path))]),path,chunk,vi)
102
+ acc=1.; brute=0.
103
+ for j,token in enumerate(path):
104
+ row=oracle(tuple(path[:j])).exp(); rem=chunk[j:]
105
+ brute+=acc*sum(float(row[i]) for i,b in vi._bytes.items() if b.startswith(rem))
106
+ acc*=float(row[token])
107
+ self.assertAlmostEqual(float(lp.log_p_trunk),math.log(brute),places=6)
108
+ exact=exact_chunk_logprob(oracle,chunk,vi)
109
+ self.assertGreaterEqual(exact['log_probability_lower']+1e-6,float(lp.log_p_trunk))
110
+
111
+ def test_T5_same_tokenizer_all_arms_have_zero_span(self):
112
+ reference=None
113
+ for arm in ('a','b','b_bl','c_str','c'):
114
+ helper,algo,_,_=self.fixture(arm); batch=helper.batch()
115
+ batch['tea_input_ids']=batch['stu_input_ids'].clone()
116
+ batch['tea_loss_mask']=batch['stu_loss_mask'].clone()
117
+ batch['tea_attn_mask']=batch['stu_attn_mask'].clone()
118
+ batch['teacher_hiddens']=torch.randn(int(batch['tea_loss_mask'].sum()),4)
119
+ info=algo.training_step(batch)
120
+ self.assertEqual(float(info['train/h1b_span_loss'].detach()),0.)
121
+ self.assertEqual(float(info['distill/h1b_mis_groups']),0.)
122
+ value=info['train/loss'].detach()
123
+ if reference is None: reference=value
124
+ else: torch.testing.assert_close(value,reference)
125
+
126
+ def test_T6_baseline_previous_window_and_evaluation_immutability(self):
127
+ helper,algo,student,_=self.fixture('c')
128
+ before=algo.baseline.prev.clone()
129
+ algo.training_step(helper.batch())
130
+ torch.testing.assert_close(algo.baseline.prev,before)
131
+ algo.on_optimizer_window_end()
132
+ self.assertTrue(bool(algo.baseline.prev.any()))
133
+ state=copy.deepcopy(algo.state_dict()); counts=algo.baseline.counts.clone()
134
+ student.eval()
135
+ with torch.no_grad(): algo.training_step(helper.batch())
136
+ torch.testing.assert_close(algo.baseline.prev,state['baseline']['prev'])
137
+ torch.testing.assert_close(algo.baseline.counts,counts)
138
+ self.assertEqual(algo.chunk_logger.rows,0)
139
+ algo.load_state_dict(state)
140
+
141
+ def test_T7_span_surrogate_gradient_equals_mse(self):
142
+ helper,algo,student,_=self.fixture('b')
143
+ algo.replay_enabled=True
144
+ info=algo.training_step(helper.batch())
145
+ expected=sum(algo.lam*(r['log_q']-(r['log_q'].detach()-r['path']))**2 for r in algo.replay_records)/helper.batch()['avg_micro_batch_token_num']
146
+ a=torch.autograd.grad(info['train/h1b_span_loss'],tuple(student.parameters()),retain_graph=True)
147
+ b=torch.autograd.grad(expected,tuple(student.parameters()))
148
+ for x,y in zip(a,b): torch.testing.assert_close(x,y,atol=1e-6,rtol=1e-5)
149
+
150
+ def test_T8_arm_a_loss_and_all_gradients_equal_strict(self):
151
+ from kdflow.algorithms.byte_align_ctkd import ByteAlignCrossTokenizerKD
152
+ helper,algo,student,teacher=self.fixture('a')
153
+ strict=ByteAlignCrossTokenizerKD(algo.strategy,student,teacher,fixtures.Tokenizer(),fixtures.Tokenizer())
154
+ batch=helper.batch(2)
155
+ a=algo.training_step(batch)['train/loss']; b=strict.training_step(batch)['train/loss']
156
+ torch.testing.assert_close(a,b,atol=1e-6,rtol=1e-6)
157
+ for x,y in zip(torch.autograd.grad(a,tuple(student.parameters())),torch.autograd.grad(b,tuple(student.parameters()))):
158
+ torch.testing.assert_close(x,y,atol=1e-6,rtol=1e-6)
159
+ self.assertIsNone(teacher.weight.grad)
160
+
161
+ def test_byte_mismatch_drops_complete_response_including_eos(self):
162
+ helper,algo,student,_=self.fixture('b')
163
+ batch=helper.batch(); batch['tea_input_ids'][0,1]=5
164
+ info=algo.training_step(batch)
165
+ self.assertEqual(float(info['train/loss'].detach()),0.)
166
+ self.assertEqual(float(info['distill/h1b_dropped_responses']),1.)
167
+ info['train/loss'].backward()
168
+ self.assertTrue(all(p.grad is None or not bool(p.grad.any()) for p in student.parameters()))
169
+
170
+ def test_tile_recomputation_preserves_path_gradient_and_statistics(self):
171
+ helper,algo,student,_=self.fixture('c_str')
172
+ batch=helper.batch(); a=algo.training_step(batch)
173
+ ga=torch.autograd.grad(a['train/loss'],tuple(student.parameters()))
174
+ algo.args.train.chunked_loss_size=1
175
+ b=algo.training_step(batch); gb=torch.autograd.grad(b['train/loss'],tuple(student.parameters()))
176
+ torch.testing.assert_close(a['train/loss'],b['train/loss'])
177
+ for x,y in zip(ga,gb): torch.testing.assert_close(x,y,atol=1e-6,rtol=1e-5)
178
+
179
+ def test_window_statistics_use_chunk_counts_not_microbatch_means(self):
180
+ from kdflow.h1b.chunklog import ChunkLogger
181
+ log=ChunkLogger(sample_rate=0,dump_steps=())
182
+ for v in (1.,2.,9.):
183
+ row=dict(path=v,str=v/2,path_bl=v-1,str_bl=v/2-2,gap=v/2,slack_stu=1.,slack_tea=1+v/2,
184
+ overhang_stu=0.,overhang_tea=0.,bucket=0,response_hash=str(v),b0=0,b1=1)
185
+ log.add(row)
186
+ out=log.end_window(0)
187
+ self.assertEqual(out['distill/h1b_gap_mean'],2.)
188
+ self.assertAlmostEqual(out['distill/h1b_var_path'],(1+4+81)/3-16)
189
+
190
+ def test_actual_actor_hook_runs_after_one_update_and_not_during_eval(self):
191
+ from kdflow.utils.logging_utils import normalize_eval_metrics
192
+ helper,algo,student,_=self.fixture('c')
193
+ tree=ast.parse((ROOT/'kdflow/ray/train/student_actor.py').read_text())
194
+ scope=dict(torch=torch,defaultdict=defaultdict,normalize_eval_metrics=normalize_eval_metrics)
195
+ methods={}
196
+ for name in ('fit','evaluate'):
197
+ node=next(n for n in ast.walk(tree) if isinstance(n,ast.FunctionDef) and n.name==name)
198
+ node.decorator_list=[]
199
+ exec(compile(ast.Module(body=[node],type_ignores=[]),'actual_h1b_actor','exec'),scope)
200
+ methods[name]=scope[name]
201
+ strategy=algo.strategy; strategy.accumulated_gradient=2
202
+ algo.args.train.use_dynamic_bsz=False; algo.args.kd.use_ema_teacher=False
203
+ optimizer=torch.optim.SGD(student.parameters(),lr=.01); updates=[]
204
+ def backward(loss,*_):
205
+ strategy.step=(strategy.step+1)%2
206
+ (loss/2).backward()
207
+ def step(*_):
208
+ if strategy.step==0: optimizer.step(); optimizer.zero_grad(); updates.append(1)
209
+ strategy.backward=backward; strategy.optimizer_step=step
210
+ strategy.all_reduce=lambda x,**_:x
211
+ actor=types.SimpleNamespace(student=student,kd_algorithm=algo,args=algo.args,strategy=strategy,
212
+ optim=optimizer,scheduler=types.SimpleNamespace(get_last_lr=lambda:[.01]),_prepare_micro_batch=lambda b:dict(b))
213
+ original=algo.on_optimizer_window_end
214
+ def end():
215
+ self.assertEqual(len(updates),1)
216
+ return original()
217
+ algo.on_optimizer_window_end=end
218
+ batches=[helper.batch(),helper.batch(2)]
219
+ result=methods['fit'](actor,batches)
220
+ self.assertEqual(result['distill/h1b_window_responses'],3.)
221
+ self.assertEqual(algo.window,1)
222
+ previous=algo.baseline.prev.clone()
223
+ with torch.no_grad(): methods['evaluate'](actor,batches)
224
+ self.assertEqual(algo.window,1)
225
+ torch.testing.assert_close(algo.baseline.prev,previous)
226
+
227
+ def test_math_eval_override_never_changes_rollout_top_p(self):
228
+ from kdflow.arguments.rollout_args import RolloutArguments
229
+ tree=ast.parse((ROOT/'kdflow/evaluation/math_evaluator.py').read_text())
230
+ method=next(n for n in ast.walk(tree) if isinstance(n,ast.FunctionDef) and n.name=='evaluate')
231
+ assignments=[n for n in method.body if isinstance(n,ast.Assign) and any(isinstance(t,ast.Name) and t.id in {'eval_top_p','sampling_params'} for t in n.targets)]
232
+ for override,expected in ((None,1.),(.95,.95)):
233
+ actor=types.SimpleNamespace(args=types.SimpleNamespace(rollout=RolloutArguments(top_p=1.,eval_math_top_p=override)),
234
+ max_tokens=8192,n_samples=32,generate_kwargs={'top_p':1.,'temperature':1.})
235
+ scope={'self':actor}
236
+ exec(compile(ast.Module(body=assignments,type_ignores=[]),'actual_math_sampling','exec'),scope)
237
+ self.assertEqual(scope['sampling_params']['top_p'],expected)
238
+ self.assertEqual(actor.generate_kwargs['top_p'],1.)
239
+ with self.assertRaises(ValueError): RolloutArguments(eval_math_top_p=0)
240
+
241
+ def test_fit_curve_and_analysis_handle_known_direction_and_zero_vectors(self):
242
+ import numpy as np
243
+ from tools.h1b.fit_cb import fit_curve,analyze_matrices
244
+ sizes=[64,128,256,512,1024,2048,4096]
245
+ fit=fit_curve(sizes,[.7/math.sqrt(1+300/b) for b in sizes])
246
+ self.assertAlmostEqual(fit['c_infinity'],.7,places=2)
247
+ x=np.tile(np.array([1.,2.,3.]),(8,1))
248
+ report=analyze_matrices(x,x,x,x)
249
+ self.assertTrue(all(abs(r['cosine']-1)<1e-9 for r in report['curve']))
250
+ zero=analyze_matrices(x,x*0,x,x)
251
+ self.assertTrue(all(r['cosine'] is None for r in zero['curve']))
252
+
253
+ def test_replay_analysis_cli_on_complete_small_saved_sketches(self):
254
+ from tools.h1b.fit_cb import main
255
+ torch.manual_seed(3)
256
+ keys=['strict','path','str','base_path','base_str','strict_0','strict_1']
257
+ with tempfile.TemporaryDirectory() as tmp:
258
+ root=Path(tmp); rows=[]
259
+ for i in range(8):
260
+ files={}
261
+ for key in keys:
262
+ file=f'{i}_{key}.pt'; torch.save(torch.randn(16),root/file); files[key]=file
263
+ rows.append(dict(files=files,tokens=20+i))
264
+ (root/'manifest.json').write_text(json.dumps(dict(records=rows,groups=8,group_size=64,bucket_classes=[])))
265
+ main(['--replay',tmp,'--output',str(root/'curves.json'),'--bootstrap','2'])
266
+ result=json.loads((root/'curves.json').read_text())
267
+ self.assertEqual(set(result['results']),{'path','str','path_bl','str_bl'})
268
+ self.assertIsNotNone(result['cos_mean_path_str'])
269
+
270
+ def test_h1b_rollout_preserves_eos_policy_and_reports_raw_reencoding(self):
271
+ from typing import Any,Dict,List,Optional
272
+ tree=ast.parse((ROOT/'kdflow/trainer/data_processor/rollout_data_processor.py').read_text())
273
+ node=next(n for n in tree.body if isinstance(n,ast.ClassDef))
274
+ scope=dict(torch=torch,Any=Any,Dict=Dict,List=List,Optional=Optional)
275
+ exec(compile(ast.Module(body=[node],type_ignores=[]),'actual_h1b_processor','exec'),scope)
276
+ class Tok:
277
+ eos_token_id=0
278
+ def __call__(self,text,return_tensors=None,**kwargs):
279
+ ids=[{'p':6,'a':1,'b':2,'c':3}[c] for c in text]
280
+ return {'input_ids':torch.tensor([ids]),'attention_mask':torch.ones(1,len(ids),dtype=torch.long)} if return_tensors else {'input_ids':ids}
281
+ obj=object.__new__(scope['RolloutDataProcessor'])
282
+ obj.args=types.SimpleNamespace(kd=types.SimpleNamespace(kd_algorithm='h1b_span'))
283
+ obj.student_processor=Tok(); obj.teacher_processors={}; obj.is_same_tokenizer=True
284
+ sample=obj._build_rollout_sample('p','p',dict(text='abc',output_ids=[1,2,3]),None,append_eos=False)
285
+ self.assertEqual(float(sample['stu_reencode_match']),1.)
286
+ self.assertEqual(sample['stu_input_ids'].tolist(),[6,1,2,3])
287
+ sample=obj._build_rollout_sample('p','p',dict(text='abc',output_ids=[2,3]),None,append_eos=True)
288
+ self.assertEqual(float(sample['stu_reencode_match']),0.)
289
+ self.assertEqual(sample['stu_input_ids'][-1],0)
290
+
291
+ def test_KV_oracle_keeps_sibling_prefix_cache_immutable(self):
292
+ from tools.h1b.common import HFPrefixOracle
293
+ class Base(torch.nn.Module):
294
+ def forward(self,ids,past_key_values=None,use_cache=False,**kwargs):
295
+ cache={'ids':[]} if past_key_values is None else past_key_values
296
+ cache['ids'].extend(ids[0].tolist()) # Deliberate in-place cache mutation.
297
+ x=torch.tensor([sum(cache['ids']),len(cache['ids']),cache['ids'][-1],1.],dtype=torch.float32)
298
+ return types.SimpleNamespace(last_hidden_state=x[None,None,:],past_key_values=cache if use_cache else None)
299
+ class Model(torch.nn.Module):
300
+ def __init__(self): super().__init__(); self.base_model=Base(); self.head=torch.nn.Linear(4,8)
301
+ def get_output_embeddings(self): return self.head
302
+ model=Model(); cached=HFPrefixOracle(model,[6,1,2],cache_depth=2); direct=HFPrefixOracle(model,[6,1,2],use_cache=False)
303
+ for branch in ((),(1,),(1,2),(1,2,3),(2,),(2,3),(1,)):
304
+ torch.testing.assert_close(cached(branch),direct(branch))
305
+ self.assertEqual(cached.stack[0][2]['ids'],[6,1,2])
306
+
307
+ def test_old_archive_prompt_recovery_requires_exact_unambiguous_match(self):
308
+ from tools.h1b.enrich_archive import recover
309
+ records=[dict(prompt='S:x',output='abc',finish_reason='stop')]
310
+ rows=[dict(s='S:x',t='T:x')]
311
+ result=recover(records,rows,lambda r:r['s'],lambda r:r['t'])
312
+ self.assertEqual(result[0]['tea_prompt'],'T:x')
313
+ self.assertEqual(result[0]['output'],'abc')
314
+ self.assertNotIn('output_ids',result[0])
315
+ with self.assertRaises(ValueError): recover(records,[],lambda r:r['s'],lambda r:r['t'])
316
+ with self.assertRaises(ValueError): recover(records,rows+[dict(s='S:x',t='T:y')],lambda r:r['s'],lambda r:r['t'])
317
+
318
+ def test_T11_sketch_cosine_norm_and_chunk_invariance(self):
319
+ from tools.h1b.sketch import count_sketch
320
+ torch.manual_seed(9); a=torch.randn(2000); b=a+.2*torch.randn_like(a)
321
+ sa,na=count_sketch([('x',a)],chunk_size=17)
322
+ sb,nb=count_sketch([('x',b)],chunk_size=99)
323
+ sa2,_=count_sketch([('x',a)],chunk_size=33)
324
+ torch.testing.assert_close(sa,sa2)
325
+ self.assertLess(abs(float(sa.norm())/na-1),.01)
326
+ self.assertLess(abs(float(sb.norm())/nb-1),.01)
327
+ self.assertLess(abs(float(torch.nn.functional.cosine_similarity(sa,sb,dim=0)-torch.nn.functional.cosine_similarity(a,b,dim=0))),.01)
328
+
329
+ def test_presets_share_full_workload_and_separate_eval_top_p(self):
330
+ from tools.launch_single_b200 import parser,build_plan,H1B_ARMS
331
+ original=build_plan(parser().parse_args(['--algorithm','strict-full','--dry-run']))
332
+ allowed={'kd_algorithm','h1b_arm','h1b_span_lambda','h1b_log_sample_rate','h1b_dump_every','top_p','eval_math_top_p','save_path','ckpt_path'}
333
+ for name,arm in H1B_ARMS.items():
334
+ plan=build_plan(parser().parse_args(['--algorithm',name,'--dry-run']))
335
+ self.assertEqual(plan['flags']['h1b_arm'],arm)
336
+ self.assertEqual(plan['flags']['top_p'],1.)
337
+ self.assertEqual(plan['flags']['eval_math_top_p'],.95)
338
+ for key,value in original['flags'].items():
339
+ if key not in allowed: self.assertEqual(plan['flags'][key],value)
340
+
341
+ def test_source_manifest_frozen_strict(self):
342
+ manifest=json.loads((ROOT/'docs/H1B_SOURCE.json').read_text())
343
+ for row in manifest['files']+manifest['frozen_strict']:
344
+ self.assertEqual(hashlib.sha256((ROOT/row['path']).read_bytes()).hexdigest(),row['sha256'])
345
+
346
+
347
+ if __name__=='__main__': unittest.main()
tools/h1b/__init__.py ADDED
@@ -0,0 +1 @@
 
 
1
+ """Offline H1b tools. Invoke from the source root with python -m tools.h1b.<tool>."""
tools/h1b/canary.py ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Separate five-step diagnostic (T10); never a reduced-workload reproduction."""
2
+ import argparse
3
+ import json
4
+ import math
5
+ import os
6
+ import re
7
+ from pathlib import Path
8
+ import subprocess
9
+ import time
10
+ from tools.launch_single_b200 import parser,build_plan,validate_files,run_environment,H1B_ARMS
11
+
12
+
13
+ def main(argv=None):
14
+ p=argparse.ArgumentParser(description=__doc__)
15
+ p.add_argument('--algorithm',choices=H1B_ARMS,required=True); p.add_argument('--runtime',required=True)
16
+ p.add_argument('--root',default='/workspace/storage-shared/nlp/tungks/ctopd_all')
17
+ p.add_argument('--run-id',required=True); p.add_argument('--execute',action='store_true')
18
+ args=p.parse_args(argv)
19
+ if not args.run_id.startswith('canary-h1b-'): p.error('Use a distinct canary-h1b-* run ID')
20
+ plan=build_plan(parser().parse_args(['--algorithm',args.algorithm,'--runtime',args.runtime,'--root',args.root,'--run-id',args.run_id]))
21
+ plan['flags']['max_steps']=5
22
+ plan['qualification']='T10 five-step diagnostic only; full 512/8192 workload per step; not reproduction'
23
+ command=['bash',str(Path(args.runtime)/'python-b200.sh'),'-u','-m','kdflow.cli.train_kd_on_policy']
24
+ for k,v in plan['flags'].items(): command.extend(['--'+k,str(v)])
25
+ plan['command']=command
26
+ if not args.execute:
27
+ print(json.dumps(plan,indent=2)); return
28
+ source=Path(plan['source'])
29
+ if subprocess.check_output(['git','-C',str(source),'status','--porcelain'],text=True).strip():
30
+ raise RuntimeError('Commit source before diagnostic execution')
31
+ validate_files(plan)
32
+ env=run_environment()
33
+ probe="import torch; assert torch.cuda.device_count()==1; assert 'B200' in torch.cuda.get_device_name(0); print(torch.cuda.get_device_name(0))"
34
+ subprocess.run(['bash',str(Path(args.runtime)/'python-b200.sh'),'-c',probe],env=env,check=True)
35
+ root=Path(plan['output']); root.mkdir(parents=True,exist_ok=False)
36
+ plan['source_commit']=subprocess.check_output(['git','-C',str(source),'rev-parse','HEAD'],text=True).strip()
37
+ (root/'canary-plan.json').write_text(json.dumps(plan,indent=2))
38
+ start=time.monotonic()
39
+ with (root/'canary.log').open('x') as log:
40
+ with subprocess.Popen(command,env=env,cwd=source,stdout=subprocess.PIPE,stderr=subprocess.STDOUT,text=True) as process:
41
+ for line in process.stdout:
42
+ log.write(line); log.flush(); print(line,end='',flush=True)
43
+ code=process.wait()
44
+ receipt=dict(source_commit=plan['source_commit'],algorithm=args.algorithm,exit_code=code,
45
+ elapsed_seconds=time.monotonic()-start,passed=False,steps=[],qualification='Company-node canary only; no metric-match claim')
46
+ import pyarrow.parquet as pq
47
+ if code==0:
48
+ values=re.findall(r'train/(?:loss|kd_loss|h1b_span_loss|grad_norm):\s*([^\s,]+)',(root/'canary.log').read_text())
49
+ if not values or not all(math.isfinite(float(v)) for v in values):
50
+ raise ValueError('Missing/nonfinite train loss/gradient metrics')
51
+ for step in range(5):
52
+ path=root/'checkpoints'/'h1b'/f'step_{step}.json'
53
+ summary=json.loads(path.read_text()); metrics=summary['metrics']
54
+ required={'gap_mean','slack_stu','slack_tea','bias_floor','var_path','var_str','var_path_bl','var_str_bl',
55
+ 'corr_path_str','overhang_share_stu','overhang_share_tea','reencode_match','dropped_byte_mismatch'}
56
+ if not required<=metrics.keys() or not all(math.isfinite(float(v)) for v in metrics.values()):
57
+ raise ValueError('Missing/nonfinite canary metrics')
58
+ file=path.parent/f'chunks_step_{step}.parquet'
59
+ rows=pq.read_table(file).num_rows
60
+ receipt['steps'].append(dict(step=step,sampled_chunks=rows,dump_bytes=file.stat().st_size))
61
+ receipt['passed']=True
62
+ (root/'canary-receipt.json').write_text(json.dumps(receipt,indent=2))
63
+ if not receipt['passed']: raise SystemExit('Canary failed; preserve log and receipt')
64
+ print(json.dumps(receipt))
65
+
66
+
67
+ if __name__=='__main__': main()
tools/h1b/common.py ADDED
@@ -0,0 +1,138 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Offline local-snapshot loading and production-tokenization reuse."""
2
+ import json
3
+ import hashlib
4
+ import copy
5
+ import subprocess
6
+ from pathlib import Path
7
+ import torch
8
+
9
+
10
+ def source_identity():
11
+ root=Path(__file__).resolve().parents[2]
12
+ commit=dirty=None
13
+ if (root/'.git').exists():
14
+ commit=subprocess.check_output(['git','-C',str(root),'rev-parse','HEAD'],text=True).strip()
15
+ dirty=bool(subprocess.check_output(['git','-C',str(root),'status','--porcelain'],text=True).strip())
16
+ manifest=root/'docs/H1B_SOURCE.json'
17
+ return dict(commit=commit,dirty=dirty,
18
+ source_manifest_sha256=hashlib.sha256(manifest.read_bytes()).hexdigest())
19
+
20
+
21
+ def snapshot_identity(path):
22
+ root=Path(path).resolve()
23
+ return dict(path=str(root),metadata_sha256={name:hashlib.sha256((root/name).read_bytes()).hexdigest()
24
+ for name in ('config.json','model.safetensors.index.json','tokenizer.json','tokenizer_config.json') if (root/name).is_file()},
25
+ qualification='Metadata identity; individual weight shards not hashed by this helper')
26
+
27
+
28
+ def archive_rows(path,limit=None):
29
+ with open(path) as f:
30
+ for i,line in enumerate(f):
31
+ if limit is not None and i>=limit: break
32
+ yield json.loads(line)
33
+
34
+
35
+ def load_pair(student_path,teacher_path,device):
36
+ from transformers import AutoModelForCausalLM,AutoTokenizer
37
+ if str(device).startswith('cuda'):
38
+ if torch.cuda.device_count()!=1 or 'B200' not in torch.cuda.get_device_name(0):
39
+ raise ValueError('Expose exactly one allocated B200 for H1b offline GPU tools')
40
+ models=[]; tokenizers=[]
41
+ for path in (student_path,teacher_path):
42
+ if not Path(path).is_dir(): raise FileNotFoundError('Require local model snapshot: '+str(path))
43
+ tokenizers.append(AutoTokenizer.from_pretrained(path,local_files_only=True))
44
+ models.append(AutoModelForCausalLM.from_pretrained(path,local_files_only=True,
45
+ torch_dtype=torch.bfloat16).to(device).eval())
46
+ config=getattr(models[-1].config,'text_config',None) or models[-1].config
47
+ if getattr(config,'final_logit_softcapping',None): raise ValueError('H1b softcapped heads unsupported')
48
+ return models,tokenizers
49
+
50
+
51
+ def tokenize_record(record,student_tok,teacher_tok,assume_shared_prompt=False):
52
+ from kdflow.trainer.data_processor.rollout_data_processor import RolloutDataProcessor
53
+ # Reuse production method rather than approximating prompt boundary masks.
54
+ processor=object.__new__(RolloutDataProcessor)
55
+ if 'tea_prompt' not in record and not assume_shared_prompt:
56
+ raise ValueError('Archive lacks tea_prompt; supply enriched H1b archive or explicitly --assume-shared-prompt')
57
+ reason=record.get('finish_reason')
58
+ output=dict(text=record['output'],output_ids=record.get('output_ids',[]),finish_reason=reason)
59
+ if reason is None and not record.get('output_ids'):
60
+ raise ValueError('Cannot reconstruct truncation without finish_reason/output_ids')
61
+ append=not processor._is_length_truncated(output,{'max_new_tokens':8192})
62
+ out={}
63
+ for prefix,tok,prompt in (('stu',student_tok,record['prompt']),('tea',teacher_tok,record.get('tea_prompt',record['prompt']))):
64
+ fields=processor._tokenize_sample(prompt,record['output'],tok,prefix,append_eos=append)
65
+ out.update({k:v[None] for k,v in fields.items() if not k.startswith('_')})
66
+ return out
67
+
68
+
69
+ def row_groups(batch,student_tok,teacher_tok):
70
+ from kdflow.token_alignment import get_tokenizer_byte_codec
71
+ from kdflow.h1b.groups import align_byte_span_groups_batch
72
+ rows=[]
73
+ for key,tok in (('tea',teacher_tok),('stu',student_tok)):
74
+ labels=batch[key+'_input_ids'].roll(-1,1)
75
+ rows.append([tok.convert_ids_to_tokens(labels[i][batch[key+'_loss_mask'][i].bool()].tolist()) for i in range(len(labels))])
76
+ return align_byte_span_groups_batch(*rows,teacher_tok.eos_token,student_tok.eos_token,
77
+ get_tokenizer_byte_codec(teacher_tok).piece_to_bytes,get_tokenizer_byte_codec(student_tok).piece_to_bytes)
78
+
79
+
80
+ def forward_hidden(model,batch,prefix):
81
+ ids=batch[prefix+'_input_ids'].to(next(model.parameters()).device)
82
+ out=model.base_model(ids,attention_mask=batch[prefix+'_attn_mask'].to(ids.device),output_hidden_states=True,use_cache=False)
83
+ return out.hidden_states[-1][batch[prefix+'_loss_mask'].to(ids.device).bool()]
84
+
85
+
86
+ class StudentWrapper(torch.nn.Module):
87
+ def __init__(self,model):
88
+ super().__init__(); self.transformer=model
89
+ # Proxy contains the actual head; avoids assigning model.lm_head to itself.
90
+ import types
91
+ head=model.get_output_embeddings()
92
+ class Head(torch.nn.Module):
93
+ def __init__(self,head): super().__init__(); self.head=head
94
+ @property
95
+ def weight(self): return self.head.weight
96
+ def forward(self,x,skip=False): return x if skip else self.head(x)
97
+ self.model=types.SimpleNamespace(config=model.config,lm_head=Head(head))
98
+ def forward(self,ids,attention_mask=None,**kwargs):
99
+ output=self.transformer.base_model(ids,attention_mask=attention_mask,output_hidden_states=True,use_cache=False)
100
+ return {'hidden_states':output.hidden_states}
101
+
102
+
103
+ class HFPrefixOracle:
104
+ """Branch-safe depth-bounded KV stack for depth-first cover enumeration.
105
+
106
+ HF DynamicCache may mutate in place. Each child receives its own copy; the
107
+ parent's cache remains reusable for siblings. Full-prefix fallback bounds
108
+ retained cache depth for unusually long chunks. No branch graph is retained.
109
+ """
110
+ @torch.no_grad()
111
+ def __init__(self,model,prefix_ids,cache_depth=16,use_cache=True):
112
+ if not prefix_ids or cache_depth<1: raise ValueError('Nonempty prefix and positive cache depth required')
113
+ self.model=model; self.prefix=list(prefix_ids); self.depth=cache_depth; self.use_cache=use_cache
114
+ self.device=next(model.parameters()).device
115
+ out=model.base_model(torch.tensor([self.prefix],device=self.device),use_cache=use_cache)
116
+ lp=model.get_output_embeddings()(out.last_hidden_state[0,-1]).float().log_softmax(-1)
117
+ self.stack=[((),lp,getattr(out,'past_key_values',None))]
118
+
119
+ @torch.no_grad()
120
+ def __call__(self,branch):
121
+ branch=tuple(branch)
122
+ if not branch: return self.stack[0][1]
123
+ if not self.use_cache or len(branch)>self.depth or self.stack[0][2] is None:
124
+ out=self.model.base_model(torch.tensor([self.prefix+list(branch)],device=self.device),use_cache=False)
125
+ return self.model.get_output_embeddings()(out.last_hidden_state[0,-1]).float().log_softmax(-1)
126
+ common=0
127
+ for level in range(1,min(len(branch),len(self.stack)-1)+1):
128
+ if self.stack[level][0]!=branch[:level]: break
129
+ common=level
130
+ self.stack=self.stack[:common+1]
131
+ for level in range(common+1,len(branch)+1):
132
+ cache=copy.deepcopy(self.stack[-1][2])
133
+ out=self.model.base_model(torch.tensor([[branch[level-1]]],device=self.device),
134
+ attention_mask=torch.ones(1,len(self.prefix)+level,device=self.device,dtype=torch.long),
135
+ past_key_values=cache,use_cache=True)
136
+ lp=self.model.get_output_embeddings()(out.last_hidden_state[0,-1]).float().log_softmax(-1)
137
+ self.stack.append((branch[:level],lp,out.past_key_values))
138
+ return self.stack[-1][1]
tools/h1b/enrich_archive.py ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Recover teacher prompts in old rollout archives by exact dataset prompt lookup."""
2
+ import argparse
3
+ import hashlib
4
+ import json
5
+ from pathlib import Path
6
+ from .common import archive_rows,source_identity,snapshot_identity
7
+
8
+
9
+ def recover(records,dataset_rows,student_prompt,teacher_prompt):
10
+ mapping={}
11
+ for row in dataset_rows:
12
+ s,t=student_prompt(row),teacher_prompt(row)
13
+ if s in mapping and mapping[s]!=t: raise ValueError('Ambiguous teacher prompt for identical student prompt')
14
+ mapping[s]=t
15
+ output=[]
16
+ for record in records:
17
+ if record['prompt'] not in mapping: raise ValueError('Archived student prompt has no exact dataset/template match')
18
+ teacher=mapping[record['prompt']]
19
+ if 'tea_prompt' in record and record['tea_prompt']!=teacher:
20
+ raise ValueError('Existing archived teacher prompt differs from recovered prompt')
21
+ output.append(dict(record,tea_prompt=teacher))
22
+ return output
23
+
24
+
25
+ def main(argv=None):
26
+ p=argparse.ArgumentParser(description=__doc__)
27
+ for name in ('archive','dataset','student','teacher','output'): p.add_argument('--'+name,required=True)
28
+ p.add_argument('--input-key',default='prompt'); p.add_argument('--teacher-input-key',default='prompt')
29
+ args=p.parse_args(argv); target=Path(args.output)
30
+ if target.exists(): raise FileExistsError(target)
31
+ import pyarrow.parquet as pq
32
+ from transformers import AutoTokenizer
33
+ from kdflow.datasets.prompts_dataset import PromptDataset
34
+ toks=[AutoTokenizer.from_pretrained(p,local_files_only=True) for p in (args.student,args.teacher)]
35
+ builder=object.__new__(PromptDataset)
36
+ builder.apply_chat_template=True; builder.image_key=None; builder.enable_thinking=False
37
+ builder.input_template=None
38
+ rows=pq.read_table(args.dataset,columns=list(dict.fromkeys((args.input_key,args.teacher_input_key)))).to_pylist()
39
+ records=recover(list(archive_rows(args.archive)),rows,
40
+ lambda r:builder._build_prompt(r,toks[0],args.input_key),
41
+ lambda r:builder._build_prompt(r,toks[1],args.teacher_input_key))
42
+ target.parent.mkdir(parents=True,exist_ok=True)
43
+ with target.open('x') as f:
44
+ for record in records: f.write(json.dumps(record,ensure_ascii=False)+'\n')
45
+ receipt=dict(source=source_identity(),records=len(records),
46
+ archive_sha256=hashlib.sha256(Path(args.archive).read_bytes()).hexdigest(),
47
+ dataset_sha256=hashlib.sha256(Path(args.dataset).read_bytes()).hexdigest(),
48
+ student=snapshot_identity(args.student),teacher=snapshot_identity(args.teacher),
49
+ output_sha256=hashlib.sha256(target.read_bytes()).hexdigest(),
50
+ contract='Exact student prompt lookup, production PromptDataset template method; enable_thinking=False; no response regeneration or sampled-ID reconstruction')
51
+ Path(str(target)+'.receipt.json').write_text(json.dumps(receipt,indent=2))
52
+
53
+
54
+ if __name__=='__main__': main()
tools/h1b/exact_strprob.py ADDED
@@ -0,0 +1,87 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Score an archived rollout; bound off-trunk mass without generating responses."""
2
+ import argparse
3
+ import hashlib
4
+ import json
5
+ import math
6
+ from pathlib import Path
7
+ import torch
8
+ from kdflow.h1b.vocab_index import VocabIndex
9
+ from kdflow.h1b.strprob import score_hidden_chunks
10
+ from kdflow.h1b.buckets import bucket_of
11
+ from kdflow.h1b.exact import exact_chunk_logprob
12
+ from .common import archive_rows,load_pair,tokenize_record,row_groups,forward_hidden,snapshot_identity,HFPrefixOracle,source_identity
13
+
14
+
15
+ def main(argv=None):
16
+ p=argparse.ArgumentParser(description=__doc__)
17
+ p.add_argument('--archive',required=True); p.add_argument('--student',required=True); p.add_argument('--teacher',required=True)
18
+ p.add_argument('--output',required=True); p.add_argument('--limit',type=int,default=200)
19
+ p.add_argument('--device',default='cuda'); p.add_argument('--tile-size',type=int,default=128)
20
+ p.add_argument('--prune',type=float,default=1e-6); p.add_argument('--max-nodes',type=int,default=10000)
21
+ p.add_argument('--trunk-only',action='store_true'); p.add_argument('--assume-shared-prompt',action='store_true')
22
+ p.add_argument('--full-prefix',action='store_true',help='Disable KV reuse for a correctness comparison')
23
+ p.add_argument('--cache-depth',type=int,default=16)
24
+ args=p.parse_args(argv)
25
+ if args.limit<1 or args.tile_size<1: p.error('limit and tile size must be positive')
26
+ target=Path(args.output)
27
+ if target.exists(): raise FileExistsError(target)
28
+ models,toks=load_pair(args.student,args.teacher,args.device)
29
+ indices=[VocabIndex(t) for t in toks]
30
+ summaries=[[] for _ in range(24)]; dropped=0
31
+ target.parent.mkdir(parents=True,exist_ok=True)
32
+ with target.open('x') as stream,torch.no_grad():
33
+ for ri,record in enumerate(archive_rows(args.archive,args.limit)):
34
+ batch=tokenize_record(record,*toks,args.assume_shared_prompt)
35
+ grouped=row_groups(batch,*toks)[0]
36
+ if not grouped.valid: dropped+=1; continue
37
+ groups=[g for g in grouped.groups if g.kind=='mis']
38
+ scored=[]
39
+ for model,tok,vi,prefix in zip(models,toks,indices,('stu','tea')):
40
+ hidden=forward_hidden(model,batch,prefix)
41
+ labels=batch[prefix+'_input_ids'].roll(-1,1)[batch[prefix+'_loss_mask'].bool()].to(hidden.device)
42
+ head=model.get_output_embeddings()
43
+ chunks=[((g.s_idx if prefix=='stu' else g.t_idx),g.content) for g in groups]
44
+ # A plain HF head has no skip keyword, so detached scoring uses teacher mode.
45
+ scores=score_hidden_chunks(head,hidden,labels,chunks,vi,args.tile_size,student=False)
46
+ results=[]
47
+ positions=batch[prefix+'_loss_mask'][0].nonzero().flatten().tolist()
48
+ for g,lp in zip(groups,scores):
49
+ data=dict(path=float(lp.log_q_path),trunk=float(lp.log_p_trunk),
50
+ overhang=float((lp.log_overhang-lp.log_p_trunk).exp()))
51
+ if not args.trunk_only:
52
+ flat=(g.s_idx if prefix=='stu' else g.t_idx)[0]
53
+ base=batch[prefix+'_input_ids'][0,:positions[flat]+1].tolist()
54
+ oracle=HFPrefixOracle(model,base,args.cache_depth,not args.full_prefix)
55
+ bounds=exact_chunk_logprob(oracle,g.content,vi,args.prune,args.max_nodes)
56
+ data.update(bounds)
57
+ data['error_upper']=max(0.,bounds['log_probability_upper']-data['trunk'])
58
+ data['error_lower']=max(0.,bounds['log_probability_lower']-data['trunk'])
59
+ results.append(data)
60
+ scored.append(results)
61
+ for g,s,t in zip(groups,*scored):
62
+ bucket=bucket_of(g.content)
63
+ row=dict(response_index=ri,response_hash=hashlib.sha256(record['output'].encode()).hexdigest(),
64
+ bucket=bucket,b0=g.b0,b1=g.b1,chunk_hex=g.content.hex(),student=s,teacher=t)
65
+ row['gap']=(s['path']-t['path'])-(s['trunk']-t['trunk'])
66
+ summaries[bucket].append(row)
67
+ # Omitted mass zero is -inf internally; JSON uses null for log(0).
68
+ def finite(obj):
69
+ if isinstance(obj,dict): return {k:finite(v) for k,v in obj.items()}
70
+ return None if isinstance(obj,float) and not math.isfinite(obj) else obj
71
+ stream.write(json.dumps(finite(row),allow_nan=False)+'\n')
72
+ buckets=[]
73
+ for b,rows in enumerate(summaries):
74
+ scale=sum(abs(r['gap']) for r in rows)/max(len(rows),1)
75
+ errors={m:sum(r[m].get('error_upper',0.) for r in rows)/max(len(rows),1) for m in ('student','teacher')}
76
+ status=('not_measured' if args.trunk_only else 'insufficient' if not rows else
77
+ 'pass' if all(e<.1*scale for e in errors.values()) else 'fail')
78
+ buckets.append(dict(bucket=b,count=len(rows),mean_abs_gap=scale,error_upper_mean=errors,status=status))
79
+ summary=dict(source=source_identity(),archive_sha256=hashlib.sha256(Path(args.archive).read_bytes()).hexdigest(),
80
+ student=snapshot_identity(args.student),teacher=snapshot_identity(args.teacher),dropped_responses=dropped,buckets=buckets,
81
+ gate='not_measured' if args.trunk_only else 'pass' if all(b['status']=='pass' for b in buckets) else 'not_passed',
82
+ search_backend='full-prefix reference' if args.full_prefix else 'depth-bounded DFS KV stack',
83
+ qualification='Canonical-prefix conditional bounds; no real-model KV/B200 throughput qualification')
84
+ Path(str(target)+'.summary.json').write_text(json.dumps(summary,indent=2,allow_nan=False))
85
+
86
+
87
+ if __name__=='__main__': main()
tools/h1b/fit_cb.py ADDED
@@ -0,0 +1,113 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """c(B), split-half, SNR and group bootstrap from recorded CountSketch groups."""
2
+ import argparse
3
+ import json
4
+ from pathlib import Path
5
+ import numpy as np
6
+ import torch
7
+
8
+
9
+ def cosine(a,b):
10
+ den=np.linalg.norm(a)*np.linalg.norm(b)
11
+ return float(np.dot(a,b)/den) if den else None
12
+
13
+
14
+ def fit_curve(batch_sizes,values):
15
+ valid=[(b,c) for b,c in zip(batch_sizes,values) if c is not None]
16
+ if len(valid)<2: return dict(c_infinity=None,b_noise=None,identified=False)
17
+ x,y=np.array(valid).T
18
+ grid=np.r_[0.,np.geomspace(1e-3,1e9,2048)]
19
+ basis=1/np.sqrt(1+grid[:,None]/x)
20
+ c=np.clip((basis*y).sum(1)/(basis*basis).sum(1),-1,1)
21
+ error=((basis*c[:,None]-y)**2).sum(1)
22
+ k=int(error.argmin())
23
+ return dict(c_infinity=float(c[k]),b_noise=float(grid[k]),identified=k not in (0,len(grid)-1),squared_error=float(error[k]))
24
+
25
+
26
+ def analyze_matrices(strict,span,control0,control1,group_size=64,seed=42,repeats=20,weights=None):
27
+ """Small reference API, also used by tests. Undefined zero-vector cosines are null."""
28
+ rng=np.random.default_rng(seed); n=len(strict)
29
+ weights=np.ones(n) if weights is None else np.asarray(weights,dtype=float)
30
+ if np.any(weights<=0) or len(weights)!=n: raise ValueError('Positive group token weights required')
31
+ def pooled(matrix,ids): return np.average(matrix[ids],axis=0,weights=weights[ids])
32
+ sizes=[b for b in (64,128,256,512,1024,2048,4096) if b>=group_size and b%group_size==0 and b//group_size<=n]
33
+ rows=[]
34
+ for b in sizes:
35
+ m=b//group_size; cs=[]; ctrl=[]; halves=[]; ratios=[]; samples=[]
36
+ for _ in range(repeats):
37
+ ids=rng.choice(n,m,replace=False); s=pooled(strict,ids); v=pooled(span,ids)
38
+ cs.append(cosine(v,s)); ctrl.append(cosine(pooled(control0,ids),pooled(control1,ids)))
39
+ ratios.append(float(np.linalg.norm(v)/np.linalg.norm(s)) if np.linalg.norm(s) else None)
40
+ if m>=2:
41
+ h=m//2; halves.append(cosine(pooled(span,ids[:h]),pooled(span,ids[h:])))
42
+ samples.append(v)
43
+ sample=np.stack(samples[:16]); mu=sample.mean(0); noise=float(np.square(sample-mu).sum(1).mean())
44
+ if m==n: noise=0. # Resampling the entire fixed pool cannot identify sampling noise.
45
+ def avg(v):
46
+ v=[x for x in v if x is not None]
47
+ return float(np.mean(v)) if v else None
48
+ rows.append(dict(batch=b,cosine=avg(cs),control=avg(ctrl),split_half=avg(halves),norm_ratio=avg(ratios),
49
+ snr=float(np.dot(mu,mu)/noise) if noise else None,
50
+ snr_defined=bool(noise),split_half_responses=b//2 if m>=2 else None,
51
+ finite_population=True))
52
+ mean=np.average(span,axis=0,weights=weights); signal=float(np.dot(mean,mean)); noise=float(np.square(span-mean).sum(1).mean())
53
+ return dict(curve=rows,fit=fit_curve(sizes,[r['cosine'] for r in rows]),
54
+ group_noise_scale=group_size*noise/signal if signal else None)
55
+
56
+
57
+ def main(argv=None):
58
+ p=argparse.ArgumentParser(description=__doc__)
59
+ p.add_argument('--replay',required=True); p.add_argument('--output',required=True)
60
+ p.add_argument('--bootstrap',type=int,default=200); p.add_argument('--seed',type=int,default=42)
61
+ args=p.parse_args(argv)
62
+ root=Path(args.replay); meta=json.loads((root/'manifest.json').read_text())
63
+ if len(meta['records'])!=meta['groups']: raise ValueError('Incomplete replay manifest')
64
+ target=Path(args.output)
65
+ if target.exists(): raise FileExistsError(target)
66
+ def load(key):
67
+ return np.stack([torch.load(root/r['files'][key],map_location='cpu',weights_only=True).numpy() for r in meta['records']])
68
+ # Gram factorization reduces all subsequent resampling to <= 7*N dimensions;
69
+ # raw sketches are loaded one term at a time, not retained for bootstraps.
70
+ keys=list(meta['records'][0]['files'])
71
+ grams={}
72
+ for key in keys:
73
+ a=load(key)
74
+ for other in ('strict','strict_0','strict_1','path','str'):
75
+ b=a if key==other else load(other)
76
+ grams[(key,other)]=a@b.T
77
+ grams[(key,key)]=a@a.T
78
+ def factor(names):
79
+ block=np.block([[grams.get((a,b),grams.get((b,a),None).T if (b,a) in grams else None)
80
+ for b in names] for a in names])
81
+ val,vec=np.linalg.eigh((block+block.T)/2)
82
+ return np.split(vec*np.sqrt(np.maximum(val,0))[None,:],len(names))
83
+ variants={'path':('path',None),'str':('str',None),'path_bl':('path','base_path'),'str_bl':('str','base_str')}
84
+ for cls in meta.get('bucket_classes',[]):
85
+ variants.update({f'class{cls}_{k}':(f'class{cls}_{v[0]}',f'class{cls}_{v[1]}' if v[1] else None) for k,v in list(variants.items()) if not k.startswith('class')})
86
+ results={}; rng=np.random.default_rng(args.seed)
87
+ # Cross Grams involving baseline buffers are required for subtraction; compute
88
+ # only the pairs needed for each analysis, retaining small matrices.
89
+ for variant,(key,base) in variants.items():
90
+ names=['strict',key,'strict_0','strict_1']+([base] if base else [])
91
+ for a in names:
92
+ for b in names:
93
+ if (a,b) not in grams and (b,a) not in grams: grams[(a,b)]=load(a)@load(b).T
94
+ mat=factor(names); strict,span,c0,c1=mat[:4]
95
+ if base: span=span-mat[4]
96
+ weights=np.array([r['tokens'] for r in meta['records']])
97
+ result=analyze_matrices(strict,span,c0,c1,meta['group_size'],args.seed,weights=weights)
98
+ bootstrap=[]
99
+ for _ in range(args.bootstrap):
100
+ ids=rng.integers(0,len(strict),len(strict))
101
+ bootstrap.append(analyze_matrices(strict[ids],span[ids],c0[ids],c1[ids],meta['group_size'],int(rng.integers(2**31)),weights=weights[ids])['fit'])
102
+ result['bootstrap_intervals']={k:(np.quantile([b[k] for b in bootstrap if b[k] is not None],[.025,.975]).tolist()
103
+ if any(b[k] is not None for b in bootstrap) else None) for k in ('c_infinity','b_noise')}
104
+ results[variant]=result
105
+ names=['path','str']; a,b=factor(names)
106
+ output=dict(results=results,cos_mean_path_str=cosine(np.average(a,axis=0,weights=weights),np.average(b,axis=0,weights=weights)),
107
+ qualification='CountSketch estimates; group-bootstrap intervals; finite-pool SNR; undefined diagnostics are null',
108
+ strict_control='disjoint alternating strict positions within each microbatch',
109
+ missing='Adam-preconditioned analysis, fresh on-policy generation, end-task statistics are separate operations')
110
+ target.parent.mkdir(parents=True,exist_ok=True); target.write_text(json.dumps(output,indent=2,allow_nan=False))
111
+
112
+
113
+ if __name__=='__main__': main()
tools/h1b/grad_replay.py ADDED
@@ -0,0 +1,129 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Replay fixed archived responses through H1b; never generate new responses."""
2
+ import argparse
3
+ import hashlib
4
+ import json
5
+ from pathlib import Path
6
+ import types
7
+ import torch
8
+ from kdflow.algorithms.h1b_span import H1bSpanCTKD
9
+ from kdflow.arguments.distillation_args import DistillationArguments
10
+ from .common import archive_rows,load_pair,tokenize_record,forward_hidden,StudentWrapper,snapshot_identity,source_identity
11
+ from .sketch import count_sketch
12
+
13
+
14
+ def collate(batches):
15
+ from torch.nn.utils.rnn import pad_sequence
16
+ return {k:pad_sequence([b[k][0] for b in batches],batch_first=True) for k in batches[0]}
17
+
18
+
19
+ def main(argv=None):
20
+ p=argparse.ArgumentParser(description=__doc__)
21
+ for name in ('archive','student','teacher','output'): p.add_argument('--'+name,required=True)
22
+ p.add_argument('--limit',type=int,default=4096); p.add_argument('--group-size',type=int,default=64)
23
+ p.add_argument('--micro-batch',type=int,default=4); p.add_argument('--tile-size',type=int,default=128)
24
+ p.add_argument('--dimension',type=int,default=2**22); p.add_argument('--seed',type=int,default=42)
25
+ p.add_argument('--device',default='cuda'); p.add_argument('--buffer-device',default='cuda')
26
+ p.add_argument('--exact-groups',type=int,default=2)
27
+ p.add_argument('--buckets',default='',help='Optional byte class IDs 0=digits,3=letters; adds gradient buffers')
28
+ p.add_argument('--assume-shared-prompt',action='store_true')
29
+ args=p.parse_args(argv)
30
+ if min(args.limit,args.group_size,args.micro_batch,args.tile_size,args.dimension)<1:
31
+ p.error('Sizes must be positive')
32
+ if args.limit%args.group_size or args.group_size%args.micro_batch:
33
+ p.error('Require complete equal groups and microbatches')
34
+ root=Path(args.output)
35
+ if root.exists(): raise FileExistsError(root)
36
+ records=list(archive_rows(args.archive,args.limit))
37
+ if len(records)!=args.limit: raise ValueError('Archive has fewer responses than requested')
38
+ selected=[int(s) for s in args.buckets.split(',') if s.strip()]
39
+ if any(not 0<=s<=5 for s in selected): raise ValueError('Byte class must be in 0..5')
40
+ models,toks=load_pair(args.student,args.teacher,args.device)
41
+ student,teacher=models
42
+ teacher.requires_grad_(False)
43
+ wrapper=StudentWrapper(student)
44
+ strategy=types.SimpleNamespace(args=types.SimpleNamespace(
45
+ kd=DistillationArguments(kd_algorithm='h1b_span',kd_ratio=1,kd_temperature=1,kd_loss_fn='rkl'),
46
+ model=types.SimpleNamespace(ring_attn_size=1),
47
+ train=types.SimpleNamespace(chunked_loss_size=args.tile_size,num_nodes=1,num_gpus_per_node=1,save_path=None),
48
+ scenario='on_policy_kd'),ring_attn_group=None)
49
+ algo=H1bSpanCTKD(strategy,wrapper,teacher.get_output_embeddings(),*toks)
50
+ algo.replay_enabled=True
51
+ algo.replay_control=False
52
+ wrapper.eval()
53
+ config=student.config
54
+ if any(getattr(config,k,0) for k in ('attention_dropout','hidden_dropout','hidden_dropout_prob')):
55
+ raise ValueError('Deterministic checkpointed replay requires zero model dropout')
56
+ student.gradient_checkpointing_enable(gradient_checkpointing_kwargs={'use_reentrant':False})
57
+ student.base_model.train() # Enables checkpointing; no dropout for qualified pair.
58
+ encoded=[tokenize_record(r,*toks,args.assume_shared_prompt) for r in records]
59
+ ngroups=args.limit//args.group_size
60
+ sums=torch.zeros(ngroups,2,24,dtype=torch.float64); counts=torch.zeros_like(sums)
61
+ def batch_at(start,denominator):
62
+ batch=collate(encoded[start:start+args.micro_batch])
63
+ with torch.no_grad(): batch['teacher_hiddens']=forward_hidden(teacher,batch,'tea')
64
+ for k,v in batch.items(): batch[k]=v.to(args.device)
65
+ batch['avg_micro_batch_token_num']=denominator
66
+ return batch
67
+ # First pass obtains all group bucket means; replay never uses its own group
68
+ # to construct the leave-group-out baseline.
69
+ with torch.no_grad():
70
+ for i in range(0,args.limit,args.micro_batch):
71
+ algo.training_step(batch_at(i,1.))
72
+ gi=i//args.group_size
73
+ for record in algo.replay_records:
74
+ b=record['bucket']
75
+ for kind,key in enumerate(('path','str')):
76
+ sums[gi,kind,b]+=float(record[key]); counts[gi,kind,b]+=1
77
+ if ngroups<2: raise ValueError('Leave-group-out replay needs at least two groups')
78
+ algo.replay_control=True
79
+ root.mkdir(parents=True,exist_ok=False)
80
+ named=list(wrapper.named_parameters()); params=tuple(v for _,v in named)
81
+ offsets=[0]
82
+ for _,param in named: offsets.append(offsets[-1]+param.numel())
83
+ keys=['strict','path','str','base_path','base_str','strict_0','strict_1']
84
+ for b in selected:
85
+ keys.extend(f'class{b}_{term}' for term in ('path','str','base_path','base_str'))
86
+ manifest=dict(version=1,source=source_identity(),archive_sha256=hashlib.sha256(Path(args.archive).read_bytes()).hexdigest(),
87
+ student=snapshot_identity(args.student),teacher=snapshot_identity(args.teacher),group_size=args.group_size,groups=ngroups,
88
+ dimension=args.dimension,seed=args.seed,normalization='sum token-normalized microbatch gradients per group',
89
+ baseline='leave-group-out bucket means; outcome-dependent bucket caveat applies',
90
+ preconditioner='raw gradients only',bucket_classes=selected,records=[])
91
+ for gi in range(ngroups):
92
+ other_s=sums.sum(0)-sums[gi]; other_n=counts.sum(0)-counts[gi]
93
+ algo.baseline.prev=torch.where(other_n>0,other_s/other_n.clamp_min(1),torch.zeros_like(other_s))
94
+ # Deliberately explicit memory contract: seven full FP32 buffers by
95
+ # default. Probe with --limit 128 before the 4096-response replay.
96
+ buffers={k:torch.zeros(offsets[-1],dtype=torch.float32,device=args.buffer_device) for k in keys}
97
+ lo=gi*args.group_size
98
+ tokens=sum(int(b['stu_loss_mask'].sum()) for b in encoded[lo:lo+args.group_size])
99
+ for i in range(lo,lo+args.group_size,args.micro_batch):
100
+ algo.training_step(batch_at(i,max(tokens,1)))
101
+ terms=dict(algo.replay_terms)
102
+ for b in selected:
103
+ chunk_records=[r for r in algo.replay_records if r['bucket']//4==b]
104
+ for term in ('path','str','base_path','base_str'):
105
+ zero=terms['strict']*0
106
+ key=term.removeprefix('base_')
107
+ terms[f'class{b}_{term}']=sum((2*algo.lam*(algo.baseline.lookup(r['bucket'],key)
108
+ if term.startswith('base_') else r[term])*r['log_q']/max(tokens,1) for r in chunk_records),zero)
109
+ for ki,key in enumerate(keys):
110
+ grads=torch.autograd.grad(terms[key],params,retain_graph=ki<len(keys)-1,allow_unused=True)
111
+ for pi,grad in enumerate(grads):
112
+ if grad is not None: buffers[key][offsets[pi]:offsets[pi+1]].add_(grad.detach().flatten().to(args.buffer_device))
113
+ files={}; norms={}
114
+ for key,flat in buffers.items():
115
+ tensors=((name,flat[offsets[j]:offsets[j+1]]) for j,(name,_) in enumerate(named))
116
+ sketch,norm=count_sketch(tensors,args.dimension,args.seed)
117
+ file=f'group_{gi:03d}_{key}.pt'
118
+ torch.save(sketch,root/file); files[key]=file; norms[key]=norm
119
+ if gi<args.exact_groups:
120
+ # Save one buffer at a time; the validator memory-maps these.
121
+ torch.save(flat.cpu(),root/f'group_{gi:03d}_{key}_exact.pt')
122
+ manifest['records'].append(dict(group=gi,tokens=tokens,files=files,exact_norms=norms,
123
+ bucket_sums=sums[gi].tolist(),bucket_counts=counts[gi].tolist()))
124
+ (root/'manifest.json').write_text(json.dumps(manifest,indent=2))
125
+ del buffers
126
+ print(json.dumps(dict(groups=ngroups,output=str(root),qualification='raw fixed-text replay; GPU execution must be independently qualified')))
127
+
128
+
129
+ if __name__=='__main__': main()
tools/h1b/sketch.py ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Streaming CountSketch keyed by tensor name, independent of chunk boundaries."""
2
+ import hashlib
3
+ import torch
4
+
5
+
6
+ def count_sketch(named_tensors,dimension=2**22,seed=42,chunk_size=2**20):
7
+ if dimension<1 or chunk_size<1: raise ValueError('Positive sketch dimensions required')
8
+ output=torch.zeros(dimension,dtype=torch.float32)
9
+ norm2=0.
10
+ prime=2147483647
11
+ for name,tensor in named_tensors:
12
+ flat=tensor.detach().reshape(-1)
13
+ digest=hashlib.sha256((str(seed)+':'+name).encode()).digest()
14
+ a,b,c,d=[int.from_bytes(digest[i:i+8],'little')%(prime-1)+1 for i in (0,8,16,24)]
15
+ for start in range(0,flat.numel(),chunk_size):
16
+ values=flat[start:start+chunk_size].float().cpu()
17
+ indices=torch.arange(start,start+len(values),dtype=torch.int64)
18
+ bucket=((a*indices+b)%prime)%dimension
19
+ signs=(((c*indices+d)%prime)%2)*2-1
20
+ output.scatter_add_(0,bucket,values*signs)
21
+ norm2+=float(values.double().square().sum())
22
+ return output,norm2**.5
tools/h1b/validate_alignment.py ADDED
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Real-tokenizer T1/T3/T9 gates; requires local tokenizer snapshots and archived text."""
2
+ import argparse
3
+ import hashlib
4
+ import json
5
+ from pathlib import Path
6
+ from kdflow.token_alignment import get_tokenizer_byte_codec,align_byte_span_batch_1to1
7
+ from kdflow.algorithms._mp_opd_port._mp_opd_atoms import SimCTAtomizer
8
+ from .common import archive_rows,tokenize_record,row_groups
9
+
10
+
11
+ def main(argv=None):
12
+ p=argparse.ArgumentParser(description=__doc__)
13
+ for key in ('student','teacher','archive','texts','output'): p.add_argument('--'+key,required=True)
14
+ p.add_argument('--assume-shared-prompt',action='store_true')
15
+ args=p.parse_args(argv)
16
+ target=Path(args.output)
17
+ if target.exists(): raise FileExistsError(target)
18
+ from transformers import AutoTokenizer
19
+ toks=[AutoTokenizer.from_pretrained(s,local_files_only=True) for s in (args.student,args.teacher)]
20
+ codecs=[get_tokenizer_byte_codec(t) for t in toks]
21
+ texts=Path(args.texts).read_text().splitlines()
22
+ if len(texts)<10000: raise ValueError('T1 requires at least 10,000 texts')
23
+ # User-supplied corpus must include challenging scripts, not just 10k copies
24
+ # of one ASCII fixture. This list is supplemental and reported separately.
25
+ probes=['Tiếng Việt: một phép thử','中文 日本語 한국어','🙂🚀','\\frac{120}{3} = 40','def f(x):\n return x+1','a\t b\n']
26
+ failures=[]
27
+ for model,tok,codec in zip(('student','teacher'),toks,codecs):
28
+ for i,text in enumerate(texts+probes):
29
+ ids=tok(text,add_special_tokens=False)['input_ids']
30
+ raw=b''.join(codec.piece_to_bytes(p) for p in tok.convert_ids_to_tokens(ids))
31
+ if raw!=text.encode(): failures.append(dict(test='T1',model=model,index=i))
32
+ records=list(archive_rows(args.archive,500))
33
+ if len(records)!=500: raise ValueError('T3/T9 require 500 real responses')
34
+ atomizer=SimCTAtomizer(toks[0],toks[1])
35
+ common=0; dropped=0
36
+ for ri,record in enumerate(records):
37
+ batch=tokenize_record(record,*toks,args.assume_shared_prompt)
38
+ grouped=row_groups(batch,*toks)[0]
39
+ sid=batch['stu_input_ids'].roll(-1,1)[batch['stu_loss_mask'].bool()].tolist()
40
+ tid=batch['tea_input_ids'].roll(-1,1)[batch['tea_loss_mask'].bool()].tolist()
41
+ st,tt=toks[0].convert_ids_to_tokens(sid),toks[1].convert_ids_to_tokens(tid)
42
+ strict=align_byte_span_batch_1to1([tt],[st],toks[1].eos_token,toks[0].eos_token,
43
+ codecs[1].piece_to_bytes,codecs[0].piece_to_bytes)
44
+ actual=([g.t_idx[0] for g in grouped.groups if g.kind=='strict'],[g.s_idx[0] for g in grouped.groups if g.kind=='strict'])
45
+ if actual!=strict:
46
+ failures.append(dict(test='T3',response=ri,reason=grouped.reason or 'strict_set_difference',author_indices=strict,h1b_indices=actual))
47
+ atomized=atomizer.atomize(sid,tid,sample_id=str(ri))
48
+ if not grouped.valid or not atomized.valid: dropped+=1; continue
49
+ common+=1
50
+ # EOS is a Strict-only synchronization event; compare content spans.
51
+ pairs=[(g.s_idx[0],g.s_idx[-1]+1,g.t_idx[0],g.t_idx[-1]+1,g.kind) for g in grouped.groups if not g.eos]
52
+ reference=[(a.student_start,a.student_end,a.teacher_start,a.teacher_end,
53
+ 'strict' if a.student_end-a.student_start==a.teacher_end-a.teacher_start==1 else 'mis') for a in atomized.atoms]
54
+ if pairs!=reference: failures.append(dict(test='T9',response=ri,h1b=pairs,mp_opd=reference))
55
+ out=dict(text_count=len(texts),supplemental_probes=len(probes),responses=500,common_accepted=common,
56
+ excluded_from_common=dropped,disagreements=failures,passed=not failures,
57
+ archive_sha256=hashlib.sha256(Path(args.archive).read_bytes()).hexdigest(),
58
+ qualification='Real-input alignment gate only; no GPU training')
59
+ target.parent.mkdir(parents=True,exist_ok=True); target.write_text(json.dumps(out,indent=2))
60
+ if failures: raise SystemExit('H1b alignment gate has disagreements; see report')
61
+
62
+
63
+ if __name__=='__main__': main()
tools/h1b/validate_sketch.py ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Validate saved group 0/1 sketches against memory-mapped exact gradients (T11)."""
2
+ import argparse
3
+ import json
4
+ from pathlib import Path
5
+ import math
6
+ import torch
7
+
8
+
9
+ def compare(root,manifest,chunk_size=2**20):
10
+ results={}; passed=True
11
+ for key in manifest['records'][0]['files']:
12
+ exact=[torch.load(root/f'group_{i:03d}_{key}_exact.pt',map_location='cpu',weights_only=True,mmap=True) for i in (0,1)]
13
+ sketch=[torch.load(root/manifest['records'][i]['files'][key],map_location='cpu',weights_only=True) for i in (0,1)]
14
+ aa=bb=ab=0.
15
+ if exact[0].shape!=exact[1].shape: raise ValueError('Exact gradient shapes differ')
16
+ for start in range(0,exact[0].numel(),chunk_size):
17
+ a,b=(x[start:start+chunk_size].double() for x in exact)
18
+ aa+=float(a.square().sum()); bb+=float(b.square().sum()); ab+=float((a*b).sum())
19
+ norms=[math.sqrt(aa),math.sqrt(bb)]
20
+ sn=[float(x.double().norm()) for x in sketch]
21
+ errors=[abs(s/n-1) if n else 0. if s==0 else math.inf for s,n in zip(sn,norms)]
22
+ ec=ab/math.sqrt(aa*bb) if aa*bb else None
23
+ sc=float(torch.nn.functional.cosine_similarity(sketch[0].double(),sketch[1].double(),dim=0)) if sn[0]*sn[1] else None
24
+ ce=abs(ec-sc) if ec is not None and sc is not None else None
25
+ ok=all(e<.01 for e in errors) and (ce is None or ce<.01)
26
+ passed=passed and ok
27
+ results[key]=dict(norm_relative_errors=errors,exact_cosine=ec,sketch_cosine=sc,cosine_abs_error=ce,passed=ok)
28
+ return dict(passed=passed,terms=results,zero_vectors='cosine undefined; reported null')
29
+
30
+
31
+ def main(argv=None):
32
+ p=argparse.ArgumentParser(description=__doc__); p.add_argument('--replay',required=True); p.add_argument('--output',required=True)
33
+ args=p.parse_args(argv); root=Path(args.replay); target=Path(args.output)
34
+ if target.exists(): raise FileExistsError(target)
35
+ meta=json.loads((root/'manifest.json').read_text())
36
+ result=compare(root,meta); target.write_text(json.dumps(result,indent=2,allow_nan=False))
37
+ if not result['passed']: raise SystemExit('Sketch accuracy gate failed')
38
+
39
+
40
+ if __name__=='__main__': main()
tools/launch_single_b200.py CHANGED
@@ -17,6 +17,7 @@ import subprocess
17
 
18
  SOURCE = pathlib.Path(__file__).resolve().parents[1]
19
  ARCHIVE_SHA = '084c321c0b7af1cd3a36d12a595c623567d4a37525aa0eecbc3933aa7f9b15c3'
 
20
 
21
 
22
  def parser():
@@ -28,7 +29,7 @@ def parser():
28
  p.add_argument('--algorithm', choices=['strict-top16', 'strict-full', 'simct', 'simct_paper', 'mp-atomic',
29
  'mp-fixed1', 'mp-fixed2', 'mp-fixed3', 'mp-fixed4', 'mp-fixed5',
30
  'mp-random1-2', 'mp-random2-5', 'mp-grass-chunk', 'mp-align', 'mp-align-new',
31
- 'mp-trust-r','mp-trust-b','mp-trust-p'], default='strict-top16')
32
  p.add_argument('--mp-chunk-source', choices=['xtoken', 'run'], default='xtoken')
33
  p.add_argument('--mp-chunk-run-length', type=int, default=2)
34
  p.add_argument('--mp-geometry', choices=['exact_head', 'diag'], default='exact_head')
@@ -103,6 +104,9 @@ def build_plan(args):
103
  flags.update(mp_opd_align_new_min_descent_fraction=0.1, mp_opd_align_new_eps_g=1e-12)
104
  if family.startswith('trust_'):
105
  flags.update(mp_opd_grass_geometry='exact_head',mp_opd_trust_eps_g=1e-12,mp_opd_trust_vocab_chunk=4096)
 
 
 
106
  command = ['bash', str(runtime / 'python-b200.sh'), '-u', '-m', 'kdflow.cli.train_kd_on_policy']
107
  for key, value in flags.items():
108
  command.extend(['--' + key, str(value)])
@@ -110,7 +114,9 @@ def build_plan(args):
110
  'source': str(SOURCE), 'flags': flags, 'command': command,
111
  'gradient_accumulation_microsteps': 512 // args.micro_batch,
112
  'expected_runtime_archive_sha256': ARCHIVE_SHA,
113
- 'qualification': ('paper-score SimCT reconstruction; no B200/full-training qualification'
 
 
114
  if args.algorithm == 'simct_paper' else
115
  'author-code SimCT port; no B200/full-training qualification'
116
  if args.algorithm == 'simct' else
 
17
 
18
  SOURCE = pathlib.Path(__file__).resolve().parents[1]
19
  ARCHIVE_SHA = '084c321c0b7af1cd3a36d12a595c623567d4a37525aa0eecbc3933aa7f9b15c3'
20
+ H1B_ARMS={'h1b-a':'a','h1b-b':'b','h1b-b-bl':'b_bl','h1b-c-str':'c_str','h1b-c':'c'}
21
 
22
 
23
  def parser():
 
29
  p.add_argument('--algorithm', choices=['strict-top16', 'strict-full', 'simct', 'simct_paper', 'mp-atomic',
30
  'mp-fixed1', 'mp-fixed2', 'mp-fixed3', 'mp-fixed4', 'mp-fixed5',
31
  'mp-random1-2', 'mp-random2-5', 'mp-grass-chunk', 'mp-align', 'mp-align-new',
32
+ 'mp-trust-r','mp-trust-b','mp-trust-p',*H1B_ARMS], default='strict-top16')
33
  p.add_argument('--mp-chunk-source', choices=['xtoken', 'run'], default='xtoken')
34
  p.add_argument('--mp-chunk-run-length', type=int, default=2)
35
  p.add_argument('--mp-geometry', choices=['exact_head', 'diag'], default='exact_head')
 
104
  flags.update(mp_opd_align_new_min_descent_fraction=0.1, mp_opd_align_new_eps_g=1e-12)
105
  if family.startswith('trust_'):
106
  flags.update(mp_opd_grass_geometry='exact_head',mp_opd_trust_eps_g=1e-12,mp_opd_trust_vocab_chunk=4096)
107
+ if args.algorithm in H1B_ARMS:
108
+ flags.update(kd_algorithm='h1b_span',h1b_arm=H1B_ARMS[args.algorithm],h1b_span_lambda=.5,
109
+ h1b_log_sample_rate=.05,h1b_dump_every='0,20,60,100',top_p=1.,eval_math_top_p=.95)
110
  command = ['bash', str(runtime / 'python-b200.sh'), '-u', '-m', 'kdflow.cli.train_kd_on_policy']
111
  for key, value in flags.items():
112
  command.extend(['--' + key, str(value)])
 
114
  'source': str(SOURCE), 'flags': flags, 'command': command,
115
  'gradient_accumulation_microsteps': 512 // args.micro_batch,
116
  'expected_runtime_archive_sha256': ARCHIVE_SHA,
117
+ 'qualification': ('experimental H1b span arms; trunk gate pending; no B200/full-training qualification'
118
+ if args.algorithm in H1B_ARMS else
119
+ 'paper-score SimCT reconstruction; no B200/full-training qualification'
120
  if args.algorithm == 'simct_paper' else
121
  'author-code SimCT port; no B200/full-training qualification'
122
  if args.algorithm == 'simct' else