feat(h1b): add span factorial arms and offline probability replay tools
Browse files- README.md +5 -0
- docs/H1B.md +220 -0
- docs/H1B_SOURCE.json +121 -0
- docs/H1B_VALIDATION.json +54 -0
- kdflow/algorithms/__init__.py +1 -0
- kdflow/algorithms/h1b_span.py +165 -0
- kdflow/arguments/distillation_args.py +21 -2
- kdflow/arguments/rollout_args.py +5 -0
- kdflow/cli/train_kd_on_policy.py +2 -1
- kdflow/evaluation/math_evaluator.py +4 -1
- kdflow/h1b/__init__.py +1 -0
- kdflow/h1b/baseline.py +30 -0
- kdflow/h1b/buckets.py +14 -0
- kdflow/h1b/chunklog.py +68 -0
- kdflow/h1b/exact.py +38 -0
- kdflow/h1b/groups.py +72 -0
- kdflow/h1b/strprob.py +87 -0
- kdflow/h1b/vocab_index.py +45 -0
- kdflow/ray/train/student_actor.py +7 -1
- kdflow/trainer/data_processor/rollout_data_processor.py +10 -1
- tests/test_h1b.py +347 -0
- tools/h1b/__init__.py +1 -0
- tools/h1b/canary.py +67 -0
- tools/h1b/common.py +138 -0
- tools/h1b/enrich_archive.py +54 -0
- tools/h1b/exact_strprob.py +87 -0
- tools/h1b/fit_cb.py +113 -0
- tools/h1b/grad_replay.py +129 -0
- tools/h1b/sketch.py +22 -0
- tools/h1b/validate_alignment.py +63 -0
- tools/h1b/validate_sketch.py +40 -0
- tools/launch_single_b200.py +8 -2
README.md
CHANGED
|
@@ -32,6 +32,11 @@ Separate [TRUST-R/B/P and SimCT paper-score modes](docs/TRUST_PAPER.md) provide
|
|
| 32 |
optimizer window; `simct_paper` corrects log-probability scores while `simct` stays
|
| 33 |
the released author-code baseline.
|
| 34 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
## Original project documentation
|
| 36 |
|
| 37 |
This repository provides the implementation and example training scripts for `byte_align_ctkd` and `topk_byte_align_ctkd`.
|
|
|
|
| 32 |
optimizer window; `simct_paper` corrects log-probability scores while `simct` stays
|
| 33 |
the released author-code baseline.
|
| 34 |
|
| 35 |
+
An opt-in [H1b span experiment](docs/H1B.md) adds five factorial presets,
|
| 36 |
+
`h1b-a`, `h1b-b`, `h1b-b-bl`, `h1b-c-str`, `h1b-c`, plus offline trunk/exact
|
| 37 |
+
scoring and gradient replay. Real-input and B200 gates remain separate from CPU
|
| 38 |
+
checks; see the contract's approximation and baseline caveats before training.
|
| 39 |
+
|
| 40 |
## Original project documentation
|
| 41 |
|
| 42 |
This repository provides the implementation and example training scripts for `byte_align_ctkd` and `topk_byte_align_ctkd`.
|
docs/H1B.md
ADDED
|
@@ -0,0 +1,220 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# H1b: path versus trunk span supervision
|
| 2 |
+
|
| 3 |
+
Implementation inputs: Tung's `H1b Implementation Plan Bias vs Variance in Span
|
| 4 |
+
Supervision.md` (19 October 2026), and the companion Experiment Design (18 October
|
| 5 |
+
2026). The implementation plan takes precedence where the two differ. Baseline
|
| 6 |
+
source: `0eeed4ca2b0d7b07dd50076e35d81f030a3b9f2d`.
|
| 7 |
+
|
| 8 |
+
## Contract
|
| 9 |
+
|
| 10 |
+
`h1b_span` subclasses ByteAlignCrossTokenizerKD, with five isolated launcher
|
| 11 |
+
presets. The shared Strict files, byte codec and loss kernels remain byte-identical
|
| 12 |
+
to the baseline; see `H1B_SOURCE.json`. Existing presets keep their defaults.
|
| 13 |
+
|
| 14 |
+
| Preset | Arm | Detached span coefficient multiplying log q_student_path |
|
| 15 |
+
|---|---|---|
|
| 16 |
+
| h1b-a | a | zero |
|
| 17 |
+
| h1b-b | b | 2 lambda delta_path |
|
| 18 |
+
| h1b-b-bl | b_bl | 2 lambda (delta_path - previous_bucket_mean_path) |
|
| 19 |
+
| h1b-c-str | c_str | 2 lambda delta_trunk |
|
| 20 |
+
| h1b-c | c | 2 lambda (delta_trunk - previous_bucket_mean_trunk) |
|
| 21 |
+
|
| 22 |
+
Lambda is 0.5. Both strict and span sums use the existing common optimizer-window
|
| 23 |
+
token denominator. The baseline is frozen during each complete fit window, then
|
| 24 |
+
updated after the optimizer step. Evaluation/no-grad does not update it or window
|
| 25 |
+
statistics. State exports contain previous bucket means, arm/lambda and window
|
| 26 |
+
index, **not** a complete optimizer/RNG/training resume checkpoint.
|
| 27 |
+
|
| 28 |
+
The workload is Qwen2.5-7B-Instruct -> Llama-3.2-3B-Instruct, math_code_20k,
|
| 29 |
+
512-response rollout/train batches, microbatch 4 with 128 accumulation microsteps,
|
| 30 |
+
8,192 generated tokens, 100 updates, and MATH500 mean@32 every 10 updates. H1b
|
| 31 |
+
rollouts use top-p 1.0; MATH evaluation independently retains top-p 0.95. There is
|
| 32 |
+
one student rank with ring size 1, no final-logit softcap, and no multirank claim.
|
| 33 |
+
The optimizer and other flags are inherited unchanged from strict-full.
|
| 34 |
+
|
| 35 |
+
## Byte groups and probability semantics
|
| 36 |
+
|
| 37 |
+
Groups use Strict's token-piece bytes and EOS segmentation. Equal complete bytes
|
| 38 |
+
are partitioned at shared cumulative byte ends. Content 1:1 groups and paired EOS
|
| 39 |
+
feed the original overlap-vocabulary RKL/chunked_loss. Nonempty 1:N/N:1/N:M groups
|
| 40 |
+
feed the span term; EOS is excluded. Responses with byte/segment mismatch, empty
|
| 41 |
+
pieces or unsupported special IDs are excluded as a whole from both terms and
|
| 42 |
+
counted. Offsets and the original denominator include excluded tokens.
|
| 43 |
+
|
| 44 |
+
**Parity boundary:** arm a has tested loss/all student-gradient parity with Strict
|
| 45 |
+
on accepted fixture rows. Strict can still keep a paired EOS when a content segment
|
| 46 |
+
has mismatched bytes; H1b's planned whole-response exclusion intentionally removes
|
| 47 |
+
that EOS too. Empty-piece/segment rejection is also stricter. Universal equality
|
| 48 |
+
on rejected responses is not claimed. Real-response differences are reported by
|
| 49 |
+
the alignment validator rather than silently passed.
|
| 50 |
+
|
| 51 |
+
The trunk probability is the sum of disjoint events: follow the canonical path
|
| 52 |
+
for j tokens, then finish or overhang the chunk in one token. It includes the full
|
| 53 |
+
canonical path and is a lower bound on the conditional byte-prefix event. All
|
| 54 |
+
duplicate byte-equivalent token IDs are retained in covers and exact branches.
|
| 55 |
+
Content paths must spell the group bytes exactly. Full-vocabulary head logits are
|
| 56 |
+
tiled; only selected path logps retain gradients. Detached cover/overhang masses
|
| 57 |
+
are reduced inside checkpointed student tiles; teacher scoring is no-grad.
|
| 58 |
+
|
| 59 |
+
This does **not** establish unbiased string-KL training. Remaining confounds are:
|
| 60 |
+
|
| 61 |
+
- Canonical prefix conditioning and outcome-dependent synchronization chunks.
|
| 62 |
+
- Re-encoding decoded rollout text instead of preserving the sampled token path;
|
| 63 |
+
this is the implementation plan's choice, shared across arms. H1b logs the raw
|
| 64 |
+
response re-encoding match rate. Top-p 1.0 alone does not eliminate this issue.
|
| 65 |
+
- A baseline indexed by the sampled chunk's type/length is outcome-dependent even
|
| 66 |
+
when its means come from the previous window. Previous-window separation removes
|
| 67 |
+
same-sample mean leakage, not all possible score-function bias.
|
| 68 |
+
- Whole-response exclusion changes the effective training distribution.
|
| 69 |
+
|
| 70 |
+
These are ablations of the specified estimators, not proof of the Design's
|
| 71 |
+
unbiasedness claims or a complete reproduction of Hou's unreleased span ablation.
|
| 72 |
+
|
| 73 |
+
## Logging and replay provenance
|
| 74 |
+
|
| 75 |
+
Every arm computes both targets, both baselines and the same diagnostics. Window
|
| 76 |
+
moments weight chunks directly, not averages of microbatch means. JSON summaries
|
| 77 |
+
include all 24 buckets, counts and flags for undefined correlations/bias-floor
|
| 78 |
+
subsets. Scalar logs use finite zero plus counts/defined flags for empty subsets.
|
| 79 |
+
Per-chunk parquet uses null for absent overhang log-mass (log(0)); it never stores
|
| 80 |
+
raw prompts, only response hash, chunk hex and diagnostics.
|
| 81 |
+
|
| 82 |
+
Files live at `checkpoints/h1b/step_K.json` and `chunks_step_K.parquet`. K is the
|
| 83 |
+
zero-based pre-update model/window index: 100 updates produce K=0..99. Full dumps
|
| 84 |
+
are requested at 0/20/60/100; index 100 is an **offline final-checkpoint scoring**
|
| 85 |
+
operation, not a fictitious 101st training window.
|
| 86 |
+
The trainer's first training archive is `rollout_data/1.jsonl` (pre-update model
|
| 87 |
+
index 0); it does not write a training `rollout_data/0.jsonl`.
|
| 88 |
+
Sampling is deterministic from
|
| 89 |
+
window/response hash/span and does not advance the training RNG. H1b rollout JSONL
|
| 90 |
+
adds the teacher prompt and sampled output IDs so offline tokenization can reuse
|
| 91 |
+
the production method. Older archives lack that prompt; tools require explicit
|
| 92 |
+
`--assume-shared-prompt` if the caller has independently established equality.
|
| 93 |
+
|
| 94 |
+
Offline tools require existing local snapshots and `local_files_only=True`; they
|
| 95 |
+
never download models or regenerate responses. Reports include archive SHA-256
|
| 96 |
+
and model/tokenizer metadata hashes; these metadata hashes do not authenticate
|
| 97 |
+
every weight shard. Keep the model/checkpoint provenance alongside each report.
|
| 98 |
+
|
| 99 |
+
## Commands (company node; existing runtime)
|
| 100 |
+
|
| 101 |
+
From the committed source root, with the allocated GPU exposed explicitly:
|
| 102 |
+
|
| 103 |
+
```bash
|
| 104 |
+
export PYTHONPATH="$PWD"
|
| 105 |
+
RUNTIME=/opt/ctopd-b200-u22
|
| 106 |
+
ROOT=/workspace/storage-shared/nlp/tungks/ctopd_all
|
| 107 |
+
S="$ROOT/models/Llama-3.2-3B-Instruct"
|
| 108 |
+
T="$ROOT/models/Qwen2.5-7B-Instruct"
|
| 109 |
+
python3 tools/launch_single_b200.py --algorithm h1b-c --runtime "$RUNTIME" --dry-run
|
| 110 |
+
```
|
| 111 |
+
|
| 112 |
+
If an old Strict archive lacks `tea_prompt`, recover it without regenerating any
|
| 113 |
+
responses. This uses the production PromptDataset template builder on the original
|
| 114 |
+
parquet pool and demands an exact, unambiguous student-prompt match. It preserves
|
| 115 |
+
finish reasons and never fabricates missing sampled IDs:
|
| 116 |
+
|
| 117 |
+
```bash
|
| 118 |
+
bash "$RUNTIME/python-b200.sh" -m tools.h1b.enrich_archive \
|
| 119 |
+
--archive "$OLD_ARCHIVE" --dataset "$ROOT/data/Train/math_code_20k/train.parquet" \
|
| 120 |
+
--student "$S" --teacher "$T" --output "$ROOT/h1b-enriched.jsonl"
|
| 121 |
+
ARCHIVE="$ROOT/h1b-enriched.jsonl"
|
| 122 |
+
```
|
| 123 |
+
|
| 124 |
+
The five-step canary retains the original launcher's initial MATH500 evaluation;
|
| 125 |
+
previewing a plan consumes no GPU, whereas executing it includes that evaluation.
|
| 126 |
+
|
| 127 |
+
T1/T3/T9 real-input validation (TEXTS must contain at least 10,000 texts, ARCHIVE
|
| 128 |
+
at least 500 real responses, with the production prompt/truncation metadata):
|
| 129 |
+
|
| 130 |
+
```bash
|
| 131 |
+
bash "$RUNTIME/python-b200.sh" -m tools.h1b.validate_alignment \
|
| 132 |
+
--student "$S" --teacher "$T" --texts "$TEXTS" --archive "$ARCHIVE" \
|
| 133 |
+
--output "$ROOT/h1b-alignment.json"
|
| 134 |
+
```
|
| 135 |
+
|
| 136 |
+
M2 trunk table, followed by M3 bounded exhaustive checking on 200 responses:
|
| 137 |
+
|
| 138 |
+
```bash
|
| 139 |
+
bash "$RUNTIME/python-b200.sh" -m tools.h1b.exact_strprob \
|
| 140 |
+
--student "$S" --teacher "$T" --archive "$ARCHIVE" --limit 512 \
|
| 141 |
+
--trunk-only --output "$ROOT/h1b-trunk.jsonl"
|
| 142 |
+
bash "$RUNTIME/python-b200.sh" -m tools.h1b.exact_strprob \
|
| 143 |
+
--student "$S" --teacher "$T" --archive "$ARCHIVE" --limit 200 \
|
| 144 |
+
--prune 1e-6 --max-nodes 10000 --output "$ROOT/h1b-exact.jsonl"
|
| 145 |
+
```
|
| 146 |
+
|
| 147 |
+
The checker uses depth-first search with a branch-safe, depth-bounded KV cache
|
| 148 |
+
stack. Siblings get independent cache copies, including when an HF cache mutates
|
| 149 |
+
in place. `--full-prefix` selects an independent reference path; branches deeper
|
| 150 |
+
than `--cache-depth` also fall back to full-prefix forwards. It is not yet batched
|
| 151 |
+
across frontier nodes; real-model cache compatibility and B200 throughput/memory
|
| 152 |
+
remain unmeasured. The finite search preserves duplicates and records an upper bound on
|
| 153 |
+
all pruned/unexpanded probability; only zero omitted mass is called exact.
|
| 154 |
+
The conservative gate uses mean *upper-bound* log-probability error < 0.1 mean
|
| 155 |
+
absolute path/trunk delta in every populated bucket, for both models. Empty
|
| 156 |
+
buckets are `insufficient`, not PASS; a zero gap cannot pass a strict relative
|
| 157 |
+
threshold. Both gate policy and raw counts must be inspected before training.
|
| 158 |
+
Gate status is currently **NOT MEASURED**. Depth-2 extension is conditional on a
|
| 159 |
+
failed measured gate and is not implemented speculatively.
|
| 160 |
+
|
| 161 |
+
T10 has a separate diagnostic launcher so the standard 100-update presets remain
|
| 162 |
+
unchanged. Preview first; `--execute` actually consumes the allocated company GPU:
|
| 163 |
+
|
| 164 |
+
```bash
|
| 165 |
+
python3 -m tools.h1b.canary --algorithm h1b-c --runtime "$RUNTIME" \
|
| 166 |
+
--run-id canary-h1b-c-s42 --root "$ROOT"
|
| 167 |
+
# Repeat for each of the five presets, adding --execute only after allocation.
|
| 168 |
+
```
|
| 169 |
+
|
| 170 |
+
Gradient replay uses a fixed 4,096-response archive; it never creates that archive.
|
| 171 |
+
Start with 128 responses (two groups of 64) to measure memory/time and sketch error:
|
| 172 |
+
|
| 173 |
+
```bash
|
| 174 |
+
bash "$RUNTIME/python-b200.sh" -m tools.h1b.grad_replay \
|
| 175 |
+
--student "$S" --teacher "$T" --archive "$REPLAY_ARCHIVE" \
|
| 176 |
+
--limit 128 --output "$ROOT/h1b-replay-probe"
|
| 177 |
+
bash "$RUNTIME/python-b200.sh" -m tools.h1b.validate_sketch \
|
| 178 |
+
--replay "$ROOT/h1b-replay-probe" --output "$ROOT/h1b-sketch-gate.json"
|
| 179 |
+
# After sizing, use --limit 4096, --buckets 0,3 and a fresh output directory.
|
| 180 |
+
bash "$RUNTIME/python-b200.sh" -m tools.h1b.fit_cb \
|
| 181 |
+
--replay "$REPLAY_DIR" --bootstrap 200 --output "$ROOT/h1b-curves.json"
|
| 182 |
+
```
|
| 183 |
+
|
| 184 |
+
Replay reuses the algorithm's targets/gradients, with leave-group-out bucket
|
| 185 |
+
means. Seven full FP32 buffers are allocated by default (five terms plus two
|
| 186 |
+
disjoint strict-position controls), more than the plan's five-buffer estimate.
|
| 187 |
+
`--buckets 0,3` adds eight buffers and may exceed B200 memory; `--buffer-device cpu`
|
| 188 |
+
is an explicit alternative requiring host RAM and transfers. Exact vectors for
|
| 189 |
+
the first two groups can be very large on disk (`--exact-groups 0` disables them,
|
| 190 |
+
but then the real T11 gate is not measured). Raw gradients only are implemented;
|
| 191 |
+
optional Adam-preconditioned replay and optimizer-state exports are not enabled.
|
| 192 |
+
The qualified architecture contract requires zero dropout; checkpointed replay
|
| 193 |
+
does not accumulate the training logger/baseline.
|
| 194 |
+
|
| 195 |
+
Analysis reports c(B), strict-position control cosine, split-half agreement,
|
| 196 |
+
finite-pool SNR, norm ratios, group noise scale, fitted c_infinity/B_noise and
|
| 197 |
+
group-bootstrap intervals. Token-weighted pooling retains common batch
|
| 198 |
+
normalization across differently sized responses. At B=64, two 32-response halves
|
| 199 |
+
cannot be recovered from 64-response group sketches: split-half is null, not a
|
| 200 |
+
fabricated measurement. At the entire finite pool SNR can be undefined; it is
|
| 201 |
+
reported null. Boundary fits are marked not identified. This is not a causal
|
| 202 |
+
bias-versus-variance verdict or an automatic decision-table result.
|
| 203 |
+
|
| 204 |
+
## Validation and milestone status
|
| 205 |
+
|
| 206 |
+
Run CPU tests only in an existing environment:
|
| 207 |
+
|
| 208 |
+
```bash
|
| 209 |
+
TORCH_COMPILE_DISABLE=1 OMP_NUM_THREADS=1 python -m unittest discover -s tests -v
|
| 210 |
+
```
|
| 211 |
+
|
| 212 |
+
See `H1B_VALIDATION.json` for the executed checks. Fixtures verify algorithmic
|
| 213 |
+
identities; they do not replace T1/T3/T9 real-tokenizer/500-response gates, T10
|
| 214 |
+
B200 five-preset canaries, or T11 on full model gradient groups.
|
| 215 |
+
|
| 216 |
+
M1/M2/M4/M5 **code** is implemented with the limitations above. No real trunk
|
| 217 |
+
table, M3 gate decision, B200 qualification, gradient curves, trained checkpoint,
|
| 218 |
+
new model/data transfer, or M6/M7 scientific result is claimed. No new GPU job was
|
| 219 |
+
launched for this implementation. Do not update checkouts used by active jobs;
|
| 220 |
+
use fresh source/run IDs and preserve the existing Strict/SimCT/MP runs.
|
docs/H1B_SOURCE.json
ADDED
|
@@ -0,0 +1,121 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"input_plans": [
|
| 3 |
+
{
|
| 4 |
+
"name": "Implementation Plan",
|
| 5 |
+
"sha256": "d99aad05f79545931f051601414eaa8b736a871a226cb898f8d4ef1f3ee24d32"
|
| 6 |
+
},
|
| 7 |
+
{
|
| 8 |
+
"name": "Experiment Design",
|
| 9 |
+
"sha256": "b13831f4182dd87fa286851b7617144ec3edce030ed148428676cf00699cba58"
|
| 10 |
+
}
|
| 11 |
+
],
|
| 12 |
+
"files": [
|
| 13 |
+
{
|
| 14 |
+
"path": "kdflow/h1b/__init__.py",
|
| 15 |
+
"sha256": "bb8fa0f3ccd318368d986568b5423b522b51100210b877b3844835b191f6279a"
|
| 16 |
+
},
|
| 17 |
+
{
|
| 18 |
+
"path": "kdflow/h1b/baseline.py",
|
| 19 |
+
"sha256": "a1b46847ae84906bdad4068a9a6bae3fca47b07748f72e20e0ac6a699d7c393d"
|
| 20 |
+
},
|
| 21 |
+
{
|
| 22 |
+
"path": "kdflow/h1b/buckets.py",
|
| 23 |
+
"sha256": "964745a54ad61fe3d7148efdabbd8af8ea6c193885db5fe5a8e72330d55a5c2f"
|
| 24 |
+
},
|
| 25 |
+
{
|
| 26 |
+
"path": "kdflow/h1b/chunklog.py",
|
| 27 |
+
"sha256": "37e53384993b3d19fe533e58f873953a0c04cc8c730a6ee5ed1ef07b954c778b"
|
| 28 |
+
},
|
| 29 |
+
{
|
| 30 |
+
"path": "kdflow/h1b/exact.py",
|
| 31 |
+
"sha256": "b97eabf9a5b6f6435311c6d3b0321bfe63723b2fd23299684078f1d08dec0f4b"
|
| 32 |
+
},
|
| 33 |
+
{
|
| 34 |
+
"path": "kdflow/h1b/groups.py",
|
| 35 |
+
"sha256": "129788526be7c11d6a4493b25ec6ab5843dc090981a5342c482ff68b7c16f20a"
|
| 36 |
+
},
|
| 37 |
+
{
|
| 38 |
+
"path": "kdflow/h1b/strprob.py",
|
| 39 |
+
"sha256": "ea7a893c15fcaa96046293af1b37a50fa663b47b13e1bd20a0fbb9bbf49ce736"
|
| 40 |
+
},
|
| 41 |
+
{
|
| 42 |
+
"path": "kdflow/h1b/vocab_index.py",
|
| 43 |
+
"sha256": "55ac8c76ced13aa98ead01375de41906be9764276569af25908683d6a5a3abf3"
|
| 44 |
+
},
|
| 45 |
+
{
|
| 46 |
+
"path": "tools/h1b/__init__.py",
|
| 47 |
+
"sha256": "bbc4b569caaf873fc370d1dbe601de3b41ea107725c328997cb6534c0cc93f4a"
|
| 48 |
+
},
|
| 49 |
+
{
|
| 50 |
+
"path": "tools/h1b/canary.py",
|
| 51 |
+
"sha256": "f106d45d41024386df56e8ccc8da74c56247edfeb9a2c35879f9fe8712512e93"
|
| 52 |
+
},
|
| 53 |
+
{
|
| 54 |
+
"path": "tools/h1b/common.py",
|
| 55 |
+
"sha256": "8adf680d9e747e40f2781613a2b53b77f44b58bfe58ac0606c08432de4e05d06"
|
| 56 |
+
},
|
| 57 |
+
{
|
| 58 |
+
"path": "tools/h1b/enrich_archive.py",
|
| 59 |
+
"sha256": "db21f7bdd5d038b5cb8fc9888bd70a7e13265107edd8bf34fe32f0d6475eb770"
|
| 60 |
+
},
|
| 61 |
+
{
|
| 62 |
+
"path": "tools/h1b/exact_strprob.py",
|
| 63 |
+
"sha256": "99b9630d4c6b7cc6d5685cd47014a5dfcb80fce0ae6f133a529a9799ce25be35"
|
| 64 |
+
},
|
| 65 |
+
{
|
| 66 |
+
"path": "tools/h1b/fit_cb.py",
|
| 67 |
+
"sha256": "077f8e0c832110052711cc81e71f3b71cc74c606197f0bbc9049ea021a21247b"
|
| 68 |
+
},
|
| 69 |
+
{
|
| 70 |
+
"path": "tools/h1b/grad_replay.py",
|
| 71 |
+
"sha256": "a1af8a33d105c448ee5a79e69ad7bf9b5b077757578dc28f61b4fc3e5c6e33fb"
|
| 72 |
+
},
|
| 73 |
+
{
|
| 74 |
+
"path": "tools/h1b/sketch.py",
|
| 75 |
+
"sha256": "69b6093ffd3a096686aaa47936bb0b8785426ca8ef88ccc58e0daf77aa214186"
|
| 76 |
+
},
|
| 77 |
+
{
|
| 78 |
+
"path": "tools/h1b/validate_alignment.py",
|
| 79 |
+
"sha256": "39bcfd71736f5f1ddf79bb4eee9abf85152d77e4b158d84cdc452fcc7217e3e5"
|
| 80 |
+
},
|
| 81 |
+
{
|
| 82 |
+
"path": "tools/h1b/validate_sketch.py",
|
| 83 |
+
"sha256": "84e47110eda3f7b1599fdeef4f1be0cd4bf4eb2bd5e78ac974f619f000c01de1"
|
| 84 |
+
},
|
| 85 |
+
{
|
| 86 |
+
"path": "kdflow/algorithms/h1b_span.py",
|
| 87 |
+
"sha256": "8c332d91549ea7e549b2f771d16b7b31dcc263cf3d77000bfe4d5eb941489acb"
|
| 88 |
+
},
|
| 89 |
+
{
|
| 90 |
+
"path": "tests/test_h1b.py",
|
| 91 |
+
"sha256": "61e61fff4bdf4a8b1dea1c6e9fd97974a34d75b06a1516ce562dfe971b9a2c40"
|
| 92 |
+
}
|
| 93 |
+
],
|
| 94 |
+
"frozen_strict": [
|
| 95 |
+
{
|
| 96 |
+
"sha256": "fdbd99c55aeac4d061527158bdfc82a93333a42bf347eceaa134d34f5ef05952",
|
| 97 |
+
"path": "kdflow/algorithms/simple_ctkd.py"
|
| 98 |
+
},
|
| 99 |
+
{
|
| 100 |
+
"sha256": "5c6647c6fb8064ffd5f7b1ed971168cd1f3392caee62e2d75916d41931828e18",
|
| 101 |
+
"path": "kdflow/algorithms/byte_align_ctkd.py"
|
| 102 |
+
},
|
| 103 |
+
{
|
| 104 |
+
"sha256": "298925da4ce973b0f2917e5ba5ad2adbc4862d7c1a73515c5c86514ac0d22ddb",
|
| 105 |
+
"path": "kdflow/algorithms/topk_byte_align_ctkd.py"
|
| 106 |
+
},
|
| 107 |
+
{
|
| 108 |
+
"sha256": "c77cd962cb0430cd97ace5312077e847398a03da3ce1473062fd2c5c9808498a",
|
| 109 |
+
"path": "kdflow/token_alignment.py"
|
| 110 |
+
},
|
| 111 |
+
{
|
| 112 |
+
"sha256": "abca69fdbe4f7d90a7ca990300189147230cce8f20d589c934d88f0b63c4edf6",
|
| 113 |
+
"path": "kdflow/loss/reverse_kl_div.py"
|
| 114 |
+
},
|
| 115 |
+
{
|
| 116 |
+
"sha256": "151c81614f97b908465a952f595abe1f4dc5c0d17cea86431a7068c381125cc5",
|
| 117 |
+
"path": "kdflow/loss/chunked_loss.py"
|
| 118 |
+
}
|
| 119 |
+
],
|
| 120 |
+
"baseline_commit": "0eeed4ca2b0d7b07dd50076e35d81f030a3b9f2d"
|
| 121 |
+
}
|
docs/H1B_VALIDATION.json
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"date": "2026-10-19",
|
| 3 |
+
"baseline_commit": "0eeed4ca2b0d7b07dd50076e35d81f030a3b9f2d",
|
| 4 |
+
"platform": "WSL Linux, existing CPU environment; no package installation",
|
| 5 |
+
"python": "3.12.3",
|
| 6 |
+
"torch": "2.14.0+cpu",
|
| 7 |
+
"full_suite": {
|
| 8 |
+
"command": "TORCH_COMPILE_DISABLE=1 OMP_NUM_THREADS=1 python -m unittest discover -s tests -v",
|
| 9 |
+
"passed": 104,
|
| 10 |
+
"skipped": 0,
|
| 11 |
+
"elapsed_seconds": 72.46
|
| 12 |
+
},
|
| 13 |
+
"verified": [
|
| 14 |
+
"2000 cover-prefix/brute-force queries, duplicate byte IDs and 0xff bounds",
|
| 15 |
+
"Strict-group fixture parity including EOS and flattened response offsets",
|
| 16 |
+
"30-token toy LM on 50 strings: one-exit enumeration equals trunk; exhaustive probability >= trunk >= canonical path",
|
| 17 |
+
"Exact search duplicate branches, pruning/node-budget omitted-mass bounds",
|
| 18 |
+
"Same-tokenizer fixture: all five arms have identical loss and zero span term",
|
| 19 |
+
"Previous-window baseline, state restore and no evaluation/no-grad mutation",
|
| 20 |
+
"Arm b student gradients equal autograd span MSE; arm a loss/all student gradients equal Strict on accepted fixtures",
|
| 21 |
+
"Tile/checkpoint gradient invariance; whole-response mismatch exclusion and connected zero backward",
|
| 22 |
+
"Actual actor fit/evaluate bodies with CPU SGD: window hook after one optimizer update and no hook during evaluation",
|
| 23 |
+
"Actual rollout tokenizer method: EOS policy and raw re-encoding match field",
|
| 24 |
+
"Actual math evaluation sampling expression: independent eval top-p and unchanged rollout kwargs",
|
| 25 |
+
"Window moments weight chunks directly rather than microbatch means",
|
| 26 |
+
"CountSketch chunk invariance, fixture norm error <1 percent and cosine error <0.01",
|
| 27 |
+
"End-to-end saved sketch analysis CLI, known noise curve and undefined zero-vector handling",
|
| 28 |
+
"KV oracle versus full-prefix fixture with deliberately mutating cache and sibling branches",
|
| 29 |
+
"Five full-workload launcher dry-run plans; separate five-step diagnostic preview",
|
| 30 |
+
"New source hashes verified; six frozen Strict sources match baseline Git blobs",
|
| 31 |
+
"All existing Strict/SimCT/MP/ALIGN/TRUST regression tests",
|
| 32 |
+
"git diff --check",
|
| 33 |
+
"Old archive teacher-prompt recovery: exact unambiguous dataset/template match, no sampled-ID fabrication"
|
| 34 |
+
],
|
| 35 |
+
"test_isolation": "CPU suite bypasses utility package tokenizer-factory initialization using a test-only namespace; real algorithm/loss/logging numerical source is executed",
|
| 36 |
+
"not_measured": [
|
| 37 |
+
"T1 on 10000 real texts with both real tokenizers",
|
| 38 |
+
"T3/T9 on 500 real archived responses",
|
| 39 |
+
"M2 real model trunk table and M3 exact/trunk gate at checkpoints 0 and 60",
|
| 40 |
+
"T10 five-preset B200/FSDP canaries and parquet validation",
|
| 41 |
+
"Real HF KV cache compatibility, head-tile throughput and memory",
|
| 42 |
+
"T11 on two full-model gradient groups; real replay curves",
|
| 43 |
+
"Full training, task accuracy, bias-versus-variance decision or paper reproduction"
|
| 44 |
+
],
|
| 45 |
+
"deliberately_not_implemented": [
|
| 46 |
+
"Batched KV frontier expansion; current checker uses depth-bounded DFS cache with full-prefix fallback",
|
| 47 |
+
"Conditional depth-2 training estimator before a failed measured trunk gate",
|
| 48 |
+
"Optional Adam-preconditioned replay and optimizer-state exports",
|
| 49 |
+
"Fresh 4096-response generation; replay requires a supplied fixed archive",
|
| 50 |
+
"Automatic causal decision-table classification or optional c-plus experiments"
|
| 51 |
+
],
|
| 52 |
+
"compute": "No new GPU job, model/dataset download, remote build or training launched",
|
| 53 |
+
"h1b_tests_in_full_suite": 21
|
| 54 |
+
}
|
kdflow/algorithms/__init__.py
CHANGED
|
@@ -12,6 +12,7 @@ def register_algorithm(name):
|
|
| 12 |
|
| 13 |
from kdflow.algorithms.simple_ctkd import SimpleCrossTokenizerKD
|
| 14 |
from kdflow.algorithms.byte_align_ctkd import ByteAlignCrossTokenizerKD
|
|
|
|
| 15 |
from kdflow.algorithms.topk_byte_align_ctkd import TopKByteAlignCrossTokenizerKD
|
| 16 |
from kdflow.algorithms.simct import SimCT
|
| 17 |
from kdflow.algorithms.simct_paper import SimCTPaper
|
|
|
|
| 12 |
|
| 13 |
from kdflow.algorithms.simple_ctkd import SimpleCrossTokenizerKD
|
| 14 |
from kdflow.algorithms.byte_align_ctkd import ByteAlignCrossTokenizerKD
|
| 15 |
+
from kdflow.algorithms.h1b_span import H1bSpanCTKD
|
| 16 |
from kdflow.algorithms.topk_byte_align_ctkd import TopKByteAlignCrossTokenizerKD
|
| 17 |
from kdflow.algorithms.simct import SimCT
|
| 18 |
from kdflow.algorithms.simct_paper import SimCTPaper
|
kdflow/algorithms/h1b_span.py
ADDED
|
@@ -0,0 +1,165 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""H1b Strict + detached path/string span coefficients, opt-in only."""
|
| 2 |
+
import hashlib
|
| 3 |
+
from dataclasses import replace
|
| 4 |
+
import math
|
| 5 |
+
import torch
|
| 6 |
+
from kdflow.algorithms import register_algorithm
|
| 7 |
+
from kdflow.algorithms.byte_align_ctkd import ByteAlignCrossTokenizerKD
|
| 8 |
+
from kdflow.loss.chunked_loss import chunked_loss
|
| 9 |
+
from kdflow.h1b.groups import align_byte_span_groups_batch
|
| 10 |
+
from kdflow.h1b.vocab_index import VocabIndex
|
| 11 |
+
from kdflow.h1b.strprob import score_hidden_chunks
|
| 12 |
+
from kdflow.h1b.baseline import BucketBaseline
|
| 13 |
+
from kdflow.h1b.buckets import bucket_of
|
| 14 |
+
from kdflow.h1b.chunklog import ChunkLogger
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
@register_algorithm('h1b_span')
|
| 18 |
+
class H1bSpanCTKD(ByteAlignCrossTokenizerKD):
|
| 19 |
+
def __init__(self,*args,**kwargs):
|
| 20 |
+
super().__init__(*args,**kwargs)
|
| 21 |
+
kd=self.args.kd
|
| 22 |
+
self.arm=kd.h1b_arm; self.lam=kd.h1b_span_lambda
|
| 23 |
+
if self.arm not in {'a','b','b_bl','c_str','c'} or not math.isfinite(self.lam) or self.lam<0:
|
| 24 |
+
raise ValueError('Invalid H1b arm or span lambda')
|
| 25 |
+
if kd.kd_ratio!=1 or kd.kd_temperature!=1 or kd.kd_loss_fn!='rkl':
|
| 26 |
+
raise ValueError('H1b requires kd_ratio=1, temperature=1 and rkl')
|
| 27 |
+
if (self.args.train.num_nodes*self.args.train.num_gpus_per_node!=1
|
| 28 |
+
or self.args.model.ring_attn_size!=1):
|
| 29 |
+
raise ValueError('H1b supports one student rank and ring size 1')
|
| 30 |
+
config=getattr(self.student.model,'config',None)
|
| 31 |
+
config=getattr(config,'text_config',None) or config
|
| 32 |
+
if getattr(config,'final_logit_softcapping',None) or getattr(self.teacher_lm_head,'_mp_opd_softcap',None):
|
| 33 |
+
raise ValueError('H1b softcapped heads are not qualified')
|
| 34 |
+
self.svi=VocabIndex(self.student_tokenizer,self.student_byte_codec)
|
| 35 |
+
self.tvi=VocabIndex(self.teacher_tokenizer,self.teacher_byte_codec)
|
| 36 |
+
self.baseline=BucketBaseline()
|
| 37 |
+
self.window=0
|
| 38 |
+
self.chunk_logger=ChunkLogger(getattr(self.args.train,'save_path',None),
|
| 39 |
+
kd.h1b_log_sample_rate,tuple(int(s) for s in kd.h1b_dump_every.split(',') if s.strip()))
|
| 40 |
+
|
| 41 |
+
def validate_optimizer_window(self,batches):
|
| 42 |
+
if self.strategy.step!=0 or len(batches)!=self.strategy.accumulated_gradient:
|
| 43 |
+
raise ValueError('H1b requires one complete optimizer window per fit')
|
| 44 |
+
|
| 45 |
+
def state_dict(self):
|
| 46 |
+
return dict(version=1,arm=self.arm,span_lambda=self.lam,window=self.window,baseline=self.baseline.state_dict())
|
| 47 |
+
|
| 48 |
+
def load_state_dict(self,state):
|
| 49 |
+
if state['version']!=1 or state['arm']!=self.arm or state['span_lambda']!=self.lam:
|
| 50 |
+
raise ValueError('Changed H1b state contract')
|
| 51 |
+
self.baseline.load_state_dict(state['baseline']); self.window=int(state['window'])
|
| 52 |
+
self.chunk_logger.reset(); self.chunk_logger.step=self.window
|
| 53 |
+
|
| 54 |
+
def on_optimizer_window_end(self):
|
| 55 |
+
if self.strategy.step!=0: raise ValueError('Cannot close partial H1b optimizer window')
|
| 56 |
+
metrics=self.chunk_logger.end_window(self.window)
|
| 57 |
+
self.baseline.end_window(); self.window+=1
|
| 58 |
+
return metrics
|
| 59 |
+
|
| 60 |
+
def training_step(self,batch):
|
| 61 |
+
replay=getattr(self,'replay_enabled',False)
|
| 62 |
+
# Offline replay intentionally retains graphs; normal training must not
|
| 63 |
+
# keep the previous microbatch graph alive during the next forward.
|
| 64 |
+
self.replay_terms={}; self.replay_records=[]
|
| 65 |
+
sm,tm=batch['stu_loss_mask'].bool(),batch['tea_loss_mask'].bool()
|
| 66 |
+
sy,ty=batch['stu_input_ids'].roll(-1,1),batch['tea_input_ids'].roll(-1,1)
|
| 67 |
+
st=[self.student_tokenizer.convert_ids_to_tokens(sy[i][sm[i]].cpu().tolist()) for i in range(len(sm))]
|
| 68 |
+
tt=[self.teacher_tokenizer.convert_ids_to_tokens(ty[i][tm[i]].cpu().tolist()) for i in range(len(tm))]
|
| 69 |
+
rows=align_byte_span_groups_batch(tt,st,self.teacher_tokenizer.eos_token,self.student_tokenizer.eos_token,
|
| 70 |
+
self.teacher_byte_codec.piece_to_bytes,self.student_byte_codec.piece_to_bytes)
|
| 71 |
+
for i,row in enumerate(rows):
|
| 72 |
+
sids=sy[i][sm[i]].cpu().tolist(); tids=ty[i][tm[i]].cpu().tolist()
|
| 73 |
+
if (any(t in self.svi.special and t!=self.student_tokenizer.eos_token_id for t in sids)
|
| 74 |
+
or any(t in self.tvi.special and t!=self.teacher_tokenizer.eos_token_id for t in tids)):
|
| 75 |
+
rows[i]=replace(row,groups=(),valid=False,reason='unsupported_special')
|
| 76 |
+
sh=self.student(batch['stu_input_ids'],attention_mask=batch['stu_attn_mask'],allgather_logits=True,
|
| 77 |
+
ring_attn_group=self.strategy.ring_attn_group,**(batch.get('stu_multi_modal_inputs') or {}))['hidden_states'][-1][sm]
|
| 78 |
+
th=batch['teacher_hiddens']
|
| 79 |
+
if isinstance(th,list): th=torch.cat(th,0)
|
| 80 |
+
if len(th)!=int(tm.sum()): raise ValueError('H1b teacher hidden/mask mismatch')
|
| 81 |
+
th=th.detach().to(self.teacher_lm_head.weight)
|
| 82 |
+
sl,tl=sy[sm],ty[tm]
|
| 83 |
+
groups=[g for row in rows for g in row.groups]
|
| 84 |
+
strict=[g for g in groups if g.kind=='strict']
|
| 85 |
+
mis=[g for g in groups if g.kind=='mis']
|
| 86 |
+
si=[g.s_idx[0] for g in strict]; ti=[g.t_idx[0] for g in strict]
|
| 87 |
+
size=self.args.train.chunked_loss_size or max(len(sh),1)
|
| 88 |
+
denom=float(batch['avg_micro_batch_token_num'])
|
| 89 |
+
if not math.isfinite(denom) or denom<=0: raise ValueError('Positive finite common normalizer required')
|
| 90 |
+
zero=sh.sum()*0.
|
| 91 |
+
strict_sum=zero; metric_sums={}
|
| 92 |
+
if si:
|
| 93 |
+
ash,ath=sh[si],th[ti]
|
| 94 |
+
def slog(h,skip=False): return self.student.model.lm_head(h,skip=skip)[:,self.student_overlap_token_ids]
|
| 95 |
+
def tlog(a,b):
|
| 96 |
+
with torch.no_grad():
|
| 97 |
+
return self.teacher_lm_head(ath[a:b])[:,self.teacher_overlap_token_ids]
|
| 98 |
+
strict_sum,metric_sums=chunked_loss(ash,self.student.model.lm_head,self.loss_fn,
|
| 99 |
+
student_logits_fn=slog,teacher_logits_fn=tlog,chunk_size=size,reduction='sum',
|
| 100 |
+
metric_fns=self.metric_fns,return_metrics=True)
|
| 101 |
+
sprob=score_hidden_chunks(self.student.model.lm_head,sh,sl,[(g.s_idx,g.content) for g in mis],self.svi,size,student=True)
|
| 102 |
+
with torch.no_grad():
|
| 103 |
+
tprob=score_hidden_chunks(self.teacher_lm_head,th,tl,[(g.t_idx,g.content) for g in mis],self.tvi,size,student=False)
|
| 104 |
+
terms={k:zero for k in ('path','str','base_path','base_str')}
|
| 105 |
+
training=self.student.training and torch.is_grad_enabled()
|
| 106 |
+
log=self.chunk_logger
|
| 107 |
+
if training:
|
| 108 |
+
log.rows+=len(rows); log.dropped+=sum(not r.valid for r in rows)
|
| 109 |
+
log.byte_dropped+=sum(r.reason=='byte_mismatch' for r in rows)
|
| 110 |
+
log.stu+=sum(r.student_tokens for r in rows); log.tea+=sum(r.teacher_tokens for r in rows)
|
| 111 |
+
log.strict_stu+=sum(len(g.s_idx) for g in strict if not g.eos)
|
| 112 |
+
log.strict_tea+=sum(len(g.t_idx) for g in strict if not g.eos)
|
| 113 |
+
log.mis_stu+=sum(len(g.s_idx) for g in mis)
|
| 114 |
+
if 'stu_reencode_match' in batch:
|
| 115 |
+
values=torch.as_tensor(batch['stu_reencode_match']).detach()
|
| 116 |
+
log.reencode_sum+=float(values.sum()); log.reencode_n+=values.numel()
|
| 117 |
+
# Response identity excludes prompt text and does not retain raw content.
|
| 118 |
+
response_hashes={i:hashlib.sha256(bytes(str((st[i],tt[i])),'utf8')).hexdigest() for i in range(len(rows))}
|
| 119 |
+
owner={id(g):i for i,r in enumerate(rows) for g in r.groups}
|
| 120 |
+
for g,s,t in zip(mis,sprob,tprob):
|
| 121 |
+
dp=(s.log_q_path-t.log_q_path).detach(); ds=(s.log_p_trunk-t.log_p_trunk).detach()
|
| 122 |
+
bucket=bucket_of(g.content)
|
| 123 |
+
bp=self.baseline.lookup(bucket,'path'); bs=self.baseline.lookup(bucket,'str')
|
| 124 |
+
if replay:
|
| 125 |
+
self.replay_records.append(dict(bucket=bucket,path=dp,str=ds,log_q=s.log_q_path))
|
| 126 |
+
for k,v in (('path',dp),('str',ds),('base_path',bp),('base_str',bs)):
|
| 127 |
+
terms[k]=terms[k]+2*self.lam*v*s.log_q_path
|
| 128 |
+
if training:
|
| 129 |
+
p,v=float(dp),float(ds)
|
| 130 |
+
coefficient=0. if self.arm=='a' else 2*self.lam*(p-bp if self.arm=='b_bl' else v-bs if self.arm=='c' else p if self.arm=='b' else v)
|
| 131 |
+
log.add(dict(response_hash=response_hashes[owner[id(g)]],step=self.window,bucket=bucket,
|
| 132 |
+
b0=g.b0,b1=g.b1,chunk_hex=g.content.hex(),s_tokens=len(g.s_idx),t_tokens=len(g.t_idx),
|
| 133 |
+
path=p,str=v,path_bl=p-bp,str_bl=v-bs,gap=p-v,baseline_path=bp,baseline_str=bs,coefficient=coefficient,
|
| 134 |
+
stu_log_q_path=float(s.log_q_path.detach()),tea_log_q_path=float(t.log_q_path),
|
| 135 |
+
stu_log_p_trunk=float(s.log_p_trunk),tea_log_p_trunk=float(t.log_p_trunk),
|
| 136 |
+
stu_log_overhang=float(s.log_overhang),tea_log_overhang=float(t.log_overhang),
|
| 137 |
+
slack_stu=float(s.log_p_trunk-s.log_q_path.detach()),slack_tea=float(t.log_p_trunk-t.log_q_path),
|
| 138 |
+
overhang_stu=float((s.log_overhang-s.log_p_trunk).exp()),overhang_tea=float((t.log_overhang-t.log_p_trunk).exp())))
|
| 139 |
+
self.baseline.accumulate(bucket,p,'path'); self.baseline.accumulate(bucket,v,'str')
|
| 140 |
+
span=zero if self.arm=='a' else (terms['path'] if self.arm=='b' else terms['path']-terms['base_path'] if self.arm=='b_bl'
|
| 141 |
+
else terms['str'] if self.arm=='c_str' else terms['str']-terms['base_str'])
|
| 142 |
+
loss=(strict_sum+span)/denom
|
| 143 |
+
if not bool(torch.isfinite(loss)): raise FloatingPointError('Nonfinite H1b loss')
|
| 144 |
+
if replay:
|
| 145 |
+
self.replay_terms={k:v/denom for k,v in terms.items()}
|
| 146 |
+
self.replay_terms['strict']=strict_sum/denom
|
| 147 |
+
if getattr(self,'replay_control',False):
|
| 148 |
+
for parity in (0,1):
|
| 149 |
+
x,y=sh[si[parity::2]],th[ti[parity::2]]
|
| 150 |
+
if len(x):
|
| 151 |
+
def replay_teacher(a,b,y=y):
|
| 152 |
+
with torch.no_grad():
|
| 153 |
+
return self.teacher_lm_head(y[a:b])[:,self.teacher_overlap_token_ids]
|
| 154 |
+
val=chunked_loss(x,self.student.model.lm_head,self.loss_fn,
|
| 155 |
+
student_logits_fn=slog,
|
| 156 |
+
teacher_logits_fn=replay_teacher,
|
| 157 |
+
chunk_size=size,reduction='sum')
|
| 158 |
+
else: val=zero
|
| 159 |
+
self.replay_terms['strict_'+str(parity)]=val/denom
|
| 160 |
+
info={'train/loss':loss,'train/kd_loss':loss,'train/h1b_span_loss':span/denom,
|
| 161 |
+
'distill/alignment_ratio':torch.tensor(len(si)/max(len(sl),1)),
|
| 162 |
+
'distill/h1b_mis_groups':loss.detach().new_tensor(float(len(mis))),
|
| 163 |
+
'distill/h1b_dropped_responses':loss.detach().new_tensor(float(sum(not r.valid for r in rows)))}
|
| 164 |
+
info.update({k:v/denom for k,v in metric_sums.items()})
|
| 165 |
+
return info
|
kdflow/arguments/distillation_args.py
CHANGED
|
@@ -23,7 +23,7 @@ class DistillationArguments:
|
|
| 23 |
default="byte_align_ctkd",
|
| 24 |
metadata={
|
| 25 |
"help": "KD algorithm for each training step.",
|
| 26 |
-
"choices": ["simple_ctkd", "byte_align_ctkd", "topk_byte_align_ctkd", "simct", "simct_paper", *MP_OPD_ALGORITHMS],
|
| 27 |
}
|
| 28 |
)
|
| 29 |
kd_loss_fn: str = field(
|
|
@@ -132,18 +132,37 @@ class DistillationArguments:
|
|
| 132 |
mp_opd_trust_eps_g: float = field(default=1e-12)
|
| 133 |
mp_opd_trust_vocab_chunk: int = field(default=4096)
|
| 134 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 135 |
def __post_init__(self):
|
| 136 |
if self.multi_teacher_config is not None:
|
| 137 |
raise ValueError(
|
| 138 |
"This release supports one teacher model per run; "
|
| 139 |
"use --teacher_name_or_path instead of --multi_teacher_config."
|
| 140 |
)
|
| 141 |
-
supported_algorithms = {"simple_ctkd", "byte_align_ctkd", "topk_byte_align_ctkd", "simct", "simct_paper", *MP_OPD_ALGORITHMS}
|
| 142 |
if self.kd_algorithm not in supported_algorithms:
|
| 143 |
raise ValueError(
|
| 144 |
f"kd_algorithm must be one of {sorted(supported_algorithms)}, "
|
| 145 |
f"got {self.kd_algorithm!r}."
|
| 146 |
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 147 |
if self.kd_algorithm in {'simct','simct_paper'}:
|
| 148 |
if self.kd_ratio != 1 or self.kd_temperature != 1 or self.kd_loss_fn != 'rkl':
|
| 149 |
raise ValueError('SimCT baseline requires kd_ratio=1, kd_temperature=1, kd_loss_fn=rkl')
|
|
|
|
| 23 |
default="byte_align_ctkd",
|
| 24 |
metadata={
|
| 25 |
"help": "KD algorithm for each training step.",
|
| 26 |
+
"choices": ["simple_ctkd", "byte_align_ctkd", "topk_byte_align_ctkd", "simct", "simct_paper", "h1b_span", *MP_OPD_ALGORITHMS],
|
| 27 |
}
|
| 28 |
)
|
| 29 |
kd_loss_fn: str = field(
|
|
|
|
| 132 |
mp_opd_trust_eps_g: float = field(default=1e-12)
|
| 133 |
mp_opd_trust_vocab_chunk: int = field(default=4096)
|
| 134 |
|
| 135 |
+
h1b_arm: str = field(default='a', metadata={'choices':['a','b','b_bl','c_str','c']})
|
| 136 |
+
h1b_span_lambda: float = field(default=0.5)
|
| 137 |
+
h1b_log_sample_rate: float = field(default=0.05)
|
| 138 |
+
h1b_dump_every: str = field(default='0,20,60,100')
|
| 139 |
+
|
| 140 |
def __post_init__(self):
|
| 141 |
if self.multi_teacher_config is not None:
|
| 142 |
raise ValueError(
|
| 143 |
"This release supports one teacher model per run; "
|
| 144 |
"use --teacher_name_or_path instead of --multi_teacher_config."
|
| 145 |
)
|
| 146 |
+
supported_algorithms = {"simple_ctkd", "byte_align_ctkd", "topk_byte_align_ctkd", "simct", "simct_paper", "h1b_span", *MP_OPD_ALGORITHMS}
|
| 147 |
if self.kd_algorithm not in supported_algorithms:
|
| 148 |
raise ValueError(
|
| 149 |
f"kd_algorithm must be one of {sorted(supported_algorithms)}, "
|
| 150 |
f"got {self.kd_algorithm!r}."
|
| 151 |
)
|
| 152 |
+
if self.kd_algorithm == 'h1b_span':
|
| 153 |
+
if self.kd_ratio!=1 or self.kd_temperature!=1 or self.kd_loss_fn!='rkl':
|
| 154 |
+
raise ValueError('H1b requires kd_ratio=1, kd_temperature=1, kd_loss_fn=rkl')
|
| 155 |
+
if self.h1b_arm not in {'a','b','b_bl','c_str','c'}:
|
| 156 |
+
raise ValueError('Invalid H1b arm')
|
| 157 |
+
if not math.isfinite(self.h1b_span_lambda) or self.h1b_span_lambda<0:
|
| 158 |
+
raise ValueError('Invalid H1b span lambda')
|
| 159 |
+
if not math.isfinite(self.h1b_log_sample_rate) or not 0<=self.h1b_log_sample_rate<=1:
|
| 160 |
+
raise ValueError('Invalid H1b sample rate')
|
| 161 |
+
try:
|
| 162 |
+
steps=[int(s) for s in self.h1b_dump_every.split(',') if s.strip()]
|
| 163 |
+
except ValueError as error:
|
| 164 |
+
raise ValueError('H1b dump steps must be comma-separated integers') from error
|
| 165 |
+
if any(s<0 for s in steps): raise ValueError('Negative H1b dump step')
|
| 166 |
if self.kd_algorithm in {'simct','simct_paper'}:
|
| 167 |
if self.kd_ratio != 1 or self.kd_temperature != 1 or self.kd_loss_fn != 'rkl':
|
| 168 |
raise ValueError('SimCT baseline requires kd_ratio=1, kd_temperature=1, kd_loss_fn=rkl')
|
kdflow/arguments/rollout_args.py
CHANGED
|
@@ -5,6 +5,10 @@ from typing import Optional
|
|
| 5 |
@dataclass
|
| 6 |
class RolloutArguments:
|
| 7 |
""" Arguments for rollout (on-policy distillation)."""
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
|
| 9 |
rollout_num_engines: int = field(
|
| 10 |
default=0,
|
|
@@ -30,6 +34,7 @@ class RolloutArguments:
|
|
| 30 |
default=1.0,
|
| 31 |
metadata={"help": "Top-p sampling for rollout."}
|
| 32 |
)
|
|
|
|
| 33 |
temperature: float = field(
|
| 34 |
default=1.0,
|
| 35 |
metadata={"help": "Temperature for rollout."}
|
|
|
|
| 5 |
@dataclass
|
| 6 |
class RolloutArguments:
|
| 7 |
""" Arguments for rollout (on-policy distillation)."""
|
| 8 |
+
|
| 9 |
+
def __post_init__(self):
|
| 10 |
+
if self.eval_math_top_p is not None and not 0 < self.eval_math_top_p <= 1:
|
| 11 |
+
raise ValueError('eval_math_top_p must be in (0,1] or None')
|
| 12 |
|
| 13 |
rollout_num_engines: int = field(
|
| 14 |
default=0,
|
|
|
|
| 34 |
default=1.0,
|
| 35 |
metadata={"help": "Top-p sampling for rollout."}
|
| 36 |
)
|
| 37 |
+
eval_math_top_p: float = field(default=None, metadata={'help':'MATH sampling top-p; None inherits rollout top_p.'})
|
| 38 |
temperature: float = field(
|
| 39 |
default=1.0,
|
| 40 |
metadata={"help": "Temperature for rollout."}
|
kdflow/cli/train_kd_on_policy.py
CHANGED
|
@@ -94,6 +94,7 @@ def train(args):
|
|
| 94 |
if not tokenizer_info.vocab_identical and args.kd.kd_algorithm not in {
|
| 95 |
"simple_ctkd",
|
| 96 |
"byte_align_ctkd",
|
|
|
|
| 97 |
"topk_byte_align_ctkd",
|
| 98 |
"simct",
|
| 99 |
"simct_paper",
|
|
@@ -101,7 +102,7 @@ def train(args):
|
|
| 101 |
}:
|
| 102 |
raise ValueError(
|
| 103 |
"Cross-tokenizer distillation supports simple_ctkd, "
|
| 104 |
-
"byte_align_ctkd, topk_byte_align_ctkd, simct, simct_paper, or an opt-in mp_opd_* mode."
|
| 105 |
)
|
| 106 |
|
| 107 |
environment_rollout = bool(args.rollout.alfworld_config)
|
|
|
|
| 94 |
if not tokenizer_info.vocab_identical and args.kd.kd_algorithm not in {
|
| 95 |
"simple_ctkd",
|
| 96 |
"byte_align_ctkd",
|
| 97 |
+
"h1b_span",
|
| 98 |
"topk_byte_align_ctkd",
|
| 99 |
"simct",
|
| 100 |
"simct_paper",
|
|
|
|
| 102 |
}:
|
| 103 |
raise ValueError(
|
| 104 |
"Cross-tokenizer distillation supports simple_ctkd, "
|
| 105 |
+
"byte_align_ctkd, topk_byte_align_ctkd, simct, simct_paper, h1b_span, or an opt-in mp_opd_* mode."
|
| 106 |
)
|
| 107 |
|
| 108 |
environment_rollout = bool(args.rollout.alfworld_config)
|
kdflow/evaluation/math_evaluator.py
CHANGED
|
@@ -130,10 +130,13 @@ class MathEvaluator:
|
|
| 130 |
def evaluate(self, global_step: int) -> Dict[str, float]:
|
| 131 |
eval_data = self._load_dataset()
|
| 132 |
prompts, labels = self._build_prompts(eval_data)
|
|
|
|
| 133 |
sampling_params = {
|
| 134 |
"max_new_tokens": self.max_tokens,
|
| 135 |
"temperature": self.generate_kwargs.get("temperature", 0.6),
|
| 136 |
-
"top_p":
|
|
|
|
|
|
|
| 137 |
"n": self.n_samples,
|
| 138 |
}
|
| 139 |
|
|
|
|
| 130 |
def evaluate(self, global_step: int) -> Dict[str, float]:
|
| 131 |
eval_data = self._load_dataset()
|
| 132 |
prompts, labels = self._build_prompts(eval_data)
|
| 133 |
+
eval_top_p=getattr(getattr(self.args,'rollout',None),'eval_math_top_p',None)
|
| 134 |
sampling_params = {
|
| 135 |
"max_new_tokens": self.max_tokens,
|
| 136 |
"temperature": self.generate_kwargs.get("temperature", 0.6),
|
| 137 |
+
"top_p": (eval_top_p
|
| 138 |
+
if eval_top_p is not None
|
| 139 |
+
else self.generate_kwargs.get("top_p", 0.95)),
|
| 140 |
"n": self.n_samples,
|
| 141 |
}
|
| 142 |
|
kdflow/h1b/__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
"""Opt-in H1b byte-span supervision; no eager model/runtime imports."""
|
kdflow/h1b/baseline.py
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Previous completed optimizer-window bucket means; no same-window leakage."""
|
| 2 |
+
import torch
|
| 3 |
+
|
| 4 |
+
|
| 5 |
+
class BucketBaseline:
|
| 6 |
+
def __init__(self):
|
| 7 |
+
self.prev=torch.zeros(2,24,dtype=torch.float64)
|
| 8 |
+
self.sums=torch.zeros_like(self.prev)
|
| 9 |
+
self.counts=torch.zeros_like(self.prev)
|
| 10 |
+
|
| 11 |
+
def lookup(self,bucket,which):
|
| 12 |
+
return float(self.prev[0 if which=='path' else 1,bucket])
|
| 13 |
+
|
| 14 |
+
def accumulate(self,bucket,delta,which):
|
| 15 |
+
i=0 if which=='path' else 1
|
| 16 |
+
value=float(delta)
|
| 17 |
+
if not torch.isfinite(torch.tensor(value)): raise FloatingPointError('Nonfinite bucket delta')
|
| 18 |
+
self.sums[i,bucket]+=value; self.counts[i,bucket]+=1
|
| 19 |
+
|
| 20 |
+
def end_window(self):
|
| 21 |
+
# H1b currently rejects multi-rank training; no collective is hidden here.
|
| 22 |
+
self.prev=torch.where(self.counts>0,self.sums/self.counts.clamp_min(1),self.prev)
|
| 23 |
+
self.sums.zero_(); self.counts.zero_()
|
| 24 |
+
|
| 25 |
+
def state_dict(self): return {'prev':self.prev.clone()}
|
| 26 |
+
|
| 27 |
+
def load_state_dict(self,state):
|
| 28 |
+
value=state['prev'].double().cpu()
|
| 29 |
+
if value.shape!=(2,24) or not bool(torch.isfinite(value).all()): raise ValueError('Bad H1b baseline state')
|
| 30 |
+
self.prev=value.clone(); self.sums.zero_(); self.counts.zero_()
|
kdflow/h1b/buckets.py
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Six byte classes times four length bins, stable bucket IDs 0..23."""
|
| 2 |
+
import string
|
| 3 |
+
|
| 4 |
+
|
| 5 |
+
def bucket_of(value):
|
| 6 |
+
if not value: raise ValueError('EOS/empty bytes are not mismatch buckets')
|
| 7 |
+
if all(48 <= b <= 57 for b in value): kind=0
|
| 8 |
+
elif value.isspace(): kind=1
|
| 9 |
+
elif any(b>127 for b in value): kind=2
|
| 10 |
+
elif all(65<=b<=90 or 97<=b<=122 for b in value): kind=3
|
| 11 |
+
elif all(chr(b) in string.punctuation for b in value): kind=4
|
| 12 |
+
else: kind=5
|
| 13 |
+
size=0 if len(value)<=2 else 1 if len(value)<=4 else 2 if len(value)<=8 else 3
|
| 14 |
+
return kind*4+size
|
kdflow/h1b/chunklog.py
ADDED
|
@@ -0,0 +1,68 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Streaming window moments and bounded, deterministic chunk sampling."""
|
| 2 |
+
import hashlib
|
| 3 |
+
import json
|
| 4 |
+
import math
|
| 5 |
+
from pathlib import Path
|
| 6 |
+
|
| 7 |
+
|
| 8 |
+
class Moments:
|
| 9 |
+
def __init__(self): self.n=0; self.s={}; self.ss={}; self.cross=0.; self.floor_n=0; self.floor_sum=0.
|
| 10 |
+
def add(self,row):
|
| 11 |
+
self.n+=1
|
| 12 |
+
for key in ('path','str','path_bl','str_bl','gap','slack_stu','slack_tea','overhang_stu','overhang_tea'):
|
| 13 |
+
v=row[key]; self.s[key]=self.s.get(key,0.)+v; self.ss[key]=self.ss.get(key,0.)+v*v
|
| 14 |
+
self.cross+=row['path']*row['str']
|
| 15 |
+
if abs(row['str'])<.05: self.floor_n+=1; self.floor_sum+=row['gap']
|
| 16 |
+
def summary(self):
|
| 17 |
+
mean={k:v/max(self.n,1) for k,v in self.s.items()}
|
| 18 |
+
var={k:max(0.,self.ss[k]/max(self.n,1)-mean[k]**2) for k in self.s}
|
| 19 |
+
denom=math.sqrt(var.get('path',0.)*var.get('str',0.))
|
| 20 |
+
corr=(self.cross/max(self.n,1)-mean.get('path',0.)*mean.get('str',0.))/denom if denom else 0.
|
| 21 |
+
return dict(count=self.n,mean=mean,variance=var,correlation=corr,
|
| 22 |
+
correlation_defined=bool(denom),bias_floor=self.floor_sum/max(self.floor_n,1),bias_floor_count=self.floor_n)
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
class ChunkLogger:
|
| 26 |
+
def __init__(self,save_path=None,sample_rate=.05,dump_steps=(0,20,60,100)):
|
| 27 |
+
self.root=Path(save_path)/'h1b' if save_path else None
|
| 28 |
+
self.sample_rate=sample_rate; self.dump_steps=set(dump_steps); self.step=0
|
| 29 |
+
self.reset()
|
| 30 |
+
def reset(self):
|
| 31 |
+
self.total=Moments(); self.buckets=[Moments() for _ in range(24)]
|
| 32 |
+
self.samples=[]; self.rows=0; self.dropped=0; self.byte_dropped=0
|
| 33 |
+
self.stu=self.tea=self.strict_stu=self.strict_tea=self.mis_stu=0
|
| 34 |
+
self.reencode_n=0; self.reencode_sum=0.
|
| 35 |
+
def add(self,row):
|
| 36 |
+
self.total.add(row); self.buckets[row['bucket']].add(row)
|
| 37 |
+
key=f"{self.step}:{row['response_hash']}:{row['b0']}:{row['b1']}"
|
| 38 |
+
u=int.from_bytes(hashlib.sha256(key.encode()).digest()[:8],'big')/2**64
|
| 39 |
+
if self.step in self.dump_steps or u<self.sample_rate:
|
| 40 |
+
# JSON/parquet store absent overhang as null, never -Infinity.
|
| 41 |
+
self.samples.append({k:(None if isinstance(v,float) and not math.isfinite(v) else v) for k,v in row.items()})
|
| 42 |
+
def end_window(self,step):
|
| 43 |
+
summary=self.total.summary()
|
| 44 |
+
buckets=[b.summary() for b in self.buckets]
|
| 45 |
+
mean=summary['mean']; var=summary['variance']
|
| 46 |
+
metrics={'strict_cov_stu':self.strict_stu/max(self.stu,1),'strict_cov_tea':self.strict_tea/max(self.tea,1),
|
| 47 |
+
'mis_tok_frac':self.mis_stu/max(self.stu,1),'gap_mean':mean.get('gap',0.),
|
| 48 |
+
'slack_tea':mean.get('slack_tea',0.),'slack_stu':mean.get('slack_stu',0.),
|
| 49 |
+
'bias_floor':summary['bias_floor'],'bias_floor_count':summary['bias_floor_count'],
|
| 50 |
+
'corr_path_str':summary['correlation'],'corr_path_str_defined':float(summary['correlation_defined']),
|
| 51 |
+
'overhang_share_tea':mean.get('overhang_tea',0.),'overhang_share_stu':mean.get('overhang_stu',0.),
|
| 52 |
+
'reencode_match':self.reencode_sum/max(self.reencode_n,1),'reencode_count':self.reencode_n,
|
| 53 |
+
'dropped_byte_mismatch':self.byte_dropped,'window_responses':self.rows,'window_mis_groups':self.total.n}
|
| 54 |
+
for k in ('path','str','path_bl','str_bl'): metrics['var_'+k]=var.get(k,0.)
|
| 55 |
+
for k,b in enumerate(buckets):
|
| 56 |
+
metrics[f'gap_mean_b{k}']=b['mean'].get('gap',0.)
|
| 57 |
+
metrics[f'count_b{k}']=b['count']
|
| 58 |
+
if self.root:
|
| 59 |
+
self.root.mkdir(parents=True,exist_ok=True)
|
| 60 |
+
target=self.root/f'step_{step}.json'
|
| 61 |
+
with target.open('x') as f:
|
| 62 |
+
json.dump(dict(step=step,metrics=metrics,total=summary,buckets=buckets),f,allow_nan=False,indent=2)
|
| 63 |
+
import pyarrow as pa
|
| 64 |
+
import pyarrow.parquet as pq
|
| 65 |
+
table=pa.Table.from_pylist(self.samples) if self.samples else pa.table({'response_hash':pa.array([],type=pa.string())})
|
| 66 |
+
pq.write_table(table,self.root/f'chunks_step_{step}.parquet')
|
| 67 |
+
self.reset(); self.step=step+1
|
| 68 |
+
return {'distill/h1b_'+k:v for k,v in metrics.items()}
|
kdflow/h1b/exact.py
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Finite exhaustive cover-path enumeration with a certified omitted-mass bound."""
|
| 2 |
+
import math
|
| 3 |
+
|
| 4 |
+
|
| 5 |
+
def logadd(a,b):
|
| 6 |
+
if a == -math.inf: return b
|
| 7 |
+
if b == -math.inf: return a
|
| 8 |
+
m=max(a,b)
|
| 9 |
+
return m+math.log(math.exp(a-m)+math.exp(b-m))
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
def exact_chunk_logprob(next_logprobs,chunk,vi,prune=0.,max_nodes=10000):
|
| 13 |
+
"""next_logprobs(tuple_of_branch_ids) -> normalized logp vector.
|
| 14 |
+
|
| 15 |
+
Empty/special token paths are outside this byte-cover event. Duplicate byte
|
| 16 |
+
token IDs are distinct branches. Returned bounds remain valid on pruning or
|
| 17 |
+
node-budget exhaustion; only zero omitted mass merits the label exact.
|
| 18 |
+
"""
|
| 19 |
+
if not chunk or not 0<=prune<1 or max_nodes<1: raise ValueError('Bad exact-search contract')
|
| 20 |
+
frontier=[(0.,0,())]; total=omitted=-math.inf; nodes=0
|
| 21 |
+
while frontier:
|
| 22 |
+
lp,used,path=frontier.pop()
|
| 23 |
+
if nodes>=max_nodes:
|
| 24 |
+
omitted=logadd(omitted,lp)
|
| 25 |
+
continue
|
| 26 |
+
row=next_logprobs(path); nodes+=1
|
| 27 |
+
rem=chunk[used:]
|
| 28 |
+
cover=-math.inf
|
| 29 |
+
for token in vi.cover(rem): cover=logadd(cover,float(row[token]))
|
| 30 |
+
total=logadd(total,lp+cover)
|
| 31 |
+
for k in range(1,len(rem)):
|
| 32 |
+
for token in vi.ids_of(rem[:k]):
|
| 33 |
+
child=lp+float(row[token])
|
| 34 |
+
if prune and child<total+math.log(prune): omitted=logadd(omitted,child)
|
| 35 |
+
else: frontier.append((child,used+k,path+(token,)))
|
| 36 |
+
upper=logadd(total,omitted)
|
| 37 |
+
return dict(log_probability_lower=total,log_probability_upper=upper,
|
| 38 |
+
log_omitted_mass_upper=omitted,nodes=nodes,exact=omitted==-math.inf)
|
kdflow/h1b/groups.py
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Byte-piece groups using Strict's codecs and EOS segmentation."""
|
| 2 |
+
from dataclasses import dataclass
|
| 3 |
+
from itertools import zip_longest
|
| 4 |
+
from kdflow.token_alignment import _split_on_eos
|
| 5 |
+
|
| 6 |
+
|
| 7 |
+
@dataclass(frozen=True)
|
| 8 |
+
class Group:
|
| 9 |
+
kind: str
|
| 10 |
+
s_idx: tuple
|
| 11 |
+
t_idx: tuple
|
| 12 |
+
b0: int
|
| 13 |
+
b1: int
|
| 14 |
+
content: bytes
|
| 15 |
+
eos: bool = False
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
@dataclass(frozen=True)
|
| 19 |
+
class RowGroups:
|
| 20 |
+
groups: tuple
|
| 21 |
+
valid: bool
|
| 22 |
+
reason: str = ''
|
| 23 |
+
student_tokens: int = 0
|
| 24 |
+
teacher_tokens: int = 0
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
def align_byte_span_groups_batch(tea_tokens, stu_tokens, teacher_eos_token,
|
| 28 |
+
student_eos_token, teacher_piece_to_bytes,
|
| 29 |
+
student_piece_to_bytes):
|
| 30 |
+
if len(tea_tokens) != len(stu_tokens):
|
| 31 |
+
raise ValueError('H1b response counts differ')
|
| 32 |
+
rows, so, to = [], 0, 0
|
| 33 |
+
for ts, ss in zip(tea_tokens, stu_tokens):
|
| 34 |
+
groups, reason, sc, tc = [], '', 0, 0
|
| 35 |
+
tsegments, ssegments = _split_on_eos(ts, teacher_eos_token), _split_on_eos(ss, student_eos_token)
|
| 36 |
+
for titem, sitem in zip_longest(tsegments, ssegments):
|
| 37 |
+
if titem is None or sitem is None:
|
| 38 |
+
reason = 'eos_segment_mismatch'
|
| 39 |
+
break
|
| 40 |
+
tseg, teos = titem
|
| 41 |
+
sseg, seos = sitem
|
| 42 |
+
tp = [teacher_piece_to_bytes(p) for _, p in tseg]
|
| 43 |
+
sp = [student_piece_to_bytes(p) for _, p in sseg]
|
| 44 |
+
tc += len(tp)
|
| 45 |
+
sc += len(sp)
|
| 46 |
+
if b''.join(tp) != b''.join(sp):
|
| 47 |
+
reason = 'byte_mismatch'
|
| 48 |
+
break
|
| 49 |
+
if any(not p for p in tp + sp):
|
| 50 |
+
reason = 'empty_piece'
|
| 51 |
+
break
|
| 52 |
+
ti = si = t0 = s0 = b0 = 0
|
| 53 |
+
tend, send = [], []
|
| 54 |
+
for p in tp: tend.append((tend[-1] if tend else 0) + len(p))
|
| 55 |
+
for p in sp: send.append((send[-1] if send else 0) + len(p))
|
| 56 |
+
while ti < len(tp) and si < len(sp):
|
| 57 |
+
if tend[ti] < send[si]: ti += 1; continue
|
| 58 |
+
if tend[ti] > send[si]: si += 1; continue
|
| 59 |
+
tt = tuple(to + tseg[k][0] for k in range(t0, ti + 1))
|
| 60 |
+
st = tuple(so + sseg[k][0] for k in range(s0, si + 1))
|
| 61 |
+
groups.append(Group('strict' if len(tt) == len(st) == 1 else 'mis',
|
| 62 |
+
st, tt, b0, tend[ti], b''.join(tp[t0:ti+1])))
|
| 63 |
+
b0 = tend[ti]
|
| 64 |
+
ti += 1; si += 1; t0, s0 = ti, si
|
| 65 |
+
if teos is not None and seos is not None:
|
| 66 |
+
groups.append(Group('strict', (so+seos,), (to+teos,), b0, b0, b'', True))
|
| 67 |
+
# Content denominators count even a rejected response completely.
|
| 68 |
+
sc = sum(p != student_eos_token for p in ss)
|
| 69 |
+
tc = sum(p != teacher_eos_token for p in ts)
|
| 70 |
+
rows.append(RowGroups(tuple(groups) if not reason else (), not reason, reason, sc, tc))
|
| 71 |
+
so += len(ss); to += len(ts)
|
| 72 |
+
return rows
|
kdflow/h1b/strprob.py
ADDED
|
@@ -0,0 +1,87 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Canonical path and one-exit trunk prefix probability (detached diagnostics)."""
|
| 2 |
+
from dataclasses import dataclass
|
| 3 |
+
import torch
|
| 4 |
+
|
| 5 |
+
|
| 6 |
+
@dataclass
|
| 7 |
+
class ChunkLP:
|
| 8 |
+
log_q_path: torch.Tensor
|
| 9 |
+
log_p_trunk: torch.Tensor
|
| 10 |
+
log_overhang: torch.Tensor
|
| 11 |
+
|
| 12 |
+
|
| 13 |
+
def chunk_logprobs(logp_rows, path_ids, chunk_bytes, vi):
|
| 14 |
+
if not chunk_bytes or not len(path_ids) or len(path_ids) != len(logp_rows):
|
| 15 |
+
raise ValueError('A chunk needs nonempty bytes and one prediction row per path token')
|
| 16 |
+
if b''.join(vi.bytes(i) for i in path_ids) != chunk_bytes:
|
| 17 |
+
raise ValueError('Canonical path does not spell chunk bytes')
|
| 18 |
+
terms, over, used = [], [], 0
|
| 19 |
+
acc = logp_rows.new_zeros(())
|
| 20 |
+
for row, token in zip(logp_rows, path_ids):
|
| 21 |
+
rem = chunk_bytes[used:]
|
| 22 |
+
ids = vi.cover_ids(rem, row.device)
|
| 23 |
+
terms.append(acc + torch.logsumexp(row[ids], 0))
|
| 24 |
+
oi = [i for i in vi.cover(rem) if len(vi.bytes(i)) > len(rem)]
|
| 25 |
+
if oi:
|
| 26 |
+
over.append(acc + torch.logsumexp(row[oi], 0))
|
| 27 |
+
acc = acc + row[int(token)]
|
| 28 |
+
used += len(vi.bytes(token))
|
| 29 |
+
trunk = torch.logsumexp(torch.stack(terms), 0)
|
| 30 |
+
if not bool(torch.isfinite(acc)) or not bool(torch.isfinite(trunk)):
|
| 31 |
+
raise FloatingPointError('Nonfinite H1b path/trunk probability')
|
| 32 |
+
if bool(trunk < acc - 2e-5): raise ArithmeticError('H1b trunk lower than canonical path')
|
| 33 |
+
return ChunkLP(acc, trunk, torch.logsumexp(torch.stack(over),0) if over else acc.new_tensor(-torch.inf))
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
def score_hidden_chunks(head, hidden, labels, chunks, vi, tile_size, *, student):
|
| 37 |
+
"""Keep selected path gradients; reduce trunk tiles without retaining logits.
|
| 38 |
+
|
| 39 |
+
chunks is a list of (flat indices, bytes). Each checkpoint contains only a
|
| 40 |
+
tile, its selected logps and detached covering-event log-masses.
|
| 41 |
+
"""
|
| 42 |
+
from torch.utils.checkpoint import checkpoint
|
| 43 |
+
if tile_size < 1: raise ValueError('Positive head tile size required')
|
| 44 |
+
paths, covers, overs = [], [], []
|
| 45 |
+
label_ids=labels.detach().cpu().tolist()
|
| 46 |
+
flat = []
|
| 47 |
+
for indices,content in chunks:
|
| 48 |
+
used = 0
|
| 49 |
+
ids = [label_ids[i] for i in indices]
|
| 50 |
+
if b''.join(vi.bytes(i) for i in ids) != content:
|
| 51 |
+
raise ValueError('H1b head path/byte mismatch')
|
| 52 |
+
for pos,tid in zip(indices,ids):
|
| 53 |
+
rem=content[used:]
|
| 54 |
+
cov=vi.cover(rem)
|
| 55 |
+
ov=tuple(i for i in cov if len(vi.bytes(i))>len(rem))
|
| 56 |
+
flat.append((pos,tid,cov,ov))
|
| 57 |
+
used += len(vi.bytes(tid))
|
| 58 |
+
for start in range(0,len(flat),tile_size):
|
| 59 |
+
records=flat[start:start+tile_size]
|
| 60 |
+
indices=torch.tensor([r[0] for r in records],device=hidden.device)
|
| 61 |
+
def tile(h, records=records):
|
| 62 |
+
logits=head(h,skip=False) if student else head(h)
|
| 63 |
+
lp=logits.float().log_softmax(-1)
|
| 64 |
+
q=lp[torch.arange(len(records),device=lp.device),[r[1] for r in records]]
|
| 65 |
+
with torch.no_grad():
|
| 66 |
+
c=torch.stack([torch.logsumexp(lp[j,list(r[2])],0) for j,r in enumerate(records)])
|
| 67 |
+
o=torch.stack([torch.logsumexp(lp[j,list(r[3])],0) if r[3] else lp.new_tensor(-torch.inf)
|
| 68 |
+
for j,r in enumerate(records)])
|
| 69 |
+
return q,c,o
|
| 70 |
+
with torch.set_grad_enabled(student and torch.is_grad_enabled()):
|
| 71 |
+
q,c,o=(checkpoint(tile,hidden[indices],use_reentrant=False)
|
| 72 |
+
if student and torch.is_grad_enabled() else tile(hidden[indices]))
|
| 73 |
+
paths.extend(q.unbind()); covers.extend(c.unbind()); overs.extend(o.unbind())
|
| 74 |
+
result=[]
|
| 75 |
+
offset=0
|
| 76 |
+
for indices,_ in chunks:
|
| 77 |
+
n=len(indices)
|
| 78 |
+
qp=torch.stack(paths[offset:offset+n])
|
| 79 |
+
prefix=torch.cat((qp.new_zeros(1),qp.detach().cumsum(0)[:-1]))
|
| 80 |
+
trunk=torch.logsumexp(prefix+torch.stack(covers[offset:offset+n]),0)
|
| 81 |
+
over=torch.logsumexp(prefix+torch.stack(overs[offset:offset+n]),0)
|
| 82 |
+
path=qp.sum()
|
| 83 |
+
if not bool(torch.isfinite(trunk)) or bool(trunk < path.detach()-2e-5):
|
| 84 |
+
raise FloatingPointError('Invalid H1b trunk probability')
|
| 85 |
+
result.append(ChunkLP(path,trunk.detach(),over.detach()))
|
| 86 |
+
offset+=n
|
| 87 |
+
return result
|
kdflow/h1b/vocab_index.py
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Duplicate-preserving byte-prefix vocabulary index."""
|
| 2 |
+
from bisect import bisect_left
|
| 3 |
+
from functools import lru_cache
|
| 4 |
+
from kdflow.token_alignment import get_tokenizer_byte_codec, _special_token_ids
|
| 5 |
+
|
| 6 |
+
|
| 7 |
+
def next_prefix(value):
|
| 8 |
+
for i in range(len(value)-1, -1, -1):
|
| 9 |
+
if value[i] < 255:
|
| 10 |
+
return value[:i] + bytes([value[i]+1])
|
| 11 |
+
return None
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
class VocabIndex:
|
| 15 |
+
def __init__(self, tok, codec=None):
|
| 16 |
+
codec = codec or get_tokenizer_byte_codec(tok)
|
| 17 |
+
self.special = _special_token_ids(tok)
|
| 18 |
+
vocab = tok.get_vocab()
|
| 19 |
+
self._bytes = {i: codec.piece_to_bytes(p) for p,i in vocab.items() if i not in self.special}
|
| 20 |
+
pairs = sorted((b,i) for i,b in self._bytes.items() if b)
|
| 21 |
+
self.keys, self.perm = [b for b,_ in pairs], [i for _,i in pairs]
|
| 22 |
+
self._ids = {}
|
| 23 |
+
for b,i in pairs: self._ids.setdefault(b, []).append(i)
|
| 24 |
+
self.duplicate_byte_strings = sum(len(v)>1 for v in self._ids.values())
|
| 25 |
+
|
| 26 |
+
def bytes(self, token_id):
|
| 27 |
+
return self._bytes[int(token_id)]
|
| 28 |
+
|
| 29 |
+
def ids_of(self, value):
|
| 30 |
+
return tuple(self._ids.get(value, ()))
|
| 31 |
+
|
| 32 |
+
def id_of(self, value):
|
| 33 |
+
ids = self.ids_of(value)
|
| 34 |
+
return ids[0] if ids else None
|
| 35 |
+
|
| 36 |
+
@lru_cache(maxsize=16384)
|
| 37 |
+
def cover(self, rem):
|
| 38 |
+
if not rem: raise ValueError('Empty remaining chunk is not a token event')
|
| 39 |
+
high = next_prefix(rem)
|
| 40 |
+
lo, hi = bisect_left(self.keys, rem), (len(self.keys) if high is None else bisect_left(self.keys, high))
|
| 41 |
+
return tuple(self.perm[lo:hi])
|
| 42 |
+
|
| 43 |
+
def cover_ids(self, rem, device=None):
|
| 44 |
+
import torch
|
| 45 |
+
return torch.tensor(self.cover(rem), dtype=torch.long, device=device)
|
kdflow/ray/train/student_actor.py
CHANGED
|
@@ -239,7 +239,7 @@ class StudentRayActor:
|
|
| 239 |
lm_head = lm_head.to(device).eval()
|
| 240 |
lm_head.requires_grad_(False)
|
| 241 |
logger.info(f"Loaded lm_head ({weight_key}), shape: {lm_head.weight.shape}")
|
| 242 |
-
if self.args.kd.kd_algorithm.startswith('mp_opd_'):
|
| 243 |
head_config = getattr(config, 'text_config', None) or config
|
| 244 |
lm_head._mp_opd_softcap = getattr(head_config, 'final_logit_softcapping', None)
|
| 245 |
return lm_head
|
|
@@ -298,6 +298,9 @@ class StudentRayActor:
|
|
| 298 |
self.strategy.accumulated_gradient = len(train_data)
|
| 299 |
self.strategy.step = 0
|
| 300 |
|
|
|
|
|
|
|
|
|
|
| 301 |
for batch in train_data:
|
| 302 |
micro_batch = self._prepare_micro_batch(batch)
|
| 303 |
|
|
@@ -333,6 +336,9 @@ class StudentRayActor:
|
|
| 333 |
|
| 334 |
status["train/lr"] = self.scheduler.get_last_lr()[0]
|
| 335 |
|
|
|
|
|
|
|
|
|
|
| 336 |
for key in status:
|
| 337 |
status[key] = self.strategy.all_reduce(status[key], op="mean")
|
| 338 |
|
|
|
|
| 239 |
lm_head = lm_head.to(device).eval()
|
| 240 |
lm_head.requires_grad_(False)
|
| 241 |
logger.info(f"Loaded lm_head ({weight_key}), shape: {lm_head.weight.shape}")
|
| 242 |
+
if self.args.kd.kd_algorithm.startswith('mp_opd_') or self.args.kd.kd_algorithm == 'h1b_span':
|
| 243 |
head_config = getattr(config, 'text_config', None) or config
|
| 244 |
lm_head._mp_opd_softcap = getattr(head_config, 'final_logit_softcapping', None)
|
| 245 |
return lm_head
|
|
|
|
| 298 |
self.strategy.accumulated_gradient = len(train_data)
|
| 299 |
self.strategy.step = 0
|
| 300 |
|
| 301 |
+
if hasattr(self.kd_algorithm,'validate_optimizer_window'):
|
| 302 |
+
self.kd_algorithm.validate_optimizer_window(train_data)
|
| 303 |
+
|
| 304 |
for batch in train_data:
|
| 305 |
micro_batch = self._prepare_micro_batch(batch)
|
| 306 |
|
|
|
|
| 336 |
|
| 337 |
status["train/lr"] = self.scheduler.get_last_lr()[0]
|
| 338 |
|
| 339 |
+
if hasattr(self.kd_algorithm,'on_optimizer_window_end'):
|
| 340 |
+
status.update(self.kd_algorithm.on_optimizer_window_end())
|
| 341 |
+
|
| 342 |
for key in status:
|
| 343 |
status[key] = self.strategy.all_reduce(status[key], op="mean")
|
| 344 |
|
kdflow/trainer/data_processor/rollout_data_processor.py
CHANGED
|
@@ -14,6 +14,7 @@ class RolloutDataProcessor:
|
|
| 14 |
CONDITIONAL_EOS_ALGORITHMS = frozenset(
|
| 15 |
{
|
| 16 |
"byte_align_ctkd",
|
|
|
|
| 17 |
"topk_byte_align_ctkd",
|
| 18 |
}
|
| 19 |
)
|
|
@@ -49,13 +50,17 @@ class RolloutDataProcessor:
|
|
| 49 |
"""Save, tokenize and collate raw rollout outputs."""
|
| 50 |
if sample_metadata is not None and len(sample_metadata) != len(outputs):
|
| 51 |
raise ValueError("sample_metadata and outputs must have the same length")
|
|
|
|
|
|
|
|
|
|
|
|
|
| 52 |
self._save_rollout_data(
|
| 53 |
stu_prompts,
|
| 54 |
outputs,
|
| 55 |
labels,
|
| 56 |
global_step,
|
| 57 |
mode,
|
| 58 |
-
sample_metadata=
|
| 59 |
)
|
| 60 |
|
| 61 |
kd_algorithm = self.args.kd.kd_algorithm
|
|
@@ -364,6 +369,10 @@ class RolloutDataProcessor:
|
|
| 364 |
"total_length": torch.FloatTensor([[total_length]]),
|
| 365 |
}
|
| 366 |
stu_multi_modal_inputs = stu_tokens.get("_stu_multi_modal_inputs")
|
|
|
|
|
|
|
|
|
|
|
|
|
| 367 |
if stu_multi_modal_inputs is not None:
|
| 368 |
sample["stu_multi_modal_inputs"] = [stu_multi_modal_inputs]
|
| 369 |
if images:
|
|
|
|
| 14 |
CONDITIONAL_EOS_ALGORITHMS = frozenset(
|
| 15 |
{
|
| 16 |
"byte_align_ctkd",
|
| 17 |
+
"h1b_span",
|
| 18 |
"topk_byte_align_ctkd",
|
| 19 |
}
|
| 20 |
)
|
|
|
|
| 50 |
"""Save, tokenize and collate raw rollout outputs."""
|
| 51 |
if sample_metadata is not None and len(sample_metadata) != len(outputs):
|
| 52 |
raise ValueError("sample_metadata and outputs must have the same length")
|
| 53 |
+
archive_metadata=sample_metadata
|
| 54 |
+
if self.args.kd.kd_algorithm=='h1b_span':
|
| 55 |
+
archive_metadata=[dict(m or {},tea_prompt=t,output_ids=o.get('output_ids',[]))
|
| 56 |
+
for m,t,o in zip(sample_metadata or [{} for _ in outputs],tea_prompts,outputs)]
|
| 57 |
self._save_rollout_data(
|
| 58 |
stu_prompts,
|
| 59 |
outputs,
|
| 60 |
labels,
|
| 61 |
global_step,
|
| 62 |
mode,
|
| 63 |
+
sample_metadata=archive_metadata,
|
| 64 |
)
|
| 65 |
|
| 66 |
kd_algorithm = self.args.kd.kd_algorithm
|
|
|
|
| 369 |
"total_length": torch.FloatTensor([[total_length]]),
|
| 370 |
}
|
| 371 |
stu_multi_modal_inputs = stu_tokens.get("_stu_multi_modal_inputs")
|
| 372 |
+
if self.args.kd.kd_algorithm=='h1b_span':
|
| 373 |
+
tok=getattr(self.student_processor,'tokenizer',self.student_processor)
|
| 374 |
+
encoded=tok(response_text,add_special_tokens=False)['input_ids']
|
| 375 |
+
sample['stu_reencode_match']=torch.FloatTensor([[float(list(encoded)==list(response_ids))]])
|
| 376 |
if stu_multi_modal_inputs is not None:
|
| 377 |
sample["stu_multi_modal_inputs"] = [stu_multi_modal_inputs]
|
| 378 |
if images:
|
tests/test_h1b.py
ADDED
|
@@ -0,0 +1,347 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Offline H1b contracts. Real tokenizer/rollout/B200 gates are separate tools."""
|
| 2 |
+
import copy
|
| 3 |
+
import ast
|
| 4 |
+
from collections import defaultdict
|
| 5 |
+
import hashlib
|
| 6 |
+
import importlib.util
|
| 7 |
+
import json
|
| 8 |
+
import math
|
| 9 |
+
from pathlib import Path
|
| 10 |
+
import random
|
| 11 |
+
import sys
|
| 12 |
+
import tempfile
|
| 13 |
+
import types
|
| 14 |
+
import unittest
|
| 15 |
+
import test_mp_opd_port as fixtures
|
| 16 |
+
|
| 17 |
+
ROOT=Path(__file__).resolve().parents[1]
|
| 18 |
+
if fixtures.HAS_TORCH:
|
| 19 |
+
import torch
|
| 20 |
+
# Test-only namespace: keep the real numerical/logging modules, bypass the
|
| 21 |
+
# utility package's eager tokenizer factory and optional HF dependencies.
|
| 22 |
+
if 'kdflow.utils' not in sys.modules:
|
| 23 |
+
package=types.ModuleType('kdflow.utils'); package.__path__=[str(ROOT/'kdflow/utils')]
|
| 24 |
+
sys.modules['kdflow.utils']=package
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
@unittest.skipUnless(fixtures.HAS_TORCH,'requires existing torch; no bootstrap')
|
| 28 |
+
class H1bTests(unittest.TestCase):
|
| 29 |
+
def fixture(self,arm='a'):
|
| 30 |
+
from kdflow.algorithms.h1b_span import H1bSpanCTKD
|
| 31 |
+
helper=fixtures.MPPortTests(); mp,student,teacher=helper.fixture()
|
| 32 |
+
kd=mp.args.kd
|
| 33 |
+
kd.h1b_arm=arm; kd.h1b_span_lambda=.5; kd.h1b_log_sample_rate=.05; kd.h1b_dump_every='0,20,60,100'
|
| 34 |
+
mp.args.train.save_path=None
|
| 35 |
+
mp.strategy.step=0; mp.strategy.accumulated_gradient=1
|
| 36 |
+
algo=H1bSpanCTKD(mp.strategy,student,teacher,fixtures.Tokenizer(),fixtures.Tokenizer())
|
| 37 |
+
return helper,algo,student,teacher
|
| 38 |
+
|
| 39 |
+
def test_T2_cover_matches_bruteforce_and_ff_edges(self):
|
| 40 |
+
from kdflow.h1b.vocab_index import VocabIndex,next_prefix
|
| 41 |
+
vi=VocabIndex(fixtures.Tokenizer())
|
| 42 |
+
rng=random.Random(42)
|
| 43 |
+
for _ in range(2000):
|
| 44 |
+
rem=bytes(rng.choice([97,98,99,100,255]) for _ in range(rng.randint(1,5)))
|
| 45 |
+
self.assertEqual(set(vi.cover(rem)),{i for i,b in vi._bytes.items() if b.startswith(rem) and b})
|
| 46 |
+
self.assertIsNone(next_prefix(b'\xff\xff'))
|
| 47 |
+
self.assertEqual(next_prefix(b'a\xff'),b'b')
|
| 48 |
+
with self.assertRaises(ValueError): vi.cover(b'')
|
| 49 |
+
|
| 50 |
+
def test_T3_strict_groups_match_author_fixture_eos_and_offsets(self):
|
| 51 |
+
from kdflow.h1b.groups import align_byte_span_groups_batch
|
| 52 |
+
from kdflow.token_alignment import align_byte_span_batch_1to1,get_tokenizer_byte_codec
|
| 53 |
+
tok=fixtures.Tokenizer(); codec=get_tokenizer_byte_codec(tok).piece_to_bytes
|
| 54 |
+
for tea,stu in [([['ab','c','<eos>'],['a','b']],[['a','b','c','<eos>'],['ab']]),
|
| 55 |
+
([['a'],[]],[['a'],[]]),([['<eos>','ab','<eos>']],[['<eos>','a','b','<eos>']])]:
|
| 56 |
+
rows=align_byte_span_groups_batch(tea,stu,tok.eos_token,tok.eos_token,codec,codec)
|
| 57 |
+
groups=[g for r in rows for g in r.groups if g.kind=='strict']
|
| 58 |
+
a=([g.t_idx[0] for g in groups],[g.s_idx[0] for g in groups])
|
| 59 |
+
self.assertEqual(a,align_byte_span_batch_1to1(tea,stu,tok.eos_token,tok.eos_token,codec,codec))
|
| 60 |
+
|
| 61 |
+
def test_T4_trunk_and_exact_enumeration_with_duplicate_byte_ids(self):
|
| 62 |
+
from kdflow.h1b.vocab_index import VocabIndex
|
| 63 |
+
from kdflow.h1b.strprob import chunk_logprobs
|
| 64 |
+
from kdflow.h1b.exact import exact_chunk_logprob
|
| 65 |
+
# 'A' and 'a' map to the same bytes. Exact proper-prefix enumeration
|
| 66 |
+
# must branch on both IDs rather than taking only id_of(...).
|
| 67 |
+
class Tok(fixtures.Tokenizer):
|
| 68 |
+
pieces={0:'<eos>',1:'a',2:'b',3:'ab',4:'A',5:'abc',6:'ba',7:'<pad>'}
|
| 69 |
+
codec=types.SimpleNamespace(piece_to_bytes=lambda s:s.lower().encode())
|
| 70 |
+
vi=VocabIndex(Tok(),codec)
|
| 71 |
+
torch.manual_seed(19)
|
| 72 |
+
table={():torch.randn(8).log_softmax(0),(1,):torch.randn(8).log_softmax(0),(4,):torch.randn(8).log_softmax(0)}
|
| 73 |
+
lp=chunk_logprobs(torch.stack([table[()],table[(1,)]]),[1,2],b'ab',vi)
|
| 74 |
+
expected=torch.logsumexp(torch.stack([table[()][3],table[()][5],table[()][1]+torch.logsumexp(table[(1,)][[2,6]],0)]),0)
|
| 75 |
+
torch.testing.assert_close(lp.log_p_trunk,expected)
|
| 76 |
+
result=exact_chunk_logprob(lambda ids:table[ids],b'ab',vi)
|
| 77 |
+
exact=math.exp(float(expected))+math.exp(float(table[()][4]+torch.logsumexp(table[(4,)][[2,6]],0)))
|
| 78 |
+
self.assertAlmostEqual(result['log_probability_lower'],math.log(exact),places=6)
|
| 79 |
+
self.assertTrue(result['exact'])
|
| 80 |
+
self.assertGreaterEqual(result['log_probability_lower']+1e-6,float(lp.log_p_trunk))
|
| 81 |
+
bounded=exact_chunk_logprob(lambda ids:table[ids],b'ab',vi,max_nodes=1)
|
| 82 |
+
self.assertLessEqual(bounded['log_probability_lower'],math.log(exact)+1e-6)
|
| 83 |
+
self.assertGreaterEqual(bounded['log_probability_upper']+1e-6,math.log(exact))
|
| 84 |
+
|
| 85 |
+
def test_T4_thirty_token_lm_fifty_strings(self):
|
| 86 |
+
import itertools
|
| 87 |
+
from kdflow.h1b.vocab_index import VocabIndex
|
| 88 |
+
from kdflow.h1b.strprob import chunk_logprobs
|
| 89 |
+
from kdflow.h1b.exact import exact_chunk_logprob
|
| 90 |
+
words=[''.join(x) for size in range(1,5) for x in itertools.product('ab',repeat=size)]
|
| 91 |
+
class Tok(fixtures.Tokenizer):
|
| 92 |
+
pieces={0:'<eos>',31:'<pad>',**{i+1:w for i,w in enumerate(words)}}
|
| 93 |
+
all_special_ids=[0,31]; pad_token_id=31
|
| 94 |
+
vi=VocabIndex(Tok()); rng=random.Random(17)
|
| 95 |
+
def oracle(path):
|
| 96 |
+
seed=int.from_bytes(hashlib.sha256(bytes(path)).digest()[:4],'big')
|
| 97 |
+
return torch.randn(32,generator=torch.Generator().manual_seed(seed),dtype=torch.float64).log_softmax(0)
|
| 98 |
+
for _ in range(50):
|
| 99 |
+
chunk=''.join(rng.choice('ab') for _ in range(rng.randint(1,4))).encode()
|
| 100 |
+
path=[vi.id_of(bytes([b])) for b in chunk]
|
| 101 |
+
lp=chunk_logprobs(torch.stack([oracle(tuple(path[:j])) for j in range(len(path))]),path,chunk,vi)
|
| 102 |
+
acc=1.; brute=0.
|
| 103 |
+
for j,token in enumerate(path):
|
| 104 |
+
row=oracle(tuple(path[:j])).exp(); rem=chunk[j:]
|
| 105 |
+
brute+=acc*sum(float(row[i]) for i,b in vi._bytes.items() if b.startswith(rem))
|
| 106 |
+
acc*=float(row[token])
|
| 107 |
+
self.assertAlmostEqual(float(lp.log_p_trunk),math.log(brute),places=6)
|
| 108 |
+
exact=exact_chunk_logprob(oracle,chunk,vi)
|
| 109 |
+
self.assertGreaterEqual(exact['log_probability_lower']+1e-6,float(lp.log_p_trunk))
|
| 110 |
+
|
| 111 |
+
def test_T5_same_tokenizer_all_arms_have_zero_span(self):
|
| 112 |
+
reference=None
|
| 113 |
+
for arm in ('a','b','b_bl','c_str','c'):
|
| 114 |
+
helper,algo,_,_=self.fixture(arm); batch=helper.batch()
|
| 115 |
+
batch['tea_input_ids']=batch['stu_input_ids'].clone()
|
| 116 |
+
batch['tea_loss_mask']=batch['stu_loss_mask'].clone()
|
| 117 |
+
batch['tea_attn_mask']=batch['stu_attn_mask'].clone()
|
| 118 |
+
batch['teacher_hiddens']=torch.randn(int(batch['tea_loss_mask'].sum()),4)
|
| 119 |
+
info=algo.training_step(batch)
|
| 120 |
+
self.assertEqual(float(info['train/h1b_span_loss'].detach()),0.)
|
| 121 |
+
self.assertEqual(float(info['distill/h1b_mis_groups']),0.)
|
| 122 |
+
value=info['train/loss'].detach()
|
| 123 |
+
if reference is None: reference=value
|
| 124 |
+
else: torch.testing.assert_close(value,reference)
|
| 125 |
+
|
| 126 |
+
def test_T6_baseline_previous_window_and_evaluation_immutability(self):
|
| 127 |
+
helper,algo,student,_=self.fixture('c')
|
| 128 |
+
before=algo.baseline.prev.clone()
|
| 129 |
+
algo.training_step(helper.batch())
|
| 130 |
+
torch.testing.assert_close(algo.baseline.prev,before)
|
| 131 |
+
algo.on_optimizer_window_end()
|
| 132 |
+
self.assertTrue(bool(algo.baseline.prev.any()))
|
| 133 |
+
state=copy.deepcopy(algo.state_dict()); counts=algo.baseline.counts.clone()
|
| 134 |
+
student.eval()
|
| 135 |
+
with torch.no_grad(): algo.training_step(helper.batch())
|
| 136 |
+
torch.testing.assert_close(algo.baseline.prev,state['baseline']['prev'])
|
| 137 |
+
torch.testing.assert_close(algo.baseline.counts,counts)
|
| 138 |
+
self.assertEqual(algo.chunk_logger.rows,0)
|
| 139 |
+
algo.load_state_dict(state)
|
| 140 |
+
|
| 141 |
+
def test_T7_span_surrogate_gradient_equals_mse(self):
|
| 142 |
+
helper,algo,student,_=self.fixture('b')
|
| 143 |
+
algo.replay_enabled=True
|
| 144 |
+
info=algo.training_step(helper.batch())
|
| 145 |
+
expected=sum(algo.lam*(r['log_q']-(r['log_q'].detach()-r['path']))**2 for r in algo.replay_records)/helper.batch()['avg_micro_batch_token_num']
|
| 146 |
+
a=torch.autograd.grad(info['train/h1b_span_loss'],tuple(student.parameters()),retain_graph=True)
|
| 147 |
+
b=torch.autograd.grad(expected,tuple(student.parameters()))
|
| 148 |
+
for x,y in zip(a,b): torch.testing.assert_close(x,y,atol=1e-6,rtol=1e-5)
|
| 149 |
+
|
| 150 |
+
def test_T8_arm_a_loss_and_all_gradients_equal_strict(self):
|
| 151 |
+
from kdflow.algorithms.byte_align_ctkd import ByteAlignCrossTokenizerKD
|
| 152 |
+
helper,algo,student,teacher=self.fixture('a')
|
| 153 |
+
strict=ByteAlignCrossTokenizerKD(algo.strategy,student,teacher,fixtures.Tokenizer(),fixtures.Tokenizer())
|
| 154 |
+
batch=helper.batch(2)
|
| 155 |
+
a=algo.training_step(batch)['train/loss']; b=strict.training_step(batch)['train/loss']
|
| 156 |
+
torch.testing.assert_close(a,b,atol=1e-6,rtol=1e-6)
|
| 157 |
+
for x,y in zip(torch.autograd.grad(a,tuple(student.parameters())),torch.autograd.grad(b,tuple(student.parameters()))):
|
| 158 |
+
torch.testing.assert_close(x,y,atol=1e-6,rtol=1e-6)
|
| 159 |
+
self.assertIsNone(teacher.weight.grad)
|
| 160 |
+
|
| 161 |
+
def test_byte_mismatch_drops_complete_response_including_eos(self):
|
| 162 |
+
helper,algo,student,_=self.fixture('b')
|
| 163 |
+
batch=helper.batch(); batch['tea_input_ids'][0,1]=5
|
| 164 |
+
info=algo.training_step(batch)
|
| 165 |
+
self.assertEqual(float(info['train/loss'].detach()),0.)
|
| 166 |
+
self.assertEqual(float(info['distill/h1b_dropped_responses']),1.)
|
| 167 |
+
info['train/loss'].backward()
|
| 168 |
+
self.assertTrue(all(p.grad is None or not bool(p.grad.any()) for p in student.parameters()))
|
| 169 |
+
|
| 170 |
+
def test_tile_recomputation_preserves_path_gradient_and_statistics(self):
|
| 171 |
+
helper,algo,student,_=self.fixture('c_str')
|
| 172 |
+
batch=helper.batch(); a=algo.training_step(batch)
|
| 173 |
+
ga=torch.autograd.grad(a['train/loss'],tuple(student.parameters()))
|
| 174 |
+
algo.args.train.chunked_loss_size=1
|
| 175 |
+
b=algo.training_step(batch); gb=torch.autograd.grad(b['train/loss'],tuple(student.parameters()))
|
| 176 |
+
torch.testing.assert_close(a['train/loss'],b['train/loss'])
|
| 177 |
+
for x,y in zip(ga,gb): torch.testing.assert_close(x,y,atol=1e-6,rtol=1e-5)
|
| 178 |
+
|
| 179 |
+
def test_window_statistics_use_chunk_counts_not_microbatch_means(self):
|
| 180 |
+
from kdflow.h1b.chunklog import ChunkLogger
|
| 181 |
+
log=ChunkLogger(sample_rate=0,dump_steps=())
|
| 182 |
+
for v in (1.,2.,9.):
|
| 183 |
+
row=dict(path=v,str=v/2,path_bl=v-1,str_bl=v/2-2,gap=v/2,slack_stu=1.,slack_tea=1+v/2,
|
| 184 |
+
overhang_stu=0.,overhang_tea=0.,bucket=0,response_hash=str(v),b0=0,b1=1)
|
| 185 |
+
log.add(row)
|
| 186 |
+
out=log.end_window(0)
|
| 187 |
+
self.assertEqual(out['distill/h1b_gap_mean'],2.)
|
| 188 |
+
self.assertAlmostEqual(out['distill/h1b_var_path'],(1+4+81)/3-16)
|
| 189 |
+
|
| 190 |
+
def test_actual_actor_hook_runs_after_one_update_and_not_during_eval(self):
|
| 191 |
+
from kdflow.utils.logging_utils import normalize_eval_metrics
|
| 192 |
+
helper,algo,student,_=self.fixture('c')
|
| 193 |
+
tree=ast.parse((ROOT/'kdflow/ray/train/student_actor.py').read_text())
|
| 194 |
+
scope=dict(torch=torch,defaultdict=defaultdict,normalize_eval_metrics=normalize_eval_metrics)
|
| 195 |
+
methods={}
|
| 196 |
+
for name in ('fit','evaluate'):
|
| 197 |
+
node=next(n for n in ast.walk(tree) if isinstance(n,ast.FunctionDef) and n.name==name)
|
| 198 |
+
node.decorator_list=[]
|
| 199 |
+
exec(compile(ast.Module(body=[node],type_ignores=[]),'actual_h1b_actor','exec'),scope)
|
| 200 |
+
methods[name]=scope[name]
|
| 201 |
+
strategy=algo.strategy; strategy.accumulated_gradient=2
|
| 202 |
+
algo.args.train.use_dynamic_bsz=False; algo.args.kd.use_ema_teacher=False
|
| 203 |
+
optimizer=torch.optim.SGD(student.parameters(),lr=.01); updates=[]
|
| 204 |
+
def backward(loss,*_):
|
| 205 |
+
strategy.step=(strategy.step+1)%2
|
| 206 |
+
(loss/2).backward()
|
| 207 |
+
def step(*_):
|
| 208 |
+
if strategy.step==0: optimizer.step(); optimizer.zero_grad(); updates.append(1)
|
| 209 |
+
strategy.backward=backward; strategy.optimizer_step=step
|
| 210 |
+
strategy.all_reduce=lambda x,**_:x
|
| 211 |
+
actor=types.SimpleNamespace(student=student,kd_algorithm=algo,args=algo.args,strategy=strategy,
|
| 212 |
+
optim=optimizer,scheduler=types.SimpleNamespace(get_last_lr=lambda:[.01]),_prepare_micro_batch=lambda b:dict(b))
|
| 213 |
+
original=algo.on_optimizer_window_end
|
| 214 |
+
def end():
|
| 215 |
+
self.assertEqual(len(updates),1)
|
| 216 |
+
return original()
|
| 217 |
+
algo.on_optimizer_window_end=end
|
| 218 |
+
batches=[helper.batch(),helper.batch(2)]
|
| 219 |
+
result=methods['fit'](actor,batches)
|
| 220 |
+
self.assertEqual(result['distill/h1b_window_responses'],3.)
|
| 221 |
+
self.assertEqual(algo.window,1)
|
| 222 |
+
previous=algo.baseline.prev.clone()
|
| 223 |
+
with torch.no_grad(): methods['evaluate'](actor,batches)
|
| 224 |
+
self.assertEqual(algo.window,1)
|
| 225 |
+
torch.testing.assert_close(algo.baseline.prev,previous)
|
| 226 |
+
|
| 227 |
+
def test_math_eval_override_never_changes_rollout_top_p(self):
|
| 228 |
+
from kdflow.arguments.rollout_args import RolloutArguments
|
| 229 |
+
tree=ast.parse((ROOT/'kdflow/evaluation/math_evaluator.py').read_text())
|
| 230 |
+
method=next(n for n in ast.walk(tree) if isinstance(n,ast.FunctionDef) and n.name=='evaluate')
|
| 231 |
+
assignments=[n for n in method.body if isinstance(n,ast.Assign) and any(isinstance(t,ast.Name) and t.id in {'eval_top_p','sampling_params'} for t in n.targets)]
|
| 232 |
+
for override,expected in ((None,1.),(.95,.95)):
|
| 233 |
+
actor=types.SimpleNamespace(args=types.SimpleNamespace(rollout=RolloutArguments(top_p=1.,eval_math_top_p=override)),
|
| 234 |
+
max_tokens=8192,n_samples=32,generate_kwargs={'top_p':1.,'temperature':1.})
|
| 235 |
+
scope={'self':actor}
|
| 236 |
+
exec(compile(ast.Module(body=assignments,type_ignores=[]),'actual_math_sampling','exec'),scope)
|
| 237 |
+
self.assertEqual(scope['sampling_params']['top_p'],expected)
|
| 238 |
+
self.assertEqual(actor.generate_kwargs['top_p'],1.)
|
| 239 |
+
with self.assertRaises(ValueError): RolloutArguments(eval_math_top_p=0)
|
| 240 |
+
|
| 241 |
+
def test_fit_curve_and_analysis_handle_known_direction_and_zero_vectors(self):
|
| 242 |
+
import numpy as np
|
| 243 |
+
from tools.h1b.fit_cb import fit_curve,analyze_matrices
|
| 244 |
+
sizes=[64,128,256,512,1024,2048,4096]
|
| 245 |
+
fit=fit_curve(sizes,[.7/math.sqrt(1+300/b) for b in sizes])
|
| 246 |
+
self.assertAlmostEqual(fit['c_infinity'],.7,places=2)
|
| 247 |
+
x=np.tile(np.array([1.,2.,3.]),(8,1))
|
| 248 |
+
report=analyze_matrices(x,x,x,x)
|
| 249 |
+
self.assertTrue(all(abs(r['cosine']-1)<1e-9 for r in report['curve']))
|
| 250 |
+
zero=analyze_matrices(x,x*0,x,x)
|
| 251 |
+
self.assertTrue(all(r['cosine'] is None for r in zero['curve']))
|
| 252 |
+
|
| 253 |
+
def test_replay_analysis_cli_on_complete_small_saved_sketches(self):
|
| 254 |
+
from tools.h1b.fit_cb import main
|
| 255 |
+
torch.manual_seed(3)
|
| 256 |
+
keys=['strict','path','str','base_path','base_str','strict_0','strict_1']
|
| 257 |
+
with tempfile.TemporaryDirectory() as tmp:
|
| 258 |
+
root=Path(tmp); rows=[]
|
| 259 |
+
for i in range(8):
|
| 260 |
+
files={}
|
| 261 |
+
for key in keys:
|
| 262 |
+
file=f'{i}_{key}.pt'; torch.save(torch.randn(16),root/file); files[key]=file
|
| 263 |
+
rows.append(dict(files=files,tokens=20+i))
|
| 264 |
+
(root/'manifest.json').write_text(json.dumps(dict(records=rows,groups=8,group_size=64,bucket_classes=[])))
|
| 265 |
+
main(['--replay',tmp,'--output',str(root/'curves.json'),'--bootstrap','2'])
|
| 266 |
+
result=json.loads((root/'curves.json').read_text())
|
| 267 |
+
self.assertEqual(set(result['results']),{'path','str','path_bl','str_bl'})
|
| 268 |
+
self.assertIsNotNone(result['cos_mean_path_str'])
|
| 269 |
+
|
| 270 |
+
def test_h1b_rollout_preserves_eos_policy_and_reports_raw_reencoding(self):
|
| 271 |
+
from typing import Any,Dict,List,Optional
|
| 272 |
+
tree=ast.parse((ROOT/'kdflow/trainer/data_processor/rollout_data_processor.py').read_text())
|
| 273 |
+
node=next(n for n in tree.body if isinstance(n,ast.ClassDef))
|
| 274 |
+
scope=dict(torch=torch,Any=Any,Dict=Dict,List=List,Optional=Optional)
|
| 275 |
+
exec(compile(ast.Module(body=[node],type_ignores=[]),'actual_h1b_processor','exec'),scope)
|
| 276 |
+
class Tok:
|
| 277 |
+
eos_token_id=0
|
| 278 |
+
def __call__(self,text,return_tensors=None,**kwargs):
|
| 279 |
+
ids=[{'p':6,'a':1,'b':2,'c':3}[c] for c in text]
|
| 280 |
+
return {'input_ids':torch.tensor([ids]),'attention_mask':torch.ones(1,len(ids),dtype=torch.long)} if return_tensors else {'input_ids':ids}
|
| 281 |
+
obj=object.__new__(scope['RolloutDataProcessor'])
|
| 282 |
+
obj.args=types.SimpleNamespace(kd=types.SimpleNamespace(kd_algorithm='h1b_span'))
|
| 283 |
+
obj.student_processor=Tok(); obj.teacher_processors={}; obj.is_same_tokenizer=True
|
| 284 |
+
sample=obj._build_rollout_sample('p','p',dict(text='abc',output_ids=[1,2,3]),None,append_eos=False)
|
| 285 |
+
self.assertEqual(float(sample['stu_reencode_match']),1.)
|
| 286 |
+
self.assertEqual(sample['stu_input_ids'].tolist(),[6,1,2,3])
|
| 287 |
+
sample=obj._build_rollout_sample('p','p',dict(text='abc',output_ids=[2,3]),None,append_eos=True)
|
| 288 |
+
self.assertEqual(float(sample['stu_reencode_match']),0.)
|
| 289 |
+
self.assertEqual(sample['stu_input_ids'][-1],0)
|
| 290 |
+
|
| 291 |
+
def test_KV_oracle_keeps_sibling_prefix_cache_immutable(self):
|
| 292 |
+
from tools.h1b.common import HFPrefixOracle
|
| 293 |
+
class Base(torch.nn.Module):
|
| 294 |
+
def forward(self,ids,past_key_values=None,use_cache=False,**kwargs):
|
| 295 |
+
cache={'ids':[]} if past_key_values is None else past_key_values
|
| 296 |
+
cache['ids'].extend(ids[0].tolist()) # Deliberate in-place cache mutation.
|
| 297 |
+
x=torch.tensor([sum(cache['ids']),len(cache['ids']),cache['ids'][-1],1.],dtype=torch.float32)
|
| 298 |
+
return types.SimpleNamespace(last_hidden_state=x[None,None,:],past_key_values=cache if use_cache else None)
|
| 299 |
+
class Model(torch.nn.Module):
|
| 300 |
+
def __init__(self): super().__init__(); self.base_model=Base(); self.head=torch.nn.Linear(4,8)
|
| 301 |
+
def get_output_embeddings(self): return self.head
|
| 302 |
+
model=Model(); cached=HFPrefixOracle(model,[6,1,2],cache_depth=2); direct=HFPrefixOracle(model,[6,1,2],use_cache=False)
|
| 303 |
+
for branch in ((),(1,),(1,2),(1,2,3),(2,),(2,3),(1,)):
|
| 304 |
+
torch.testing.assert_close(cached(branch),direct(branch))
|
| 305 |
+
self.assertEqual(cached.stack[0][2]['ids'],[6,1,2])
|
| 306 |
+
|
| 307 |
+
def test_old_archive_prompt_recovery_requires_exact_unambiguous_match(self):
|
| 308 |
+
from tools.h1b.enrich_archive import recover
|
| 309 |
+
records=[dict(prompt='S:x',output='abc',finish_reason='stop')]
|
| 310 |
+
rows=[dict(s='S:x',t='T:x')]
|
| 311 |
+
result=recover(records,rows,lambda r:r['s'],lambda r:r['t'])
|
| 312 |
+
self.assertEqual(result[0]['tea_prompt'],'T:x')
|
| 313 |
+
self.assertEqual(result[0]['output'],'abc')
|
| 314 |
+
self.assertNotIn('output_ids',result[0])
|
| 315 |
+
with self.assertRaises(ValueError): recover(records,[],lambda r:r['s'],lambda r:r['t'])
|
| 316 |
+
with self.assertRaises(ValueError): recover(records,rows+[dict(s='S:x',t='T:y')],lambda r:r['s'],lambda r:r['t'])
|
| 317 |
+
|
| 318 |
+
def test_T11_sketch_cosine_norm_and_chunk_invariance(self):
|
| 319 |
+
from tools.h1b.sketch import count_sketch
|
| 320 |
+
torch.manual_seed(9); a=torch.randn(2000); b=a+.2*torch.randn_like(a)
|
| 321 |
+
sa,na=count_sketch([('x',a)],chunk_size=17)
|
| 322 |
+
sb,nb=count_sketch([('x',b)],chunk_size=99)
|
| 323 |
+
sa2,_=count_sketch([('x',a)],chunk_size=33)
|
| 324 |
+
torch.testing.assert_close(sa,sa2)
|
| 325 |
+
self.assertLess(abs(float(sa.norm())/na-1),.01)
|
| 326 |
+
self.assertLess(abs(float(sb.norm())/nb-1),.01)
|
| 327 |
+
self.assertLess(abs(float(torch.nn.functional.cosine_similarity(sa,sb,dim=0)-torch.nn.functional.cosine_similarity(a,b,dim=0))),.01)
|
| 328 |
+
|
| 329 |
+
def test_presets_share_full_workload_and_separate_eval_top_p(self):
|
| 330 |
+
from tools.launch_single_b200 import parser,build_plan,H1B_ARMS
|
| 331 |
+
original=build_plan(parser().parse_args(['--algorithm','strict-full','--dry-run']))
|
| 332 |
+
allowed={'kd_algorithm','h1b_arm','h1b_span_lambda','h1b_log_sample_rate','h1b_dump_every','top_p','eval_math_top_p','save_path','ckpt_path'}
|
| 333 |
+
for name,arm in H1B_ARMS.items():
|
| 334 |
+
plan=build_plan(parser().parse_args(['--algorithm',name,'--dry-run']))
|
| 335 |
+
self.assertEqual(plan['flags']['h1b_arm'],arm)
|
| 336 |
+
self.assertEqual(plan['flags']['top_p'],1.)
|
| 337 |
+
self.assertEqual(plan['flags']['eval_math_top_p'],.95)
|
| 338 |
+
for key,value in original['flags'].items():
|
| 339 |
+
if key not in allowed: self.assertEqual(plan['flags'][key],value)
|
| 340 |
+
|
| 341 |
+
def test_source_manifest_frozen_strict(self):
|
| 342 |
+
manifest=json.loads((ROOT/'docs/H1B_SOURCE.json').read_text())
|
| 343 |
+
for row in manifest['files']+manifest['frozen_strict']:
|
| 344 |
+
self.assertEqual(hashlib.sha256((ROOT/row['path']).read_bytes()).hexdigest(),row['sha256'])
|
| 345 |
+
|
| 346 |
+
|
| 347 |
+
if __name__=='__main__': unittest.main()
|
tools/h1b/__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
"""Offline H1b tools. Invoke from the source root with python -m tools.h1b.<tool>."""
|
tools/h1b/canary.py
ADDED
|
@@ -0,0 +1,67 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Separate five-step diagnostic (T10); never a reduced-workload reproduction."""
|
| 2 |
+
import argparse
|
| 3 |
+
import json
|
| 4 |
+
import math
|
| 5 |
+
import os
|
| 6 |
+
import re
|
| 7 |
+
from pathlib import Path
|
| 8 |
+
import subprocess
|
| 9 |
+
import time
|
| 10 |
+
from tools.launch_single_b200 import parser,build_plan,validate_files,run_environment,H1B_ARMS
|
| 11 |
+
|
| 12 |
+
|
| 13 |
+
def main(argv=None):
|
| 14 |
+
p=argparse.ArgumentParser(description=__doc__)
|
| 15 |
+
p.add_argument('--algorithm',choices=H1B_ARMS,required=True); p.add_argument('--runtime',required=True)
|
| 16 |
+
p.add_argument('--root',default='/workspace/storage-shared/nlp/tungks/ctopd_all')
|
| 17 |
+
p.add_argument('--run-id',required=True); p.add_argument('--execute',action='store_true')
|
| 18 |
+
args=p.parse_args(argv)
|
| 19 |
+
if not args.run_id.startswith('canary-h1b-'): p.error('Use a distinct canary-h1b-* run ID')
|
| 20 |
+
plan=build_plan(parser().parse_args(['--algorithm',args.algorithm,'--runtime',args.runtime,'--root',args.root,'--run-id',args.run_id]))
|
| 21 |
+
plan['flags']['max_steps']=5
|
| 22 |
+
plan['qualification']='T10 five-step diagnostic only; full 512/8192 workload per step; not reproduction'
|
| 23 |
+
command=['bash',str(Path(args.runtime)/'python-b200.sh'),'-u','-m','kdflow.cli.train_kd_on_policy']
|
| 24 |
+
for k,v in plan['flags'].items(): command.extend(['--'+k,str(v)])
|
| 25 |
+
plan['command']=command
|
| 26 |
+
if not args.execute:
|
| 27 |
+
print(json.dumps(plan,indent=2)); return
|
| 28 |
+
source=Path(plan['source'])
|
| 29 |
+
if subprocess.check_output(['git','-C',str(source),'status','--porcelain'],text=True).strip():
|
| 30 |
+
raise RuntimeError('Commit source before diagnostic execution')
|
| 31 |
+
validate_files(plan)
|
| 32 |
+
env=run_environment()
|
| 33 |
+
probe="import torch; assert torch.cuda.device_count()==1; assert 'B200' in torch.cuda.get_device_name(0); print(torch.cuda.get_device_name(0))"
|
| 34 |
+
subprocess.run(['bash',str(Path(args.runtime)/'python-b200.sh'),'-c',probe],env=env,check=True)
|
| 35 |
+
root=Path(plan['output']); root.mkdir(parents=True,exist_ok=False)
|
| 36 |
+
plan['source_commit']=subprocess.check_output(['git','-C',str(source),'rev-parse','HEAD'],text=True).strip()
|
| 37 |
+
(root/'canary-plan.json').write_text(json.dumps(plan,indent=2))
|
| 38 |
+
start=time.monotonic()
|
| 39 |
+
with (root/'canary.log').open('x') as log:
|
| 40 |
+
with subprocess.Popen(command,env=env,cwd=source,stdout=subprocess.PIPE,stderr=subprocess.STDOUT,text=True) as process:
|
| 41 |
+
for line in process.stdout:
|
| 42 |
+
log.write(line); log.flush(); print(line,end='',flush=True)
|
| 43 |
+
code=process.wait()
|
| 44 |
+
receipt=dict(source_commit=plan['source_commit'],algorithm=args.algorithm,exit_code=code,
|
| 45 |
+
elapsed_seconds=time.monotonic()-start,passed=False,steps=[],qualification='Company-node canary only; no metric-match claim')
|
| 46 |
+
import pyarrow.parquet as pq
|
| 47 |
+
if code==0:
|
| 48 |
+
values=re.findall(r'train/(?:loss|kd_loss|h1b_span_loss|grad_norm):\s*([^\s,]+)',(root/'canary.log').read_text())
|
| 49 |
+
if not values or not all(math.isfinite(float(v)) for v in values):
|
| 50 |
+
raise ValueError('Missing/nonfinite train loss/gradient metrics')
|
| 51 |
+
for step in range(5):
|
| 52 |
+
path=root/'checkpoints'/'h1b'/f'step_{step}.json'
|
| 53 |
+
summary=json.loads(path.read_text()); metrics=summary['metrics']
|
| 54 |
+
required={'gap_mean','slack_stu','slack_tea','bias_floor','var_path','var_str','var_path_bl','var_str_bl',
|
| 55 |
+
'corr_path_str','overhang_share_stu','overhang_share_tea','reencode_match','dropped_byte_mismatch'}
|
| 56 |
+
if not required<=metrics.keys() or not all(math.isfinite(float(v)) for v in metrics.values()):
|
| 57 |
+
raise ValueError('Missing/nonfinite canary metrics')
|
| 58 |
+
file=path.parent/f'chunks_step_{step}.parquet'
|
| 59 |
+
rows=pq.read_table(file).num_rows
|
| 60 |
+
receipt['steps'].append(dict(step=step,sampled_chunks=rows,dump_bytes=file.stat().st_size))
|
| 61 |
+
receipt['passed']=True
|
| 62 |
+
(root/'canary-receipt.json').write_text(json.dumps(receipt,indent=2))
|
| 63 |
+
if not receipt['passed']: raise SystemExit('Canary failed; preserve log and receipt')
|
| 64 |
+
print(json.dumps(receipt))
|
| 65 |
+
|
| 66 |
+
|
| 67 |
+
if __name__=='__main__': main()
|
tools/h1b/common.py
ADDED
|
@@ -0,0 +1,138 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Offline local-snapshot loading and production-tokenization reuse."""
|
| 2 |
+
import json
|
| 3 |
+
import hashlib
|
| 4 |
+
import copy
|
| 5 |
+
import subprocess
|
| 6 |
+
from pathlib import Path
|
| 7 |
+
import torch
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
def source_identity():
|
| 11 |
+
root=Path(__file__).resolve().parents[2]
|
| 12 |
+
commit=dirty=None
|
| 13 |
+
if (root/'.git').exists():
|
| 14 |
+
commit=subprocess.check_output(['git','-C',str(root),'rev-parse','HEAD'],text=True).strip()
|
| 15 |
+
dirty=bool(subprocess.check_output(['git','-C',str(root),'status','--porcelain'],text=True).strip())
|
| 16 |
+
manifest=root/'docs/H1B_SOURCE.json'
|
| 17 |
+
return dict(commit=commit,dirty=dirty,
|
| 18 |
+
source_manifest_sha256=hashlib.sha256(manifest.read_bytes()).hexdigest())
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
def snapshot_identity(path):
|
| 22 |
+
root=Path(path).resolve()
|
| 23 |
+
return dict(path=str(root),metadata_sha256={name:hashlib.sha256((root/name).read_bytes()).hexdigest()
|
| 24 |
+
for name in ('config.json','model.safetensors.index.json','tokenizer.json','tokenizer_config.json') if (root/name).is_file()},
|
| 25 |
+
qualification='Metadata identity; individual weight shards not hashed by this helper')
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
def archive_rows(path,limit=None):
|
| 29 |
+
with open(path) as f:
|
| 30 |
+
for i,line in enumerate(f):
|
| 31 |
+
if limit is not None and i>=limit: break
|
| 32 |
+
yield json.loads(line)
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
def load_pair(student_path,teacher_path,device):
|
| 36 |
+
from transformers import AutoModelForCausalLM,AutoTokenizer
|
| 37 |
+
if str(device).startswith('cuda'):
|
| 38 |
+
if torch.cuda.device_count()!=1 or 'B200' not in torch.cuda.get_device_name(0):
|
| 39 |
+
raise ValueError('Expose exactly one allocated B200 for H1b offline GPU tools')
|
| 40 |
+
models=[]; tokenizers=[]
|
| 41 |
+
for path in (student_path,teacher_path):
|
| 42 |
+
if not Path(path).is_dir(): raise FileNotFoundError('Require local model snapshot: '+str(path))
|
| 43 |
+
tokenizers.append(AutoTokenizer.from_pretrained(path,local_files_only=True))
|
| 44 |
+
models.append(AutoModelForCausalLM.from_pretrained(path,local_files_only=True,
|
| 45 |
+
torch_dtype=torch.bfloat16).to(device).eval())
|
| 46 |
+
config=getattr(models[-1].config,'text_config',None) or models[-1].config
|
| 47 |
+
if getattr(config,'final_logit_softcapping',None): raise ValueError('H1b softcapped heads unsupported')
|
| 48 |
+
return models,tokenizers
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def tokenize_record(record,student_tok,teacher_tok,assume_shared_prompt=False):
|
| 52 |
+
from kdflow.trainer.data_processor.rollout_data_processor import RolloutDataProcessor
|
| 53 |
+
# Reuse production method rather than approximating prompt boundary masks.
|
| 54 |
+
processor=object.__new__(RolloutDataProcessor)
|
| 55 |
+
if 'tea_prompt' not in record and not assume_shared_prompt:
|
| 56 |
+
raise ValueError('Archive lacks tea_prompt; supply enriched H1b archive or explicitly --assume-shared-prompt')
|
| 57 |
+
reason=record.get('finish_reason')
|
| 58 |
+
output=dict(text=record['output'],output_ids=record.get('output_ids',[]),finish_reason=reason)
|
| 59 |
+
if reason is None and not record.get('output_ids'):
|
| 60 |
+
raise ValueError('Cannot reconstruct truncation without finish_reason/output_ids')
|
| 61 |
+
append=not processor._is_length_truncated(output,{'max_new_tokens':8192})
|
| 62 |
+
out={}
|
| 63 |
+
for prefix,tok,prompt in (('stu',student_tok,record['prompt']),('tea',teacher_tok,record.get('tea_prompt',record['prompt']))):
|
| 64 |
+
fields=processor._tokenize_sample(prompt,record['output'],tok,prefix,append_eos=append)
|
| 65 |
+
out.update({k:v[None] for k,v in fields.items() if not k.startswith('_')})
|
| 66 |
+
return out
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
def row_groups(batch,student_tok,teacher_tok):
|
| 70 |
+
from kdflow.token_alignment import get_tokenizer_byte_codec
|
| 71 |
+
from kdflow.h1b.groups import align_byte_span_groups_batch
|
| 72 |
+
rows=[]
|
| 73 |
+
for key,tok in (('tea',teacher_tok),('stu',student_tok)):
|
| 74 |
+
labels=batch[key+'_input_ids'].roll(-1,1)
|
| 75 |
+
rows.append([tok.convert_ids_to_tokens(labels[i][batch[key+'_loss_mask'][i].bool()].tolist()) for i in range(len(labels))])
|
| 76 |
+
return align_byte_span_groups_batch(*rows,teacher_tok.eos_token,student_tok.eos_token,
|
| 77 |
+
get_tokenizer_byte_codec(teacher_tok).piece_to_bytes,get_tokenizer_byte_codec(student_tok).piece_to_bytes)
|
| 78 |
+
|
| 79 |
+
|
| 80 |
+
def forward_hidden(model,batch,prefix):
|
| 81 |
+
ids=batch[prefix+'_input_ids'].to(next(model.parameters()).device)
|
| 82 |
+
out=model.base_model(ids,attention_mask=batch[prefix+'_attn_mask'].to(ids.device),output_hidden_states=True,use_cache=False)
|
| 83 |
+
return out.hidden_states[-1][batch[prefix+'_loss_mask'].to(ids.device).bool()]
|
| 84 |
+
|
| 85 |
+
|
| 86 |
+
class StudentWrapper(torch.nn.Module):
|
| 87 |
+
def __init__(self,model):
|
| 88 |
+
super().__init__(); self.transformer=model
|
| 89 |
+
# Proxy contains the actual head; avoids assigning model.lm_head to itself.
|
| 90 |
+
import types
|
| 91 |
+
head=model.get_output_embeddings()
|
| 92 |
+
class Head(torch.nn.Module):
|
| 93 |
+
def __init__(self,head): super().__init__(); self.head=head
|
| 94 |
+
@property
|
| 95 |
+
def weight(self): return self.head.weight
|
| 96 |
+
def forward(self,x,skip=False): return x if skip else self.head(x)
|
| 97 |
+
self.model=types.SimpleNamespace(config=model.config,lm_head=Head(head))
|
| 98 |
+
def forward(self,ids,attention_mask=None,**kwargs):
|
| 99 |
+
output=self.transformer.base_model(ids,attention_mask=attention_mask,output_hidden_states=True,use_cache=False)
|
| 100 |
+
return {'hidden_states':output.hidden_states}
|
| 101 |
+
|
| 102 |
+
|
| 103 |
+
class HFPrefixOracle:
|
| 104 |
+
"""Branch-safe depth-bounded KV stack for depth-first cover enumeration.
|
| 105 |
+
|
| 106 |
+
HF DynamicCache may mutate in place. Each child receives its own copy; the
|
| 107 |
+
parent's cache remains reusable for siblings. Full-prefix fallback bounds
|
| 108 |
+
retained cache depth for unusually long chunks. No branch graph is retained.
|
| 109 |
+
"""
|
| 110 |
+
@torch.no_grad()
|
| 111 |
+
def __init__(self,model,prefix_ids,cache_depth=16,use_cache=True):
|
| 112 |
+
if not prefix_ids or cache_depth<1: raise ValueError('Nonempty prefix and positive cache depth required')
|
| 113 |
+
self.model=model; self.prefix=list(prefix_ids); self.depth=cache_depth; self.use_cache=use_cache
|
| 114 |
+
self.device=next(model.parameters()).device
|
| 115 |
+
out=model.base_model(torch.tensor([self.prefix],device=self.device),use_cache=use_cache)
|
| 116 |
+
lp=model.get_output_embeddings()(out.last_hidden_state[0,-1]).float().log_softmax(-1)
|
| 117 |
+
self.stack=[((),lp,getattr(out,'past_key_values',None))]
|
| 118 |
+
|
| 119 |
+
@torch.no_grad()
|
| 120 |
+
def __call__(self,branch):
|
| 121 |
+
branch=tuple(branch)
|
| 122 |
+
if not branch: return self.stack[0][1]
|
| 123 |
+
if not self.use_cache or len(branch)>self.depth or self.stack[0][2] is None:
|
| 124 |
+
out=self.model.base_model(torch.tensor([self.prefix+list(branch)],device=self.device),use_cache=False)
|
| 125 |
+
return self.model.get_output_embeddings()(out.last_hidden_state[0,-1]).float().log_softmax(-1)
|
| 126 |
+
common=0
|
| 127 |
+
for level in range(1,min(len(branch),len(self.stack)-1)+1):
|
| 128 |
+
if self.stack[level][0]!=branch[:level]: break
|
| 129 |
+
common=level
|
| 130 |
+
self.stack=self.stack[:common+1]
|
| 131 |
+
for level in range(common+1,len(branch)+1):
|
| 132 |
+
cache=copy.deepcopy(self.stack[-1][2])
|
| 133 |
+
out=self.model.base_model(torch.tensor([[branch[level-1]]],device=self.device),
|
| 134 |
+
attention_mask=torch.ones(1,len(self.prefix)+level,device=self.device,dtype=torch.long),
|
| 135 |
+
past_key_values=cache,use_cache=True)
|
| 136 |
+
lp=self.model.get_output_embeddings()(out.last_hidden_state[0,-1]).float().log_softmax(-1)
|
| 137 |
+
self.stack.append((branch[:level],lp,out.past_key_values))
|
| 138 |
+
return self.stack[-1][1]
|
tools/h1b/enrich_archive.py
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Recover teacher prompts in old rollout archives by exact dataset prompt lookup."""
|
| 2 |
+
import argparse
|
| 3 |
+
import hashlib
|
| 4 |
+
import json
|
| 5 |
+
from pathlib import Path
|
| 6 |
+
from .common import archive_rows,source_identity,snapshot_identity
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
def recover(records,dataset_rows,student_prompt,teacher_prompt):
|
| 10 |
+
mapping={}
|
| 11 |
+
for row in dataset_rows:
|
| 12 |
+
s,t=student_prompt(row),teacher_prompt(row)
|
| 13 |
+
if s in mapping and mapping[s]!=t: raise ValueError('Ambiguous teacher prompt for identical student prompt')
|
| 14 |
+
mapping[s]=t
|
| 15 |
+
output=[]
|
| 16 |
+
for record in records:
|
| 17 |
+
if record['prompt'] not in mapping: raise ValueError('Archived student prompt has no exact dataset/template match')
|
| 18 |
+
teacher=mapping[record['prompt']]
|
| 19 |
+
if 'tea_prompt' in record and record['tea_prompt']!=teacher:
|
| 20 |
+
raise ValueError('Existing archived teacher prompt differs from recovered prompt')
|
| 21 |
+
output.append(dict(record,tea_prompt=teacher))
|
| 22 |
+
return output
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
def main(argv=None):
|
| 26 |
+
p=argparse.ArgumentParser(description=__doc__)
|
| 27 |
+
for name in ('archive','dataset','student','teacher','output'): p.add_argument('--'+name,required=True)
|
| 28 |
+
p.add_argument('--input-key',default='prompt'); p.add_argument('--teacher-input-key',default='prompt')
|
| 29 |
+
args=p.parse_args(argv); target=Path(args.output)
|
| 30 |
+
if target.exists(): raise FileExistsError(target)
|
| 31 |
+
import pyarrow.parquet as pq
|
| 32 |
+
from transformers import AutoTokenizer
|
| 33 |
+
from kdflow.datasets.prompts_dataset import PromptDataset
|
| 34 |
+
toks=[AutoTokenizer.from_pretrained(p,local_files_only=True) for p in (args.student,args.teacher)]
|
| 35 |
+
builder=object.__new__(PromptDataset)
|
| 36 |
+
builder.apply_chat_template=True; builder.image_key=None; builder.enable_thinking=False
|
| 37 |
+
builder.input_template=None
|
| 38 |
+
rows=pq.read_table(args.dataset,columns=list(dict.fromkeys((args.input_key,args.teacher_input_key)))).to_pylist()
|
| 39 |
+
records=recover(list(archive_rows(args.archive)),rows,
|
| 40 |
+
lambda r:builder._build_prompt(r,toks[0],args.input_key),
|
| 41 |
+
lambda r:builder._build_prompt(r,toks[1],args.teacher_input_key))
|
| 42 |
+
target.parent.mkdir(parents=True,exist_ok=True)
|
| 43 |
+
with target.open('x') as f:
|
| 44 |
+
for record in records: f.write(json.dumps(record,ensure_ascii=False)+'\n')
|
| 45 |
+
receipt=dict(source=source_identity(),records=len(records),
|
| 46 |
+
archive_sha256=hashlib.sha256(Path(args.archive).read_bytes()).hexdigest(),
|
| 47 |
+
dataset_sha256=hashlib.sha256(Path(args.dataset).read_bytes()).hexdigest(),
|
| 48 |
+
student=snapshot_identity(args.student),teacher=snapshot_identity(args.teacher),
|
| 49 |
+
output_sha256=hashlib.sha256(target.read_bytes()).hexdigest(),
|
| 50 |
+
contract='Exact student prompt lookup, production PromptDataset template method; enable_thinking=False; no response regeneration or sampled-ID reconstruction')
|
| 51 |
+
Path(str(target)+'.receipt.json').write_text(json.dumps(receipt,indent=2))
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
if __name__=='__main__': main()
|
tools/h1b/exact_strprob.py
ADDED
|
@@ -0,0 +1,87 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Score an archived rollout; bound off-trunk mass without generating responses."""
|
| 2 |
+
import argparse
|
| 3 |
+
import hashlib
|
| 4 |
+
import json
|
| 5 |
+
import math
|
| 6 |
+
from pathlib import Path
|
| 7 |
+
import torch
|
| 8 |
+
from kdflow.h1b.vocab_index import VocabIndex
|
| 9 |
+
from kdflow.h1b.strprob import score_hidden_chunks
|
| 10 |
+
from kdflow.h1b.buckets import bucket_of
|
| 11 |
+
from kdflow.h1b.exact import exact_chunk_logprob
|
| 12 |
+
from .common import archive_rows,load_pair,tokenize_record,row_groups,forward_hidden,snapshot_identity,HFPrefixOracle,source_identity
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
def main(argv=None):
|
| 16 |
+
p=argparse.ArgumentParser(description=__doc__)
|
| 17 |
+
p.add_argument('--archive',required=True); p.add_argument('--student',required=True); p.add_argument('--teacher',required=True)
|
| 18 |
+
p.add_argument('--output',required=True); p.add_argument('--limit',type=int,default=200)
|
| 19 |
+
p.add_argument('--device',default='cuda'); p.add_argument('--tile-size',type=int,default=128)
|
| 20 |
+
p.add_argument('--prune',type=float,default=1e-6); p.add_argument('--max-nodes',type=int,default=10000)
|
| 21 |
+
p.add_argument('--trunk-only',action='store_true'); p.add_argument('--assume-shared-prompt',action='store_true')
|
| 22 |
+
p.add_argument('--full-prefix',action='store_true',help='Disable KV reuse for a correctness comparison')
|
| 23 |
+
p.add_argument('--cache-depth',type=int,default=16)
|
| 24 |
+
args=p.parse_args(argv)
|
| 25 |
+
if args.limit<1 or args.tile_size<1: p.error('limit and tile size must be positive')
|
| 26 |
+
target=Path(args.output)
|
| 27 |
+
if target.exists(): raise FileExistsError(target)
|
| 28 |
+
models,toks=load_pair(args.student,args.teacher,args.device)
|
| 29 |
+
indices=[VocabIndex(t) for t in toks]
|
| 30 |
+
summaries=[[] for _ in range(24)]; dropped=0
|
| 31 |
+
target.parent.mkdir(parents=True,exist_ok=True)
|
| 32 |
+
with target.open('x') as stream,torch.no_grad():
|
| 33 |
+
for ri,record in enumerate(archive_rows(args.archive,args.limit)):
|
| 34 |
+
batch=tokenize_record(record,*toks,args.assume_shared_prompt)
|
| 35 |
+
grouped=row_groups(batch,*toks)[0]
|
| 36 |
+
if not grouped.valid: dropped+=1; continue
|
| 37 |
+
groups=[g for g in grouped.groups if g.kind=='mis']
|
| 38 |
+
scored=[]
|
| 39 |
+
for model,tok,vi,prefix in zip(models,toks,indices,('stu','tea')):
|
| 40 |
+
hidden=forward_hidden(model,batch,prefix)
|
| 41 |
+
labels=batch[prefix+'_input_ids'].roll(-1,1)[batch[prefix+'_loss_mask'].bool()].to(hidden.device)
|
| 42 |
+
head=model.get_output_embeddings()
|
| 43 |
+
chunks=[((g.s_idx if prefix=='stu' else g.t_idx),g.content) for g in groups]
|
| 44 |
+
# A plain HF head has no skip keyword, so detached scoring uses teacher mode.
|
| 45 |
+
scores=score_hidden_chunks(head,hidden,labels,chunks,vi,args.tile_size,student=False)
|
| 46 |
+
results=[]
|
| 47 |
+
positions=batch[prefix+'_loss_mask'][0].nonzero().flatten().tolist()
|
| 48 |
+
for g,lp in zip(groups,scores):
|
| 49 |
+
data=dict(path=float(lp.log_q_path),trunk=float(lp.log_p_trunk),
|
| 50 |
+
overhang=float((lp.log_overhang-lp.log_p_trunk).exp()))
|
| 51 |
+
if not args.trunk_only:
|
| 52 |
+
flat=(g.s_idx if prefix=='stu' else g.t_idx)[0]
|
| 53 |
+
base=batch[prefix+'_input_ids'][0,:positions[flat]+1].tolist()
|
| 54 |
+
oracle=HFPrefixOracle(model,base,args.cache_depth,not args.full_prefix)
|
| 55 |
+
bounds=exact_chunk_logprob(oracle,g.content,vi,args.prune,args.max_nodes)
|
| 56 |
+
data.update(bounds)
|
| 57 |
+
data['error_upper']=max(0.,bounds['log_probability_upper']-data['trunk'])
|
| 58 |
+
data['error_lower']=max(0.,bounds['log_probability_lower']-data['trunk'])
|
| 59 |
+
results.append(data)
|
| 60 |
+
scored.append(results)
|
| 61 |
+
for g,s,t in zip(groups,*scored):
|
| 62 |
+
bucket=bucket_of(g.content)
|
| 63 |
+
row=dict(response_index=ri,response_hash=hashlib.sha256(record['output'].encode()).hexdigest(),
|
| 64 |
+
bucket=bucket,b0=g.b0,b1=g.b1,chunk_hex=g.content.hex(),student=s,teacher=t)
|
| 65 |
+
row['gap']=(s['path']-t['path'])-(s['trunk']-t['trunk'])
|
| 66 |
+
summaries[bucket].append(row)
|
| 67 |
+
# Omitted mass zero is -inf internally; JSON uses null for log(0).
|
| 68 |
+
def finite(obj):
|
| 69 |
+
if isinstance(obj,dict): return {k:finite(v) for k,v in obj.items()}
|
| 70 |
+
return None if isinstance(obj,float) and not math.isfinite(obj) else obj
|
| 71 |
+
stream.write(json.dumps(finite(row),allow_nan=False)+'\n')
|
| 72 |
+
buckets=[]
|
| 73 |
+
for b,rows in enumerate(summaries):
|
| 74 |
+
scale=sum(abs(r['gap']) for r in rows)/max(len(rows),1)
|
| 75 |
+
errors={m:sum(r[m].get('error_upper',0.) for r in rows)/max(len(rows),1) for m in ('student','teacher')}
|
| 76 |
+
status=('not_measured' if args.trunk_only else 'insufficient' if not rows else
|
| 77 |
+
'pass' if all(e<.1*scale for e in errors.values()) else 'fail')
|
| 78 |
+
buckets.append(dict(bucket=b,count=len(rows),mean_abs_gap=scale,error_upper_mean=errors,status=status))
|
| 79 |
+
summary=dict(source=source_identity(),archive_sha256=hashlib.sha256(Path(args.archive).read_bytes()).hexdigest(),
|
| 80 |
+
student=snapshot_identity(args.student),teacher=snapshot_identity(args.teacher),dropped_responses=dropped,buckets=buckets,
|
| 81 |
+
gate='not_measured' if args.trunk_only else 'pass' if all(b['status']=='pass' for b in buckets) else 'not_passed',
|
| 82 |
+
search_backend='full-prefix reference' if args.full_prefix else 'depth-bounded DFS KV stack',
|
| 83 |
+
qualification='Canonical-prefix conditional bounds; no real-model KV/B200 throughput qualification')
|
| 84 |
+
Path(str(target)+'.summary.json').write_text(json.dumps(summary,indent=2,allow_nan=False))
|
| 85 |
+
|
| 86 |
+
|
| 87 |
+
if __name__=='__main__': main()
|
tools/h1b/fit_cb.py
ADDED
|
@@ -0,0 +1,113 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""c(B), split-half, SNR and group bootstrap from recorded CountSketch groups."""
|
| 2 |
+
import argparse
|
| 3 |
+
import json
|
| 4 |
+
from pathlib import Path
|
| 5 |
+
import numpy as np
|
| 6 |
+
import torch
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
def cosine(a,b):
|
| 10 |
+
den=np.linalg.norm(a)*np.linalg.norm(b)
|
| 11 |
+
return float(np.dot(a,b)/den) if den else None
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
def fit_curve(batch_sizes,values):
|
| 15 |
+
valid=[(b,c) for b,c in zip(batch_sizes,values) if c is not None]
|
| 16 |
+
if len(valid)<2: return dict(c_infinity=None,b_noise=None,identified=False)
|
| 17 |
+
x,y=np.array(valid).T
|
| 18 |
+
grid=np.r_[0.,np.geomspace(1e-3,1e9,2048)]
|
| 19 |
+
basis=1/np.sqrt(1+grid[:,None]/x)
|
| 20 |
+
c=np.clip((basis*y).sum(1)/(basis*basis).sum(1),-1,1)
|
| 21 |
+
error=((basis*c[:,None]-y)**2).sum(1)
|
| 22 |
+
k=int(error.argmin())
|
| 23 |
+
return dict(c_infinity=float(c[k]),b_noise=float(grid[k]),identified=k not in (0,len(grid)-1),squared_error=float(error[k]))
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
def analyze_matrices(strict,span,control0,control1,group_size=64,seed=42,repeats=20,weights=None):
|
| 27 |
+
"""Small reference API, also used by tests. Undefined zero-vector cosines are null."""
|
| 28 |
+
rng=np.random.default_rng(seed); n=len(strict)
|
| 29 |
+
weights=np.ones(n) if weights is None else np.asarray(weights,dtype=float)
|
| 30 |
+
if np.any(weights<=0) or len(weights)!=n: raise ValueError('Positive group token weights required')
|
| 31 |
+
def pooled(matrix,ids): return np.average(matrix[ids],axis=0,weights=weights[ids])
|
| 32 |
+
sizes=[b for b in (64,128,256,512,1024,2048,4096) if b>=group_size and b%group_size==0 and b//group_size<=n]
|
| 33 |
+
rows=[]
|
| 34 |
+
for b in sizes:
|
| 35 |
+
m=b//group_size; cs=[]; ctrl=[]; halves=[]; ratios=[]; samples=[]
|
| 36 |
+
for _ in range(repeats):
|
| 37 |
+
ids=rng.choice(n,m,replace=False); s=pooled(strict,ids); v=pooled(span,ids)
|
| 38 |
+
cs.append(cosine(v,s)); ctrl.append(cosine(pooled(control0,ids),pooled(control1,ids)))
|
| 39 |
+
ratios.append(float(np.linalg.norm(v)/np.linalg.norm(s)) if np.linalg.norm(s) else None)
|
| 40 |
+
if m>=2:
|
| 41 |
+
h=m//2; halves.append(cosine(pooled(span,ids[:h]),pooled(span,ids[h:])))
|
| 42 |
+
samples.append(v)
|
| 43 |
+
sample=np.stack(samples[:16]); mu=sample.mean(0); noise=float(np.square(sample-mu).sum(1).mean())
|
| 44 |
+
if m==n: noise=0. # Resampling the entire fixed pool cannot identify sampling noise.
|
| 45 |
+
def avg(v):
|
| 46 |
+
v=[x for x in v if x is not None]
|
| 47 |
+
return float(np.mean(v)) if v else None
|
| 48 |
+
rows.append(dict(batch=b,cosine=avg(cs),control=avg(ctrl),split_half=avg(halves),norm_ratio=avg(ratios),
|
| 49 |
+
snr=float(np.dot(mu,mu)/noise) if noise else None,
|
| 50 |
+
snr_defined=bool(noise),split_half_responses=b//2 if m>=2 else None,
|
| 51 |
+
finite_population=True))
|
| 52 |
+
mean=np.average(span,axis=0,weights=weights); signal=float(np.dot(mean,mean)); noise=float(np.square(span-mean).sum(1).mean())
|
| 53 |
+
return dict(curve=rows,fit=fit_curve(sizes,[r['cosine'] for r in rows]),
|
| 54 |
+
group_noise_scale=group_size*noise/signal if signal else None)
|
| 55 |
+
|
| 56 |
+
|
| 57 |
+
def main(argv=None):
|
| 58 |
+
p=argparse.ArgumentParser(description=__doc__)
|
| 59 |
+
p.add_argument('--replay',required=True); p.add_argument('--output',required=True)
|
| 60 |
+
p.add_argument('--bootstrap',type=int,default=200); p.add_argument('--seed',type=int,default=42)
|
| 61 |
+
args=p.parse_args(argv)
|
| 62 |
+
root=Path(args.replay); meta=json.loads((root/'manifest.json').read_text())
|
| 63 |
+
if len(meta['records'])!=meta['groups']: raise ValueError('Incomplete replay manifest')
|
| 64 |
+
target=Path(args.output)
|
| 65 |
+
if target.exists(): raise FileExistsError(target)
|
| 66 |
+
def load(key):
|
| 67 |
+
return np.stack([torch.load(root/r['files'][key],map_location='cpu',weights_only=True).numpy() for r in meta['records']])
|
| 68 |
+
# Gram factorization reduces all subsequent resampling to <= 7*N dimensions;
|
| 69 |
+
# raw sketches are loaded one term at a time, not retained for bootstraps.
|
| 70 |
+
keys=list(meta['records'][0]['files'])
|
| 71 |
+
grams={}
|
| 72 |
+
for key in keys:
|
| 73 |
+
a=load(key)
|
| 74 |
+
for other in ('strict','strict_0','strict_1','path','str'):
|
| 75 |
+
b=a if key==other else load(other)
|
| 76 |
+
grams[(key,other)]=a@b.T
|
| 77 |
+
grams[(key,key)]=a@a.T
|
| 78 |
+
def factor(names):
|
| 79 |
+
block=np.block([[grams.get((a,b),grams.get((b,a),None).T if (b,a) in grams else None)
|
| 80 |
+
for b in names] for a in names])
|
| 81 |
+
val,vec=np.linalg.eigh((block+block.T)/2)
|
| 82 |
+
return np.split(vec*np.sqrt(np.maximum(val,0))[None,:],len(names))
|
| 83 |
+
variants={'path':('path',None),'str':('str',None),'path_bl':('path','base_path'),'str_bl':('str','base_str')}
|
| 84 |
+
for cls in meta.get('bucket_classes',[]):
|
| 85 |
+
variants.update({f'class{cls}_{k}':(f'class{cls}_{v[0]}',f'class{cls}_{v[1]}' if v[1] else None) for k,v in list(variants.items()) if not k.startswith('class')})
|
| 86 |
+
results={}; rng=np.random.default_rng(args.seed)
|
| 87 |
+
# Cross Grams involving baseline buffers are required for subtraction; compute
|
| 88 |
+
# only the pairs needed for each analysis, retaining small matrices.
|
| 89 |
+
for variant,(key,base) in variants.items():
|
| 90 |
+
names=['strict',key,'strict_0','strict_1']+([base] if base else [])
|
| 91 |
+
for a in names:
|
| 92 |
+
for b in names:
|
| 93 |
+
if (a,b) not in grams and (b,a) not in grams: grams[(a,b)]=load(a)@load(b).T
|
| 94 |
+
mat=factor(names); strict,span,c0,c1=mat[:4]
|
| 95 |
+
if base: span=span-mat[4]
|
| 96 |
+
weights=np.array([r['tokens'] for r in meta['records']])
|
| 97 |
+
result=analyze_matrices(strict,span,c0,c1,meta['group_size'],args.seed,weights=weights)
|
| 98 |
+
bootstrap=[]
|
| 99 |
+
for _ in range(args.bootstrap):
|
| 100 |
+
ids=rng.integers(0,len(strict),len(strict))
|
| 101 |
+
bootstrap.append(analyze_matrices(strict[ids],span[ids],c0[ids],c1[ids],meta['group_size'],int(rng.integers(2**31)),weights=weights[ids])['fit'])
|
| 102 |
+
result['bootstrap_intervals']={k:(np.quantile([b[k] for b in bootstrap if b[k] is not None],[.025,.975]).tolist()
|
| 103 |
+
if any(b[k] is not None for b in bootstrap) else None) for k in ('c_infinity','b_noise')}
|
| 104 |
+
results[variant]=result
|
| 105 |
+
names=['path','str']; a,b=factor(names)
|
| 106 |
+
output=dict(results=results,cos_mean_path_str=cosine(np.average(a,axis=0,weights=weights),np.average(b,axis=0,weights=weights)),
|
| 107 |
+
qualification='CountSketch estimates; group-bootstrap intervals; finite-pool SNR; undefined diagnostics are null',
|
| 108 |
+
strict_control='disjoint alternating strict positions within each microbatch',
|
| 109 |
+
missing='Adam-preconditioned analysis, fresh on-policy generation, end-task statistics are separate operations')
|
| 110 |
+
target.parent.mkdir(parents=True,exist_ok=True); target.write_text(json.dumps(output,indent=2,allow_nan=False))
|
| 111 |
+
|
| 112 |
+
|
| 113 |
+
if __name__=='__main__': main()
|
tools/h1b/grad_replay.py
ADDED
|
@@ -0,0 +1,129 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Replay fixed archived responses through H1b; never generate new responses."""
|
| 2 |
+
import argparse
|
| 3 |
+
import hashlib
|
| 4 |
+
import json
|
| 5 |
+
from pathlib import Path
|
| 6 |
+
import types
|
| 7 |
+
import torch
|
| 8 |
+
from kdflow.algorithms.h1b_span import H1bSpanCTKD
|
| 9 |
+
from kdflow.arguments.distillation_args import DistillationArguments
|
| 10 |
+
from .common import archive_rows,load_pair,tokenize_record,forward_hidden,StudentWrapper,snapshot_identity,source_identity
|
| 11 |
+
from .sketch import count_sketch
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
def collate(batches):
|
| 15 |
+
from torch.nn.utils.rnn import pad_sequence
|
| 16 |
+
return {k:pad_sequence([b[k][0] for b in batches],batch_first=True) for k in batches[0]}
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
def main(argv=None):
|
| 20 |
+
p=argparse.ArgumentParser(description=__doc__)
|
| 21 |
+
for name in ('archive','student','teacher','output'): p.add_argument('--'+name,required=True)
|
| 22 |
+
p.add_argument('--limit',type=int,default=4096); p.add_argument('--group-size',type=int,default=64)
|
| 23 |
+
p.add_argument('--micro-batch',type=int,default=4); p.add_argument('--tile-size',type=int,default=128)
|
| 24 |
+
p.add_argument('--dimension',type=int,default=2**22); p.add_argument('--seed',type=int,default=42)
|
| 25 |
+
p.add_argument('--device',default='cuda'); p.add_argument('--buffer-device',default='cuda')
|
| 26 |
+
p.add_argument('--exact-groups',type=int,default=2)
|
| 27 |
+
p.add_argument('--buckets',default='',help='Optional byte class IDs 0=digits,3=letters; adds gradient buffers')
|
| 28 |
+
p.add_argument('--assume-shared-prompt',action='store_true')
|
| 29 |
+
args=p.parse_args(argv)
|
| 30 |
+
if min(args.limit,args.group_size,args.micro_batch,args.tile_size,args.dimension)<1:
|
| 31 |
+
p.error('Sizes must be positive')
|
| 32 |
+
if args.limit%args.group_size or args.group_size%args.micro_batch:
|
| 33 |
+
p.error('Require complete equal groups and microbatches')
|
| 34 |
+
root=Path(args.output)
|
| 35 |
+
if root.exists(): raise FileExistsError(root)
|
| 36 |
+
records=list(archive_rows(args.archive,args.limit))
|
| 37 |
+
if len(records)!=args.limit: raise ValueError('Archive has fewer responses than requested')
|
| 38 |
+
selected=[int(s) for s in args.buckets.split(',') if s.strip()]
|
| 39 |
+
if any(not 0<=s<=5 for s in selected): raise ValueError('Byte class must be in 0..5')
|
| 40 |
+
models,toks=load_pair(args.student,args.teacher,args.device)
|
| 41 |
+
student,teacher=models
|
| 42 |
+
teacher.requires_grad_(False)
|
| 43 |
+
wrapper=StudentWrapper(student)
|
| 44 |
+
strategy=types.SimpleNamespace(args=types.SimpleNamespace(
|
| 45 |
+
kd=DistillationArguments(kd_algorithm='h1b_span',kd_ratio=1,kd_temperature=1,kd_loss_fn='rkl'),
|
| 46 |
+
model=types.SimpleNamespace(ring_attn_size=1),
|
| 47 |
+
train=types.SimpleNamespace(chunked_loss_size=args.tile_size,num_nodes=1,num_gpus_per_node=1,save_path=None),
|
| 48 |
+
scenario='on_policy_kd'),ring_attn_group=None)
|
| 49 |
+
algo=H1bSpanCTKD(strategy,wrapper,teacher.get_output_embeddings(),*toks)
|
| 50 |
+
algo.replay_enabled=True
|
| 51 |
+
algo.replay_control=False
|
| 52 |
+
wrapper.eval()
|
| 53 |
+
config=student.config
|
| 54 |
+
if any(getattr(config,k,0) for k in ('attention_dropout','hidden_dropout','hidden_dropout_prob')):
|
| 55 |
+
raise ValueError('Deterministic checkpointed replay requires zero model dropout')
|
| 56 |
+
student.gradient_checkpointing_enable(gradient_checkpointing_kwargs={'use_reentrant':False})
|
| 57 |
+
student.base_model.train() # Enables checkpointing; no dropout for qualified pair.
|
| 58 |
+
encoded=[tokenize_record(r,*toks,args.assume_shared_prompt) for r in records]
|
| 59 |
+
ngroups=args.limit//args.group_size
|
| 60 |
+
sums=torch.zeros(ngroups,2,24,dtype=torch.float64); counts=torch.zeros_like(sums)
|
| 61 |
+
def batch_at(start,denominator):
|
| 62 |
+
batch=collate(encoded[start:start+args.micro_batch])
|
| 63 |
+
with torch.no_grad(): batch['teacher_hiddens']=forward_hidden(teacher,batch,'tea')
|
| 64 |
+
for k,v in batch.items(): batch[k]=v.to(args.device)
|
| 65 |
+
batch['avg_micro_batch_token_num']=denominator
|
| 66 |
+
return batch
|
| 67 |
+
# First pass obtains all group bucket means; replay never uses its own group
|
| 68 |
+
# to construct the leave-group-out baseline.
|
| 69 |
+
with torch.no_grad():
|
| 70 |
+
for i in range(0,args.limit,args.micro_batch):
|
| 71 |
+
algo.training_step(batch_at(i,1.))
|
| 72 |
+
gi=i//args.group_size
|
| 73 |
+
for record in algo.replay_records:
|
| 74 |
+
b=record['bucket']
|
| 75 |
+
for kind,key in enumerate(('path','str')):
|
| 76 |
+
sums[gi,kind,b]+=float(record[key]); counts[gi,kind,b]+=1
|
| 77 |
+
if ngroups<2: raise ValueError('Leave-group-out replay needs at least two groups')
|
| 78 |
+
algo.replay_control=True
|
| 79 |
+
root.mkdir(parents=True,exist_ok=False)
|
| 80 |
+
named=list(wrapper.named_parameters()); params=tuple(v for _,v in named)
|
| 81 |
+
offsets=[0]
|
| 82 |
+
for _,param in named: offsets.append(offsets[-1]+param.numel())
|
| 83 |
+
keys=['strict','path','str','base_path','base_str','strict_0','strict_1']
|
| 84 |
+
for b in selected:
|
| 85 |
+
keys.extend(f'class{b}_{term}' for term in ('path','str','base_path','base_str'))
|
| 86 |
+
manifest=dict(version=1,source=source_identity(),archive_sha256=hashlib.sha256(Path(args.archive).read_bytes()).hexdigest(),
|
| 87 |
+
student=snapshot_identity(args.student),teacher=snapshot_identity(args.teacher),group_size=args.group_size,groups=ngroups,
|
| 88 |
+
dimension=args.dimension,seed=args.seed,normalization='sum token-normalized microbatch gradients per group',
|
| 89 |
+
baseline='leave-group-out bucket means; outcome-dependent bucket caveat applies',
|
| 90 |
+
preconditioner='raw gradients only',bucket_classes=selected,records=[])
|
| 91 |
+
for gi in range(ngroups):
|
| 92 |
+
other_s=sums.sum(0)-sums[gi]; other_n=counts.sum(0)-counts[gi]
|
| 93 |
+
algo.baseline.prev=torch.where(other_n>0,other_s/other_n.clamp_min(1),torch.zeros_like(other_s))
|
| 94 |
+
# Deliberately explicit memory contract: seven full FP32 buffers by
|
| 95 |
+
# default. Probe with --limit 128 before the 4096-response replay.
|
| 96 |
+
buffers={k:torch.zeros(offsets[-1],dtype=torch.float32,device=args.buffer_device) for k in keys}
|
| 97 |
+
lo=gi*args.group_size
|
| 98 |
+
tokens=sum(int(b['stu_loss_mask'].sum()) for b in encoded[lo:lo+args.group_size])
|
| 99 |
+
for i in range(lo,lo+args.group_size,args.micro_batch):
|
| 100 |
+
algo.training_step(batch_at(i,max(tokens,1)))
|
| 101 |
+
terms=dict(algo.replay_terms)
|
| 102 |
+
for b in selected:
|
| 103 |
+
chunk_records=[r for r in algo.replay_records if r['bucket']//4==b]
|
| 104 |
+
for term in ('path','str','base_path','base_str'):
|
| 105 |
+
zero=terms['strict']*0
|
| 106 |
+
key=term.removeprefix('base_')
|
| 107 |
+
terms[f'class{b}_{term}']=sum((2*algo.lam*(algo.baseline.lookup(r['bucket'],key)
|
| 108 |
+
if term.startswith('base_') else r[term])*r['log_q']/max(tokens,1) for r in chunk_records),zero)
|
| 109 |
+
for ki,key in enumerate(keys):
|
| 110 |
+
grads=torch.autograd.grad(terms[key],params,retain_graph=ki<len(keys)-1,allow_unused=True)
|
| 111 |
+
for pi,grad in enumerate(grads):
|
| 112 |
+
if grad is not None: buffers[key][offsets[pi]:offsets[pi+1]].add_(grad.detach().flatten().to(args.buffer_device))
|
| 113 |
+
files={}; norms={}
|
| 114 |
+
for key,flat in buffers.items():
|
| 115 |
+
tensors=((name,flat[offsets[j]:offsets[j+1]]) for j,(name,_) in enumerate(named))
|
| 116 |
+
sketch,norm=count_sketch(tensors,args.dimension,args.seed)
|
| 117 |
+
file=f'group_{gi:03d}_{key}.pt'
|
| 118 |
+
torch.save(sketch,root/file); files[key]=file; norms[key]=norm
|
| 119 |
+
if gi<args.exact_groups:
|
| 120 |
+
# Save one buffer at a time; the validator memory-maps these.
|
| 121 |
+
torch.save(flat.cpu(),root/f'group_{gi:03d}_{key}_exact.pt')
|
| 122 |
+
manifest['records'].append(dict(group=gi,tokens=tokens,files=files,exact_norms=norms,
|
| 123 |
+
bucket_sums=sums[gi].tolist(),bucket_counts=counts[gi].tolist()))
|
| 124 |
+
(root/'manifest.json').write_text(json.dumps(manifest,indent=2))
|
| 125 |
+
del buffers
|
| 126 |
+
print(json.dumps(dict(groups=ngroups,output=str(root),qualification='raw fixed-text replay; GPU execution must be independently qualified')))
|
| 127 |
+
|
| 128 |
+
|
| 129 |
+
if __name__=='__main__': main()
|
tools/h1b/sketch.py
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Streaming CountSketch keyed by tensor name, independent of chunk boundaries."""
|
| 2 |
+
import hashlib
|
| 3 |
+
import torch
|
| 4 |
+
|
| 5 |
+
|
| 6 |
+
def count_sketch(named_tensors,dimension=2**22,seed=42,chunk_size=2**20):
|
| 7 |
+
if dimension<1 or chunk_size<1: raise ValueError('Positive sketch dimensions required')
|
| 8 |
+
output=torch.zeros(dimension,dtype=torch.float32)
|
| 9 |
+
norm2=0.
|
| 10 |
+
prime=2147483647
|
| 11 |
+
for name,tensor in named_tensors:
|
| 12 |
+
flat=tensor.detach().reshape(-1)
|
| 13 |
+
digest=hashlib.sha256((str(seed)+':'+name).encode()).digest()
|
| 14 |
+
a,b,c,d=[int.from_bytes(digest[i:i+8],'little')%(prime-1)+1 for i in (0,8,16,24)]
|
| 15 |
+
for start in range(0,flat.numel(),chunk_size):
|
| 16 |
+
values=flat[start:start+chunk_size].float().cpu()
|
| 17 |
+
indices=torch.arange(start,start+len(values),dtype=torch.int64)
|
| 18 |
+
bucket=((a*indices+b)%prime)%dimension
|
| 19 |
+
signs=(((c*indices+d)%prime)%2)*2-1
|
| 20 |
+
output.scatter_add_(0,bucket,values*signs)
|
| 21 |
+
norm2+=float(values.double().square().sum())
|
| 22 |
+
return output,norm2**.5
|
tools/h1b/validate_alignment.py
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Real-tokenizer T1/T3/T9 gates; requires local tokenizer snapshots and archived text."""
|
| 2 |
+
import argparse
|
| 3 |
+
import hashlib
|
| 4 |
+
import json
|
| 5 |
+
from pathlib import Path
|
| 6 |
+
from kdflow.token_alignment import get_tokenizer_byte_codec,align_byte_span_batch_1to1
|
| 7 |
+
from kdflow.algorithms._mp_opd_port._mp_opd_atoms import SimCTAtomizer
|
| 8 |
+
from .common import archive_rows,tokenize_record,row_groups
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
def main(argv=None):
|
| 12 |
+
p=argparse.ArgumentParser(description=__doc__)
|
| 13 |
+
for key in ('student','teacher','archive','texts','output'): p.add_argument('--'+key,required=True)
|
| 14 |
+
p.add_argument('--assume-shared-prompt',action='store_true')
|
| 15 |
+
args=p.parse_args(argv)
|
| 16 |
+
target=Path(args.output)
|
| 17 |
+
if target.exists(): raise FileExistsError(target)
|
| 18 |
+
from transformers import AutoTokenizer
|
| 19 |
+
toks=[AutoTokenizer.from_pretrained(s,local_files_only=True) for s in (args.student,args.teacher)]
|
| 20 |
+
codecs=[get_tokenizer_byte_codec(t) for t in toks]
|
| 21 |
+
texts=Path(args.texts).read_text().splitlines()
|
| 22 |
+
if len(texts)<10000: raise ValueError('T1 requires at least 10,000 texts')
|
| 23 |
+
# User-supplied corpus must include challenging scripts, not just 10k copies
|
| 24 |
+
# of one ASCII fixture. This list is supplemental and reported separately.
|
| 25 |
+
probes=['Tiếng Việt: một phép thử','中文 日本語 한국어','🙂🚀','\\frac{120}{3} = 40','def f(x):\n return x+1','a\t b\n']
|
| 26 |
+
failures=[]
|
| 27 |
+
for model,tok,codec in zip(('student','teacher'),toks,codecs):
|
| 28 |
+
for i,text in enumerate(texts+probes):
|
| 29 |
+
ids=tok(text,add_special_tokens=False)['input_ids']
|
| 30 |
+
raw=b''.join(codec.piece_to_bytes(p) for p in tok.convert_ids_to_tokens(ids))
|
| 31 |
+
if raw!=text.encode(): failures.append(dict(test='T1',model=model,index=i))
|
| 32 |
+
records=list(archive_rows(args.archive,500))
|
| 33 |
+
if len(records)!=500: raise ValueError('T3/T9 require 500 real responses')
|
| 34 |
+
atomizer=SimCTAtomizer(toks[0],toks[1])
|
| 35 |
+
common=0; dropped=0
|
| 36 |
+
for ri,record in enumerate(records):
|
| 37 |
+
batch=tokenize_record(record,*toks,args.assume_shared_prompt)
|
| 38 |
+
grouped=row_groups(batch,*toks)[0]
|
| 39 |
+
sid=batch['stu_input_ids'].roll(-1,1)[batch['stu_loss_mask'].bool()].tolist()
|
| 40 |
+
tid=batch['tea_input_ids'].roll(-1,1)[batch['tea_loss_mask'].bool()].tolist()
|
| 41 |
+
st,tt=toks[0].convert_ids_to_tokens(sid),toks[1].convert_ids_to_tokens(tid)
|
| 42 |
+
strict=align_byte_span_batch_1to1([tt],[st],toks[1].eos_token,toks[0].eos_token,
|
| 43 |
+
codecs[1].piece_to_bytes,codecs[0].piece_to_bytes)
|
| 44 |
+
actual=([g.t_idx[0] for g in grouped.groups if g.kind=='strict'],[g.s_idx[0] for g in grouped.groups if g.kind=='strict'])
|
| 45 |
+
if actual!=strict:
|
| 46 |
+
failures.append(dict(test='T3',response=ri,reason=grouped.reason or 'strict_set_difference',author_indices=strict,h1b_indices=actual))
|
| 47 |
+
atomized=atomizer.atomize(sid,tid,sample_id=str(ri))
|
| 48 |
+
if not grouped.valid or not atomized.valid: dropped+=1; continue
|
| 49 |
+
common+=1
|
| 50 |
+
# EOS is a Strict-only synchronization event; compare content spans.
|
| 51 |
+
pairs=[(g.s_idx[0],g.s_idx[-1]+1,g.t_idx[0],g.t_idx[-1]+1,g.kind) for g in grouped.groups if not g.eos]
|
| 52 |
+
reference=[(a.student_start,a.student_end,a.teacher_start,a.teacher_end,
|
| 53 |
+
'strict' if a.student_end-a.student_start==a.teacher_end-a.teacher_start==1 else 'mis') for a in atomized.atoms]
|
| 54 |
+
if pairs!=reference: failures.append(dict(test='T9',response=ri,h1b=pairs,mp_opd=reference))
|
| 55 |
+
out=dict(text_count=len(texts),supplemental_probes=len(probes),responses=500,common_accepted=common,
|
| 56 |
+
excluded_from_common=dropped,disagreements=failures,passed=not failures,
|
| 57 |
+
archive_sha256=hashlib.sha256(Path(args.archive).read_bytes()).hexdigest(),
|
| 58 |
+
qualification='Real-input alignment gate only; no GPU training')
|
| 59 |
+
target.parent.mkdir(parents=True,exist_ok=True); target.write_text(json.dumps(out,indent=2))
|
| 60 |
+
if failures: raise SystemExit('H1b alignment gate has disagreements; see report')
|
| 61 |
+
|
| 62 |
+
|
| 63 |
+
if __name__=='__main__': main()
|
tools/h1b/validate_sketch.py
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Validate saved group 0/1 sketches against memory-mapped exact gradients (T11)."""
|
| 2 |
+
import argparse
|
| 3 |
+
import json
|
| 4 |
+
from pathlib import Path
|
| 5 |
+
import math
|
| 6 |
+
import torch
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
def compare(root,manifest,chunk_size=2**20):
|
| 10 |
+
results={}; passed=True
|
| 11 |
+
for key in manifest['records'][0]['files']:
|
| 12 |
+
exact=[torch.load(root/f'group_{i:03d}_{key}_exact.pt',map_location='cpu',weights_only=True,mmap=True) for i in (0,1)]
|
| 13 |
+
sketch=[torch.load(root/manifest['records'][i]['files'][key],map_location='cpu',weights_only=True) for i in (0,1)]
|
| 14 |
+
aa=bb=ab=0.
|
| 15 |
+
if exact[0].shape!=exact[1].shape: raise ValueError('Exact gradient shapes differ')
|
| 16 |
+
for start in range(0,exact[0].numel(),chunk_size):
|
| 17 |
+
a,b=(x[start:start+chunk_size].double() for x in exact)
|
| 18 |
+
aa+=float(a.square().sum()); bb+=float(b.square().sum()); ab+=float((a*b).sum())
|
| 19 |
+
norms=[math.sqrt(aa),math.sqrt(bb)]
|
| 20 |
+
sn=[float(x.double().norm()) for x in sketch]
|
| 21 |
+
errors=[abs(s/n-1) if n else 0. if s==0 else math.inf for s,n in zip(sn,norms)]
|
| 22 |
+
ec=ab/math.sqrt(aa*bb) if aa*bb else None
|
| 23 |
+
sc=float(torch.nn.functional.cosine_similarity(sketch[0].double(),sketch[1].double(),dim=0)) if sn[0]*sn[1] else None
|
| 24 |
+
ce=abs(ec-sc) if ec is not None and sc is not None else None
|
| 25 |
+
ok=all(e<.01 for e in errors) and (ce is None or ce<.01)
|
| 26 |
+
passed=passed and ok
|
| 27 |
+
results[key]=dict(norm_relative_errors=errors,exact_cosine=ec,sketch_cosine=sc,cosine_abs_error=ce,passed=ok)
|
| 28 |
+
return dict(passed=passed,terms=results,zero_vectors='cosine undefined; reported null')
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
def main(argv=None):
|
| 32 |
+
p=argparse.ArgumentParser(description=__doc__); p.add_argument('--replay',required=True); p.add_argument('--output',required=True)
|
| 33 |
+
args=p.parse_args(argv); root=Path(args.replay); target=Path(args.output)
|
| 34 |
+
if target.exists(): raise FileExistsError(target)
|
| 35 |
+
meta=json.loads((root/'manifest.json').read_text())
|
| 36 |
+
result=compare(root,meta); target.write_text(json.dumps(result,indent=2,allow_nan=False))
|
| 37 |
+
if not result['passed']: raise SystemExit('Sketch accuracy gate failed')
|
| 38 |
+
|
| 39 |
+
|
| 40 |
+
if __name__=='__main__': main()
|
tools/launch_single_b200.py
CHANGED
|
@@ -17,6 +17,7 @@ import subprocess
|
|
| 17 |
|
| 18 |
SOURCE = pathlib.Path(__file__).resolve().parents[1]
|
| 19 |
ARCHIVE_SHA = '084c321c0b7af1cd3a36d12a595c623567d4a37525aa0eecbc3933aa7f9b15c3'
|
|
|
|
| 20 |
|
| 21 |
|
| 22 |
def parser():
|
|
@@ -28,7 +29,7 @@ def parser():
|
|
| 28 |
p.add_argument('--algorithm', choices=['strict-top16', 'strict-full', 'simct', 'simct_paper', 'mp-atomic',
|
| 29 |
'mp-fixed1', 'mp-fixed2', 'mp-fixed3', 'mp-fixed4', 'mp-fixed5',
|
| 30 |
'mp-random1-2', 'mp-random2-5', 'mp-grass-chunk', 'mp-align', 'mp-align-new',
|
| 31 |
-
'mp-trust-r','mp-trust-b','mp-trust-p'], default='strict-top16')
|
| 32 |
p.add_argument('--mp-chunk-source', choices=['xtoken', 'run'], default='xtoken')
|
| 33 |
p.add_argument('--mp-chunk-run-length', type=int, default=2)
|
| 34 |
p.add_argument('--mp-geometry', choices=['exact_head', 'diag'], default='exact_head')
|
|
@@ -103,6 +104,9 @@ def build_plan(args):
|
|
| 103 |
flags.update(mp_opd_align_new_min_descent_fraction=0.1, mp_opd_align_new_eps_g=1e-12)
|
| 104 |
if family.startswith('trust_'):
|
| 105 |
flags.update(mp_opd_grass_geometry='exact_head',mp_opd_trust_eps_g=1e-12,mp_opd_trust_vocab_chunk=4096)
|
|
|
|
|
|
|
|
|
|
| 106 |
command = ['bash', str(runtime / 'python-b200.sh'), '-u', '-m', 'kdflow.cli.train_kd_on_policy']
|
| 107 |
for key, value in flags.items():
|
| 108 |
command.extend(['--' + key, str(value)])
|
|
@@ -110,7 +114,9 @@ def build_plan(args):
|
|
| 110 |
'source': str(SOURCE), 'flags': flags, 'command': command,
|
| 111 |
'gradient_accumulation_microsteps': 512 // args.micro_batch,
|
| 112 |
'expected_runtime_archive_sha256': ARCHIVE_SHA,
|
| 113 |
-
'qualification': ('
|
|
|
|
|
|
|
| 114 |
if args.algorithm == 'simct_paper' else
|
| 115 |
'author-code SimCT port; no B200/full-training qualification'
|
| 116 |
if args.algorithm == 'simct' else
|
|
|
|
| 17 |
|
| 18 |
SOURCE = pathlib.Path(__file__).resolve().parents[1]
|
| 19 |
ARCHIVE_SHA = '084c321c0b7af1cd3a36d12a595c623567d4a37525aa0eecbc3933aa7f9b15c3'
|
| 20 |
+
H1B_ARMS={'h1b-a':'a','h1b-b':'b','h1b-b-bl':'b_bl','h1b-c-str':'c_str','h1b-c':'c'}
|
| 21 |
|
| 22 |
|
| 23 |
def parser():
|
|
|
|
| 29 |
p.add_argument('--algorithm', choices=['strict-top16', 'strict-full', 'simct', 'simct_paper', 'mp-atomic',
|
| 30 |
'mp-fixed1', 'mp-fixed2', 'mp-fixed3', 'mp-fixed4', 'mp-fixed5',
|
| 31 |
'mp-random1-2', 'mp-random2-5', 'mp-grass-chunk', 'mp-align', 'mp-align-new',
|
| 32 |
+
'mp-trust-r','mp-trust-b','mp-trust-p',*H1B_ARMS], default='strict-top16')
|
| 33 |
p.add_argument('--mp-chunk-source', choices=['xtoken', 'run'], default='xtoken')
|
| 34 |
p.add_argument('--mp-chunk-run-length', type=int, default=2)
|
| 35 |
p.add_argument('--mp-geometry', choices=['exact_head', 'diag'], default='exact_head')
|
|
|
|
| 104 |
flags.update(mp_opd_align_new_min_descent_fraction=0.1, mp_opd_align_new_eps_g=1e-12)
|
| 105 |
if family.startswith('trust_'):
|
| 106 |
flags.update(mp_opd_grass_geometry='exact_head',mp_opd_trust_eps_g=1e-12,mp_opd_trust_vocab_chunk=4096)
|
| 107 |
+
if args.algorithm in H1B_ARMS:
|
| 108 |
+
flags.update(kd_algorithm='h1b_span',h1b_arm=H1B_ARMS[args.algorithm],h1b_span_lambda=.5,
|
| 109 |
+
h1b_log_sample_rate=.05,h1b_dump_every='0,20,60,100',top_p=1.,eval_math_top_p=.95)
|
| 110 |
command = ['bash', str(runtime / 'python-b200.sh'), '-u', '-m', 'kdflow.cli.train_kd_on_policy']
|
| 111 |
for key, value in flags.items():
|
| 112 |
command.extend(['--' + key, str(value)])
|
|
|
|
| 114 |
'source': str(SOURCE), 'flags': flags, 'command': command,
|
| 115 |
'gradient_accumulation_microsteps': 512 // args.micro_batch,
|
| 116 |
'expected_runtime_archive_sha256': ARCHIVE_SHA,
|
| 117 |
+
'qualification': ('experimental H1b span arms; trunk gate pending; no B200/full-training qualification'
|
| 118 |
+
if args.algorithm in H1B_ARMS else
|
| 119 |
+
'paper-score SimCT reconstruction; no B200/full-training qualification'
|
| 120 |
if args.algorithm == 'simct_paper' else
|
| 121 |
'author-code SimCT port; no B200/full-training qualification'
|
| 122 |
if args.algorithm == 'simct' else
|