feat(align): port isolated ALIGN-NEW with donor gradient parity
Browse files- README.md +3 -0
- docs/ALIGN_NEW.md +88 -0
- docs/ALIGN_NEW_SOURCE.json +71 -0
- docs/ALIGN_NEW_VALIDATION.json +37 -0
- docs/MP_OPD.md +4 -1
- docs/SINGLE_B200.md +2 -0
- kdflow/algorithms/__init__.py +1 -0
- kdflow/algorithms/_mp_opd_port/_mp_opd_align_new.py +58 -0
- kdflow/algorithms/_mp_opd_port/_mp_opd_head_pair.py +110 -0
- kdflow/algorithms/mp_opd_align_new.py +53 -0
- kdflow/arguments/distillation_args.py +9 -0
- kdflow/mp_opd_modes.py +1 -1
- tests/fixtures/align_new/_align_loss.py.txt +114 -0
- tests/fixtures/align_new/_grass_chunk_assignment.py.txt +19 -0
- tests/fixtures/align_new/_mp_opd_align_new.py.txt +58 -0
- tests/fixtures/align_new/_mp_opd_head_pair.py.txt +110 -0
- tests/test_align_new.py +245 -0
- tests/test_single_b200.py +12 -1
- tools/launch_single_b200.py +6 -2
README.md
CHANGED
|
@@ -17,6 +17,9 @@ GRASS-Chunk and ALIGN as separate modes. The existing Strict algorithm sources
|
|
| 17 |
remain unchanged. The new modes use sampled atom credits rather than Strict KL;
|
| 18 |
they have CPU verification, with B200/full training qualification still pending.
|
| 19 |
|
|
|
|
|
|
|
|
|
|
| 20 |
A separate [author-code SimCT baseline](docs/SIMCT.md) uses `--algorithm simct`
|
| 21 |
in the same launcher. It preserves the pinned released implementation, with
|
| 22 |
original-source loss/gradient parity tests and an explicit paper/code distinction.
|
|
|
|
| 17 |
remain unchanged. The new modes use sampled atom credits rather than Strict KL;
|
| 18 |
they have CPU verification, with B200/full training qualification still pending.
|
| 19 |
|
| 20 |
+
The separate [ALIGN-NEW port](docs/ALIGN_NEW.md), `--algorithm mp-align-new`, adds
|
| 21 |
+
the donor's full-response head-descent guard while keeping `mp-align` v1 unchanged.
|
| 22 |
+
|
| 23 |
A separate [author-code SimCT baseline](docs/SIMCT.md) uses `--algorithm simct`
|
| 24 |
in the same launcher. It preserves the pinned released implementation, with
|
| 25 |
original-source loss/gradient parity tests and an explicit paper/code distinction.
|
docs/ALIGN_NEW.md
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ALIGN-NEW from SimCT
|
| 2 |
+
|
| 3 |
+
The SimCT inventory at `529ee4b691732e1040d6a45f05fd12f58def96ec` contains
|
| 4 |
+
`align` (v1) and `align_new`. No implementations explicitly named ALIGN v2/v3
|
| 5 |
+
were found in its current branches, algorithm/docs files or relevant commit
|
| 6 |
+
messages; the user confirmed no additional version pointer. This port preserves
|
| 7 |
+
the real donor name, without assigning a new version number.
|
| 8 |
+
|
| 9 |
+
## Modes
|
| 10 |
+
|
| 11 |
+
`mp-align` remains ALIGN v1: cyclic coefficient-space PCGrad within each chunk.
|
| 12 |
+
The separate `mp-align-new` first constructs the same local candidate, then
|
| 13 |
+
checks the **summed output-head update across the entire response**, including
|
| 14 |
+
cross-chunk terms. It interpolates toward Atomic only as much as required to
|
| 15 |
+
retain at least `rho=0.1` of Atomic's first-order descent in this proxy geometry.
|
| 16 |
+
This is the donor's specified guard, not an exception fallback or a norm cap.
|
| 17 |
+
A vanished anchor deliberately yields Atomic and sets the no-anchor metric.
|
| 18 |
+
|
| 19 |
+
The geometry is the LM-head output edge (weight plus bias if present), not the
|
| 20 |
+
full Transformer, tied-input embedding path or Adam-preconditioned update. Positive
|
| 21 |
+
proxy descent is not an accuracy guarantee. The guard uses detached coefficients;
|
| 22 |
+
no gradient flows through its beta or geometry. Precision context is restored.
|
| 23 |
+
|
| 24 |
+
New code lives in `kdflow/algorithms/mp_opd_align_new.py` and two additional donor
|
| 25 |
+
kernels inside `_mp_opd_port/`. The existing `mp_opd.py`, ALIGN v1, GRASS-Chunk,
|
| 26 |
+
Strict and author-code SimCT numerical files remain unchanged. Shared edits add
|
| 27 |
+
the mode, isolated arguments and launcher choice. Donor MIT notice is retained
|
| 28 |
+
in the same vendored directory. [Source hashes](ALIGN_NEW_SOURCE.json) pin the
|
| 29 |
+
new files, original test fixtures and unchanged dependencies.
|
| 30 |
+
|
| 31 |
+
## Run configuration
|
| 32 |
+
|
| 33 |
+
Print the full plan from this checkout on the company node:
|
| 34 |
+
|
| 35 |
+
```bash
|
| 36 |
+
git pull --ff-only
|
| 37 |
+
python3 tools/launch_single_b200.py --algorithm mp-align-new \
|
| 38 |
+
--runtime /opt/ctopd-b200-u22 --dry-run
|
| 39 |
+
```
|
| 40 |
+
|
| 41 |
+
Default chunks come from the donor tokenizer/string aligner (`xtoken`), with no
|
| 42 |
+
learned vocabulary projection. The explicit fixed-run ablation is
|
| 43 |
+
`--mp-chunk-source run --mp-chunk-run-length 2`. Both sources are tested. Generation,
|
| 44 |
+
optimizer batch, training iterations, models/data and MATH500 evaluation remain
|
| 45 |
+
the common CTOPD workload. Runtime archive and dependencies do not change.
|
| 46 |
+
|
| 47 |
+
The separate KDFlow arguments are:
|
| 48 |
+
|
| 49 |
+
- `mp_opd_align_new_min_descent_fraction=0.1` (finite, between 0 and 1).
|
| 50 |
+
- `mp_opd_align_new_eps_g=1e-12` (finite, nonnegative).
|
| 51 |
+
|
| 52 |
+
ALIGN-NEW requires `exact_head`, unit KD weight/temperature, RKL, one student rank
|
| 53 |
+
and ring size 1. `--mp-geometry diag` is rejected for this mode. Existing v1/GRASS
|
| 54 |
+
choices retain their contracts. Algorithm state identifies `align_new` separately
|
| 55 |
+
and refuses a v1 state; this adds no full optimizer/RNG resume support.
|
| 56 |
+
|
| 57 |
+
Only remove `--dry-run` on the actual allocated GPU with models/data ready and a
|
| 58 |
+
new run ID. No training was launched by this port. Full-response head geometry
|
| 59 |
+
adds full-vocabulary work; B200/FSDP integration, long-response memory/throughput
|
| 60 |
+
and scientific efficacy remain unqualified.
|
| 61 |
+
|
| 62 |
+
## Verification contract
|
| 63 |
+
|
| 64 |
+
The oracle executes the donor `_align_loss` and `_grass_chunk_assignment` method
|
| 65 |
+
bodies, plus separately loaded original copies of `_mp_opd_align_new.py` and
|
| 66 |
+
`_mp_opd_head_pair.py`. Existing v1 Gram, credit, chunk and atom helpers are frozen
|
| 67 |
+
shared dependencies, verified by source hashes; the oracle never calls the new
|
| 68 |
+
adapter. Source-method fixtures were checked against both the local donor and
|
| 69 |
+
the fetched GitHub revision. Kernel changes are LF normalization only.
|
| 70 |
+
|
| 71 |
+
CPU tests compare complete adapter loss and every student gradient for both chunk
|
| 72 |
+
sources, final softcap and rho endpoints. A constructed cross-chunk counterexample
|
| 73 |
+
activates the guard and checks the resulting loss/head gradient against the donor.
|
| 74 |
+
An independent autograd calculation checks full-response head norms/dot including
|
| 75 |
+
bias, softcap and different vocabulary tile widths. Other tests cover teacher
|
| 76 |
+
stop-gradient, microbatch normalization, degenerate/invalid geometry, argument
|
| 77 |
+
guards and v1 state rejection. Existing Strict/SimCT/MP regressions run alongside.
|
| 78 |
+
|
| 79 |
+
Use an existing CPU Torch environment, with no package installation:
|
| 80 |
+
|
| 81 |
+
```bash
|
| 82 |
+
TORCH_COMPILE_DISABLE=1 OMP_NUM_THREADS=1 python -m unittest discover -s tests -v
|
| 83 |
+
```
|
| 84 |
+
|
| 85 |
+
The [CPU receipt](ALIGN_NEW_VALIDATION.json) records 65 passing tests. Across
|
| 86 |
+
seven FP32 donor comparisons, maximum loss error is `0.0` and maximum gradient
|
| 87 |
+
error is `5.960464477539063e-08`. A separate CPU BF16 autocast test checks that
|
| 88 |
+
geometry reconstructs the same logits used for credits; it is not a B200 test.
|
docs/ALIGN_NEW_SOURCE.json
ADDED
|
@@ -0,0 +1,71 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"source_repository": "https://github.com/sontungkieu/SimCT",
|
| 3 |
+
"source_commit": "529ee4b691732e1040d6a45f05fd12f58def96ec",
|
| 4 |
+
"local_donor_commit": "99096a927a3e4ae92d7a105e89ca7c18a28c810a",
|
| 5 |
+
"algorithm": "align_new",
|
| 6 |
+
"version_inventory": "align v1 and align_new exist; no named v2/v3 found; user confirmed no further versions",
|
| 7 |
+
"files": [
|
| 8 |
+
{
|
| 9 |
+
"sha256": "a3c3e343ded03f4b88a415756558251f6eb08fd639f6997eba9a78c72bcb3c6a",
|
| 10 |
+
"path": "kdflow/algorithms/_mp_opd_port/_mp_opd_align_new.py"
|
| 11 |
+
},
|
| 12 |
+
{
|
| 13 |
+
"sha256": "be9163ccd7dbdd4e5a5228be7e06ce68f062a290709383c15b934658e0a83d58",
|
| 14 |
+
"path": "kdflow/algorithms/_mp_opd_port/_mp_opd_head_pair.py"
|
| 15 |
+
},
|
| 16 |
+
{
|
| 17 |
+
"sha256": "a3c3e343ded03f4b88a415756558251f6eb08fd639f6997eba9a78c72bcb3c6a",
|
| 18 |
+
"path": "tests/fixtures/align_new/_mp_opd_align_new.py.txt"
|
| 19 |
+
},
|
| 20 |
+
{
|
| 21 |
+
"sha256": "be9163ccd7dbdd4e5a5228be7e06ce68f062a290709383c15b934658e0a83d58",
|
| 22 |
+
"path": "tests/fixtures/align_new/_mp_opd_head_pair.py.txt"
|
| 23 |
+
},
|
| 24 |
+
{
|
| 25 |
+
"sha256": "b5f0d0aeb90c53848cb5b8ad2c91150b3200e8acc60fe1b0e05e7d9642e8c78a",
|
| 26 |
+
"path": "tests/fixtures/align_new/_align_loss.py.txt"
|
| 27 |
+
},
|
| 28 |
+
{
|
| 29 |
+
"sha256": "2ee28d6e55a868910b8b47193704d6a257b2d2dd17df9691d45724a09cf79e89",
|
| 30 |
+
"path": "tests/fixtures/align_new/_grass_chunk_assignment.py.txt"
|
| 31 |
+
}
|
| 32 |
+
],
|
| 33 |
+
"unchanged_existing_files": [
|
| 34 |
+
{
|
| 35 |
+
"sha256": "b2387d7f36502dea16c28f4391af4bc97a7e77f367d35dea2867c744e91277a6",
|
| 36 |
+
"path": "kdflow/algorithms/mp_opd.py"
|
| 37 |
+
},
|
| 38 |
+
{
|
| 39 |
+
"sha256": "d946926dd21fce6041e2fc3b2fd8d317d9dd8e845c5931783e1e6a2bcd70ca89",
|
| 40 |
+
"path": "kdflow/algorithms/simct.py"
|
| 41 |
+
},
|
| 42 |
+
{
|
| 43 |
+
"sha256": "fdbd99c55aeac4d061527158bdfc82a93333a42bf347eceaa134d34f5ef05952",
|
| 44 |
+
"path": "kdflow/algorithms/simple_ctkd.py"
|
| 45 |
+
},
|
| 46 |
+
{
|
| 47 |
+
"sha256": "5c6647c6fb8064ffd5f7b1ed971168cd1f3392caee62e2d75916d41931828e18",
|
| 48 |
+
"path": "kdflow/algorithms/byte_align_ctkd.py"
|
| 49 |
+
},
|
| 50 |
+
{
|
| 51 |
+
"sha256": "298925da4ce973b0f2917e5ba5ad2adbc4862d7c1a73515c5c86514ac0d22ddb",
|
| 52 |
+
"path": "kdflow/algorithms/topk_byte_align_ctkd.py"
|
| 53 |
+
},
|
| 54 |
+
{
|
| 55 |
+
"sha256": "97a8495d68da45c05aa87ba8d87fe6db6dd520cb15de4352cb2ea0e927ddb076",
|
| 56 |
+
"path": "kdflow/algorithms/_mp_opd_port/_mp_opd_grass_chunk.py"
|
| 57 |
+
},
|
| 58 |
+
{
|
| 59 |
+
"sha256": "524bd57bf3726552af3c997e99eea38a821f33aaac584943d1b81356f26b88c8",
|
| 60 |
+
"path": "kdflow/algorithms/_mp_opd_port/_mp_opd_grass_span.py"
|
| 61 |
+
},
|
| 62 |
+
{
|
| 63 |
+
"sha256": "cc26d6fe2c5b1893077dbbae5d5a311d6134441979926df4dde2be968678f224",
|
| 64 |
+
"path": "kdflow/algorithms/_mp_opd_port/_mp_opd_credit.py"
|
| 65 |
+
},
|
| 66 |
+
{
|
| 67 |
+
"sha256": "c6c0286c74215a8d30888c15b2bb5820ab77f54a45afd832db99ca6eb81cf1e0",
|
| 68 |
+
"path": "kdflow/algorithms/_mp_opd_port/_mp_opd_align.py"
|
| 69 |
+
}
|
| 70 |
+
]
|
| 71 |
+
}
|
docs/ALIGN_NEW_VALIDATION.json
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"date": "2026-10-19",
|
| 3 |
+
"donor_commit": "529ee4b691732e1040d6a45f05fd12f58def96ec",
|
| 4 |
+
"platform": "WSL Linux, CPU only",
|
| 5 |
+
"python": "3.12.3",
|
| 6 |
+
"torch": "2.14.0+cpu",
|
| 7 |
+
"command": "TORCH_COMPILE_DISABLE=1 OMP_NUM_THREADS=1 python -m unittest discover -s tests -v",
|
| 8 |
+
"full_suite": {
|
| 9 |
+
"passed": 65,
|
| 10 |
+
"skipped": 0,
|
| 11 |
+
"elapsed_seconds": 76.884
|
| 12 |
+
},
|
| 13 |
+
"align_new_parity": {
|
| 14 |
+
"cases": 7,
|
| 15 |
+
"max_loss_abs_error": 0.0,
|
| 16 |
+
"max_gradient_abs_error": 5.960464477539063e-08,
|
| 17 |
+
"dtype": "float32",
|
| 18 |
+
"atol": 1e-6,
|
| 19 |
+
"rtol": 1e-5,
|
| 20 |
+
"coverage": "Run/xtoken chunks, final softcap, rho=0/0.1/1, active cross-chunk guard",
|
| 21 |
+
"oracle": "Donor method bodies plus separately loaded original guard/head-pair copies; frozen source-verified v1 dependencies shared"
|
| 22 |
+
},
|
| 23 |
+
"additional_checks": [
|
| 24 |
+
"Independent autograd full-response head norms and dot, including bias and softcap",
|
| 25 |
+
"Vocabulary tiling invariance",
|
| 26 |
+
"CPU BF16 ambient autocast preserves original credit-logit/geometry composition",
|
| 27 |
+
"Teacher stop-gradient, microbatch normalization, invalid and degenerate geometry",
|
| 28 |
+
"Existing Strict, author-code SimCT and MP numerical regressions/source hashes",
|
| 29 |
+
"mp-align-new dry-run retains batch512, steps100, xtoken, exact_head and rho0.1",
|
| 30 |
+
"git diff --check"
|
| 31 |
+
],
|
| 32 |
+
"not_qualified": [
|
| 33 |
+
"Target Torch 2.11 CUDA, B200/FSDP integration and full-size memory/throughput",
|
| 34 |
+
"Full Transformer or Adam descent guarantees",
|
| 35 |
+
"Scientific efficacy, paper-score reproduction and full training resume"
|
| 36 |
+
]
|
| 37 |
+
}
|
docs/MP_OPD.md
CHANGED
|
@@ -17,6 +17,7 @@ the CTOPD launcher. The initial adapter requires one student rank, ring size 1,
|
|
| 17 |
| `mp-random2-5` | `mp_opd_random` | Hard pooling, group length sampled uniformly from [2,5] |
|
| 18 |
| `mp-grass-chunk` | `mp_opd_grass_chunk` | Risk-adaptive shrinkage within supplied chunks; no DP |
|
| 19 |
| `mp-align` | `mp_opd_align` | Donor ALIGN v1 cyclic coefficient-space PCGrad within the same chunks |
|
|
|
|
| 20 |
|
| 21 |
Fixed/random groups tile each response independently. The final tail may be
|
| 22 |
shorter than the requested length/minimum. Randomness uses the configured seed
|
|
@@ -51,7 +52,9 @@ vendored tokenizer/string aligner as the donor. This use does not need a learned
|
|
| 51 |
vocabulary projection. `--mp-chunk-source run --mp-chunk-run-length 2` explicitly
|
| 52 |
chooses the fixed-run ablation instead. `--mp-geometry diag` is a separate diagonal
|
| 53 |
geometry ablation; the default is `exact_head`. ALIGN here is v1 `align`, not
|
| 54 |
-
`align_new` or a full-Transformer/global-descent guard.
|
|
|
|
|
|
|
| 55 |
|
| 56 |
## Scientific and numerical contract
|
| 57 |
|
|
|
|
| 17 |
| `mp-random2-5` | `mp_opd_random` | Hard pooling, group length sampled uniformly from [2,5] |
|
| 18 |
| `mp-grass-chunk` | `mp_opd_grass_chunk` | Risk-adaptive shrinkage within supplied chunks; no DP |
|
| 19 |
| `mp-align` | `mp_opd_align` | Donor ALIGN v1 cyclic coefficient-space PCGrad within the same chunks |
|
| 20 |
+
| `mp-align-new` | `mp_opd_align_new` | Local ALIGN plus full-response head-descent guard; see [ALIGN_NEW.md](ALIGN_NEW.md) |
|
| 21 |
|
| 22 |
Fixed/random groups tile each response independently. The final tail may be
|
| 23 |
shorter than the requested length/minimum. Randomness uses the configured seed
|
|
|
|
| 52 |
vocabulary projection. `--mp-chunk-source run --mp-chunk-run-length 2` explicitly
|
| 53 |
chooses the fixed-run ablation instead. `--mp-geometry diag` is a separate diagonal
|
| 54 |
geometry ablation; the default is `exact_head`. ALIGN here is v1 `align`, not
|
| 55 |
+
`align_new` or a full-Transformer/global-descent guard. The separately ported
|
| 56 |
+
`mp-align-new` has a full-response **head** guard; it still does not measure the
|
| 57 |
+
full Transformer gradient. The donor does not name these modes v2/v3.
|
| 58 |
|
| 59 |
## Scientific and numerical contract
|
| 60 |
|
docs/SINGLE_B200.md
CHANGED
|
@@ -6,6 +6,8 @@ Separate experimental MP-OPD presets (Atomic, Fixed1-5, Random1-2/2-5, GRASS-Chu
|
|
| 6 |
|
| 7 |
The separate [SimCT author-code preset](SIMCT.md), `--algorithm simct`, uses the same common workload. It adds no SFT warmup; it is not the original SimCT end-to-end recipe.
|
| 8 |
|
|
|
|
|
|
|
| 9 |
## Preserved training and evaluation settings
|
| 10 |
|
| 11 |
| Setting | Value |
|
|
|
|
| 6 |
|
| 7 |
The separate [SimCT author-code preset](SIMCT.md), `--algorithm simct`, uses the same common workload. It adds no SFT warmup; it is not the original SimCT end-to-end recipe.
|
| 8 |
|
| 9 |
+
The separate [ALIGN-NEW preset](ALIGN_NEW.md), `--algorithm mp-align-new`, uses the same workload with the donor's full-response head guard. It requires exact-head geometry and adds no GPU/full-training qualification.
|
| 10 |
+
|
| 11 |
## Preserved training and evaluation settings
|
| 12 |
|
| 13 |
| Setting | Value |
|
kdflow/algorithms/__init__.py
CHANGED
|
@@ -14,6 +14,7 @@ from kdflow.algorithms.simple_ctkd import SimpleCrossTokenizerKD
|
|
| 14 |
from kdflow.algorithms.byte_align_ctkd import ByteAlignCrossTokenizerKD
|
| 15 |
from kdflow.algorithms.topk_byte_align_ctkd import TopKByteAlignCrossTokenizerKD
|
| 16 |
from kdflow.algorithms.simct import SimCT
|
|
|
|
| 17 |
from kdflow.algorithms.mp_opd import (
|
| 18 |
AtomicMPOnPolicyKD, FixedMPOnPolicyKD, RandomMPOnPolicyKD,
|
| 19 |
GrassChunkMPOnPolicyKD, AlignMPOnPolicyKD,
|
|
|
|
| 14 |
from kdflow.algorithms.byte_align_ctkd import ByteAlignCrossTokenizerKD
|
| 15 |
from kdflow.algorithms.topk_byte_align_ctkd import TopKByteAlignCrossTokenizerKD
|
| 16 |
from kdflow.algorithms.simct import SimCT
|
| 17 |
+
from kdflow.algorithms.mp_opd_align_new import AlignNewMPOnPolicyKD
|
| 18 |
from kdflow.algorithms.mp_opd import (
|
| 19 |
AtomicMPOnPolicyKD, FixedMPOnPolicyKD, RandomMPOnPolicyKD,
|
| 20 |
GrassChunkMPOnPolicyKD, AlignMPOnPolicyKD,
|
kdflow/algorithms/_mp_opd_port/_mp_opd_align_new.py
ADDED
|
@@ -0,0 +1,58 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""ALIGN-NEW: local ALIGN candidate, guarded by the full-response head update."""
|
| 2 |
+
from __future__ import annotations
|
| 3 |
+
|
| 4 |
+
import math
|
| 5 |
+
import torch
|
| 6 |
+
|
| 7 |
+
from ._mp_opd_head_pair import checked_geometry
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
def align_new_guard(rate, candidate, first_norm2, second_norm2, dot, *,
|
| 11 |
+
min_descent_fraction=.1, eps_g=1e-12):
|
| 12 |
+
"""Largest beta in [0,1] retaining rho of Atomic first-order descent.
|
| 13 |
+
|
| 14 |
+
g(beta) = (1-beta) g_atomic + beta g_candidate. Require
|
| 15 |
+
<g_atomic,g(beta)> >= rho ||g_atomic||^2, with rho fixed in the recipe.
|
| 16 |
+
No descent is defined for a vanished anchor: return Atomic and flag it.
|
| 17 |
+
Scalars are detached; there is no gradient through the geometry/guard.
|
| 18 |
+
"""
|
| 19 |
+
rho, eps = float(min_descent_fraction), float(eps_g)
|
| 20 |
+
if not (math.isfinite(rho) and 0 <= rho <= 1):
|
| 21 |
+
raise ValueError("ALIGN-NEW minimum descent fraction must be in [0,1]")
|
| 22 |
+
if not (math.isfinite(eps) and eps >= 0):
|
| 23 |
+
raise ValueError("ALIGN-NEW eps_g must be finite and nonnegative")
|
| 24 |
+
if rate.ndim != 1 or candidate.shape != rate.shape:
|
| 25 |
+
raise ValueError("ALIGN-NEW rates must have the same flat shape")
|
| 26 |
+
if not all(bool(torch.isfinite(t).all()) for t in (rate, candidate)):
|
| 27 |
+
raise FloatingPointError("non-finite ALIGN-NEW credit")
|
| 28 |
+
pair = checked_geometry(first_norm2, second_norm2, dot)
|
| 29 |
+
a, b, d = pair.first_norm2, pair.second_norm2, pair.dot
|
| 30 |
+
no_anchor = a <= eps
|
| 31 |
+
beta = 0.0 if no_anchor else (min(1., (1-rho)*a/(a-d)) if d < rho*a else 1.)
|
| 32 |
+
# Move infinitesimally into the feasible interval to absorb coefficient
|
| 33 |
+
# rounding, while retaining the largest feasible beta to numerical precision.
|
| 34 |
+
if 0 < beta < 1:
|
| 35 |
+
beta *= 1 - 1e-6
|
| 36 |
+
effective = rate.detach() + beta * (candidate.detach().to(rate.dtype) - rate.detach())
|
| 37 |
+
if not bool(torch.isfinite(effective).all()):
|
| 38 |
+
raise FloatingPointError("non-finite guarded ALIGN-NEW credit")
|
| 39 |
+
post_dot = (1-beta)*a + beta*d
|
| 40 |
+
post2 = max((1-beta)**2*a + beta**2*b + 2*beta*(1-beta)*d, 0.)
|
| 41 |
+
cosine = post_dot / math.sqrt(a*post2) if a > 0 and post2 > 0 else (1. if a == post2 == 0 else 0.)
|
| 42 |
+
candidate_cos = d / math.sqrt(a*b) if a > 0 and b > 0 else 0.
|
| 43 |
+
def m(value):
|
| 44 |
+
return rate.detach().new_tensor(float(value))
|
| 45 |
+
metrics = {
|
| 46 |
+
"mp_opd_align_new_beta": m(beta),
|
| 47 |
+
"mp_opd_align_new_guard_active_fraction": m(beta < 1),
|
| 48 |
+
"mp_opd_align_new_no_anchor_fraction": m(no_anchor),
|
| 49 |
+
"mp_opd_align_new_min_descent_fraction": m(rho),
|
| 50 |
+
"mp_opd_align_new_atomic_head_norm2": m(a),
|
| 51 |
+
"mp_opd_align_new_candidate_head_norm2": m(b),
|
| 52 |
+
"mp_opd_align_new_candidate_atomic_dot": m(d),
|
| 53 |
+
"mp_opd_align_new_candidate_head_update_cosine": m(max(-1., min(candidate_cos, 1.))),
|
| 54 |
+
"mp_opd_align_new_head_update_cosine": m(max(-1., min(cosine, 1.))),
|
| 55 |
+
"mp_opd_align_new_head_update_norm_ratio": m(math.sqrt(post2/a) if a > 0 else 0.),
|
| 56 |
+
"mp_opd_align_new_retained_descent_fraction": m(post_dot/a if a > 0 else 0.),
|
| 57 |
+
}
|
| 58 |
+
return effective, metrics
|
kdflow/algorithms/_mp_opd_port/_mp_opd_head_pair.py
ADDED
|
@@ -0,0 +1,110 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Full-response output-edge head geometry, including cross-chunk terms.
|
| 2 |
+
|
| 3 |
+
Two coefficient-weighted head updates are reduced in vocabulary tiles. No
|
| 4 |
+
global atom Gram or persistent [vocab, hidden] update is allocated. This is the
|
| 5 |
+
LM-head output contribution, not a full-Transformer (or tied-input) gradient.
|
| 6 |
+
"""
|
| 7 |
+
from __future__ import annotations
|
| 8 |
+
|
| 9 |
+
import math
|
| 10 |
+
from dataclasses import dataclass
|
| 11 |
+
|
| 12 |
+
import torch
|
| 13 |
+
|
| 14 |
+
from ._mp_opd_grass_span import _exact_fp32, _log_partition_from_selected
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
@dataclass(frozen=True)
|
| 18 |
+
class HeadPairGeometry:
|
| 19 |
+
first_norm2: float
|
| 20 |
+
second_norm2: float
|
| 21 |
+
dot: float
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
def checked_geometry(first_norm2, second_norm2, dot) -> HeadPairGeometry:
|
| 25 |
+
values = tuple(float(v) for v in (first_norm2, second_norm2, dot))
|
| 26 |
+
if not all(math.isfinite(v) for v in values):
|
| 27 |
+
raise FloatingPointError("head pair geometry must be finite")
|
| 28 |
+
a, b, d = values
|
| 29 |
+
if a < 0 or b < 0:
|
| 30 |
+
raise ValueError("head pair squared norms must be nonnegative")
|
| 31 |
+
bound = math.sqrt(a) * math.sqrt(b)
|
| 32 |
+
if abs(d) > bound * (1 + 1e-6) + 1e-300:
|
| 33 |
+
raise ValueError("head pair geometry violates Cauchy-Schwarz")
|
| 34 |
+
return HeadPairGeometry(a, b, max(-bound, min(d, bound)))
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
@torch.no_grad()
|
| 38 |
+
def head_pair_geometry(logits, hidden, labels, ranges, first, second, *,
|
| 39 |
+
selected_log_prob, softcap=None, head_bias=False,
|
| 40 |
+
vocab_chunk=4096) -> HeadPairGeometry:
|
| 41 |
+
"""Read <sum first_i G_i, sum second_i G_i> in exact fp32 head geometry.
|
| 42 |
+
|
| 43 |
+
Each atom loss is a TOKEN SUM. Thus an atom's coefficient is repeated on
|
| 44 |
+
every token, with no division by atom length. Bias and final softcap use
|
| 45 |
+
precisely the same output-edge derivative as the training loss.
|
| 46 |
+
"""
|
| 47 |
+
ranges = tuple(ranges)
|
| 48 |
+
if (not ranges or ranges[0][0] != 0 or
|
| 49 |
+
any(b <= a for a, b in ranges) or
|
| 50 |
+
any(ranges[i][1] != ranges[i+1][0] for i in range(len(ranges)-1))):
|
| 51 |
+
raise ValueError("head pair ranges must tile a nonempty covered prefix")
|
| 52 |
+
covered = ranges[-1][1]
|
| 53 |
+
if logits.ndim != 2 or hidden.ndim != 2 or labels.ndim != 1:
|
| 54 |
+
raise ValueError("head pair expects token-by-vocab/hidden and flat labels")
|
| 55 |
+
if min(logits.shape[0], hidden.shape[0], labels.numel(), selected_log_prob.numel()) < covered:
|
| 56 |
+
raise ValueError("head pair input is shorter than covered prefix")
|
| 57 |
+
if first.ndim != 1 or second.shape != first.shape or first.numel() != len(ranges):
|
| 58 |
+
raise ValueError("head pair coefficients must match atom count")
|
| 59 |
+
if labels.dtype not in (torch.int32, torch.int64):
|
| 60 |
+
raise ValueError("head pair labels must be integer token IDs")
|
| 61 |
+
if any(t.device != logits.device for t in (hidden, labels, first, second, selected_log_prob)):
|
| 62 |
+
raise ValueError("head pair inputs must share one device")
|
| 63 |
+
vocab, width = logits.shape[1], hidden.shape[1]
|
| 64 |
+
if vocab < 1 or width < 1 or int(vocab_chunk) < 1:
|
| 65 |
+
raise ValueError("head pair dimensions and vocabulary tile must be positive")
|
| 66 |
+
if softcap is not None and not (math.isfinite(float(softcap)) and float(softcap) > 0):
|
| 67 |
+
raise ValueError("head pair softcap must be finite and positive")
|
| 68 |
+
logits, hidden, labels = logits[:covered], hidden[:covered], labels[:covered]
|
| 69 |
+
selected_log_prob = selected_log_prob[:covered]
|
| 70 |
+
if not all(bool(torch.isfinite(t).all()) for t in (logits, hidden, first, second, selected_log_prob)):
|
| 71 |
+
raise FloatingPointError("non-finite head pair input")
|
| 72 |
+
if bool(((labels < 0) | (labels >= vocab)).any()):
|
| 73 |
+
raise ValueError("head pair label outside vocabulary")
|
| 74 |
+
tile = min(int(vocab_chunk), vocab)
|
| 75 |
+
# Conservative transient bound: logits/probabilities/Jacobian tiles, two
|
| 76 |
+
# head tiles including fp64 reduction temporaries, and weighted hidden states.
|
| 77 |
+
need = 16 * covered * tile + 48 * tile * width + 16 * covered * width
|
| 78 |
+
if logits.is_cuda and need > torch.cuda.mem_get_info(logits.device)[0] * .5:
|
| 79 |
+
raise ValueError("head pair vocabulary tiles exceed memory guard")
|
| 80 |
+
lengths = torch.tensor([b-a for a, b in ranges], device=logits.device)
|
| 81 |
+
qa = first.detach().float().repeat_interleave(lengths)
|
| 82 |
+
qb = second.detach().float().repeat_interleave(lengths)
|
| 83 |
+
ha = hidden.detach().float() * qa[:, None]
|
| 84 |
+
hb = hidden.detach().float() * qb[:, None]
|
| 85 |
+
logz = _log_partition_from_selected(logits, labels, selected_log_prob).float()
|
| 86 |
+
a2 = b2 = dot = 0.0
|
| 87 |
+
# Force exact fp32 irrespective of outer autocast; a bf16 matmul would make
|
| 88 |
+
# this guard measure a different update from the stated fp32 head proxy.
|
| 89 |
+
with _exact_fp32(), torch.autocast(device_type=logits.device.type, enabled=False):
|
| 90 |
+
for low in range(0, vocab, tile):
|
| 91 |
+
high = min(low+tile, vocab)
|
| 92 |
+
z = logits[:, low:high].detach().float()
|
| 93 |
+
delta = torch.exp(z - logz[:, None])
|
| 94 |
+
jac = None
|
| 95 |
+
if softcap is not None:
|
| 96 |
+
jac = (1 - (z / float(softcap)).clamp(-1, 1).square()).clamp_min(0)
|
| 97 |
+
delta *= jac
|
| 98 |
+
rows = ((labels >= low) & (labels < high)).nonzero().flatten()
|
| 99 |
+
cols = labels[rows] - low
|
| 100 |
+
delta[rows, cols] -= 1.0 if jac is None else jac[rows, cols]
|
| 101 |
+
ga, gb = (delta.T @ ha).double(), (delta.T @ hb).double()
|
| 102 |
+
a2 += float(ga.square().sum())
|
| 103 |
+
b2 += float(gb.square().sum())
|
| 104 |
+
dot += float((ga*gb).sum())
|
| 105 |
+
if head_bias:
|
| 106 |
+
ba, bb = (qa @ delta).double(), (qb @ delta).double()
|
| 107 |
+
a2 += float(ba.square().sum())
|
| 108 |
+
b2 += float(bb.square().sum())
|
| 109 |
+
dot += float((ba*bb).sum())
|
| 110 |
+
return checked_geometry(a2, b2, dot)
|
kdflow/algorithms/mp_opd_align_new.py
ADDED
|
@@ -0,0 +1,53 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Isolated donor ALIGN-NEW; no change to the ALIGN v1/GRASS adapter."""
|
| 2 |
+
import math
|
| 3 |
+
|
| 4 |
+
import torch
|
| 5 |
+
|
| 6 |
+
from . import register_algorithm
|
| 7 |
+
from .mp_opd import MPOnPolicyKD
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
@register_algorithm('mp_opd_align_new')
|
| 11 |
+
class AlignNewMPOnPolicyKD(MPOnPolicyKD):
|
| 12 |
+
# Let the unchanged parent initialize the same chunk aligner as ALIGN v1.
|
| 13 |
+
mode = 'align'
|
| 14 |
+
|
| 15 |
+
def __init__(self, *args, **kwargs):
|
| 16 |
+
super().__init__(*args, **kwargs)
|
| 17 |
+
self.mode = 'align_new'
|
| 18 |
+
self.rho = float(self.kd.mp_opd_align_new_min_descent_fraction)
|
| 19 |
+
self.eps_g = float(self.kd.mp_opd_align_new_eps_g)
|
| 20 |
+
if self.kd.mp_opd_grass_geometry != 'exact_head':
|
| 21 |
+
raise ValueError('ALIGN-NEW requires exact_head geometry')
|
| 22 |
+
if not math.isfinite(self.rho) or not 0 <= self.rho <= 1:
|
| 23 |
+
raise ValueError('ALIGN-NEW minimum descent fraction must be in [0,1]')
|
| 24 |
+
if not math.isfinite(self.eps_g) or self.eps_g < 0:
|
| 25 |
+
raise ValueError('ALIGN-NEW eps_g must be finite and nonnegative')
|
| 26 |
+
|
| 27 |
+
def _effective_rates(self, credits, atoms, partition, hidden, labels, **_kwargs):
|
| 28 |
+
from ._mp_opd_port._mp_opd_align import align_chunks, align_metrics
|
| 29 |
+
from ._mp_opd_port._mp_opd_align_new import align_new_guard
|
| 30 |
+
from ._mp_opd_port._mp_opd_grass_chunk import chunk_head_gram
|
| 31 |
+
from ._mp_opd_port._mp_opd_head_pair import head_pair_geometry
|
| 32 |
+
|
| 33 |
+
ranges = tuple((a.student_start, a.student_end) for a in atoms)
|
| 34 |
+
with torch.no_grad():
|
| 35 |
+
logits = self._head_logits(self.student.model.lm_head, hidden, self.student_cap, True)
|
| 36 |
+
has_bias = getattr(self.student.model.lm_head, 'bias', None) is not None
|
| 37 |
+
# Reconstruct the same logits as the credit path before disabling
|
| 38 |
+
# ambient autocast for donor geometry/local coefficient arithmetic.
|
| 39 |
+
with torch.autocast(device_type=hidden.device.type, enabled=False):
|
| 40 |
+
head = chunk_head_gram(logits, hidden.detach(), labels, ranges, partition,
|
| 41 |
+
selected_log_prob=-credits.student_token_nll, softcap=self.student_cap,
|
| 42 |
+
head_bias=has_bias, diagonal_only=False)
|
| 43 |
+
stats = align_chunks(credits.rate, head.gram, partition, eps_h=self.kd.mp_opd_align_eps_h)
|
| 44 |
+
candidate = stats['rate'].to(credits.rate.dtype)
|
| 45 |
+
metrics = align_metrics(credits.rate, candidate, head.gram, stats)
|
| 46 |
+
pair = head_pair_geometry(logits, hidden.detach(), labels, ranges, credits.rate, candidate,
|
| 47 |
+
selected_log_prob=-credits.student_token_nll, softcap=self.student_cap, head_bias=has_bias)
|
| 48 |
+
effective, global_metrics = align_new_guard(credits.rate, candidate,
|
| 49 |
+
pair.first_norm2, pair.second_norm2, pair.dot,
|
| 50 |
+
min_descent_fraction=self.rho, eps_g=self.eps_g)
|
| 51 |
+
metrics = {k.replace('mp_opd_align_', 'mp_opd_align_new_local_'): v for k, v in metrics.items()}
|
| 52 |
+
metrics.update(global_metrics)
|
| 53 |
+
return effective, metrics
|
kdflow/arguments/distillation_args.py
CHANGED
|
@@ -127,6 +127,8 @@ class DistillationArguments:
|
|
| 127 |
mp_opd_grass_sigma_rho: float = field(default=0.99)
|
| 128 |
mp_opd_grass_sigma_min_pairs: int = field(default=8)
|
| 129 |
mp_opd_align_eps_h: float = field(default=1e-12)
|
|
|
|
|
|
|
| 130 |
|
| 131 |
def __post_init__(self):
|
| 132 |
if self.multi_teacher_config is not None:
|
|
@@ -146,6 +148,13 @@ class DistillationArguments:
|
|
| 146 |
if not math.isfinite(self.simct_gh_mask_threshold) or self.simct_gh_mask_threshold < 0:
|
| 147 |
raise ValueError('SimCT G(h) threshold must be finite and nonnegative')
|
| 148 |
if self.kd_algorithm in MP_OPD_ALGORITHMS:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 149 |
if self.kd_ratio != 1.0 or self.kd_temperature != 1.0 or self.kd_loss_fn != 'rkl':
|
| 150 |
raise ValueError('MP-OPD requires kd_ratio=1, kd_temperature=1, kd_loss_fn=rkl')
|
| 151 |
if self.mp_opd_fixed_span_length < 1 or not 1 <= self.mp_opd_random_min_length <= self.mp_opd_random_max_length:
|
|
|
|
| 127 |
mp_opd_grass_sigma_rho: float = field(default=0.99)
|
| 128 |
mp_opd_grass_sigma_min_pairs: int = field(default=8)
|
| 129 |
mp_opd_align_eps_h: float = field(default=1e-12)
|
| 130 |
+
mp_opd_align_new_min_descent_fraction: float = field(default=0.1)
|
| 131 |
+
mp_opd_align_new_eps_g: float = field(default=1e-12)
|
| 132 |
|
| 133 |
def __post_init__(self):
|
| 134 |
if self.multi_teacher_config is not None:
|
|
|
|
| 148 |
if not math.isfinite(self.simct_gh_mask_threshold) or self.simct_gh_mask_threshold < 0:
|
| 149 |
raise ValueError('SimCT G(h) threshold must be finite and nonnegative')
|
| 150 |
if self.kd_algorithm in MP_OPD_ALGORITHMS:
|
| 151 |
+
if self.kd_algorithm == 'mp_opd_align_new':
|
| 152 |
+
if self.mp_opd_grass_geometry != 'exact_head':
|
| 153 |
+
raise ValueError('ALIGN-NEW requires exact_head geometry')
|
| 154 |
+
if not math.isfinite(self.mp_opd_align_new_min_descent_fraction) or not 0 <= self.mp_opd_align_new_min_descent_fraction <= 1:
|
| 155 |
+
raise ValueError('ALIGN-NEW minimum descent fraction must be in [0,1]')
|
| 156 |
+
if not math.isfinite(self.mp_opd_align_new_eps_g) or self.mp_opd_align_new_eps_g < 0:
|
| 157 |
+
raise ValueError('ALIGN-NEW eps_g must be finite and nonnegative')
|
| 158 |
if self.kd_ratio != 1.0 or self.kd_temperature != 1.0 or self.kd_loss_fn != 'rkl':
|
| 159 |
raise ValueError('MP-OPD requires kd_ratio=1, kd_temperature=1, kd_loss_fn=rkl')
|
| 160 |
if self.mp_opd_fixed_span_length < 1 or not 1 <= self.mp_opd_random_min_length <= self.mp_opd_random_max_length:
|
kdflow/mp_opd_modes.py
CHANGED
|
@@ -2,5 +2,5 @@
|
|
| 2 |
|
| 3 |
MP_OPD_ALGORITHMS = (
|
| 4 |
'mp_opd_atomic', 'mp_opd_fixed', 'mp_opd_random',
|
| 5 |
-
'mp_opd_grass_chunk', 'mp_opd_align',
|
| 6 |
)
|
|
|
|
| 2 |
|
| 3 |
MP_OPD_ALGORITHMS = (
|
| 4 |
'mp_opd_atomic', 'mp_opd_fixed', 'mp_opd_random',
|
| 5 |
+
'mp_opd_grass_chunk', 'mp_opd_align', 'mp_opd_align_new',
|
| 6 |
)
|
tests/fixtures/align_new/_align_loss.py.txt
ADDED
|
@@ -0,0 +1,114 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
def _align_loss(
|
| 2 |
+
self,
|
| 3 |
+
credits,
|
| 4 |
+
atoms,
|
| 5 |
+
*,
|
| 6 |
+
student_logits: torch.Tensor | None,
|
| 7 |
+
student_labels: torch.Tensor | None,
|
| 8 |
+
student_hidden: torch.Tensor | None,
|
| 9 |
+
student_ids: list[int] | None = None,
|
| 10 |
+
teacher_ids: list[int] | None = None,
|
| 11 |
+
):
|
| 12 |
+
"""ALIGN: remove negative pairwise interference inside alignment chunks.
|
| 13 |
+
|
| 14 |
+
The chunk boundaries, the exact LM-head Gram and the credits themselves are
|
| 15 |
+
the GRASS-Chunk ones, reused unchanged, so the only difference between the
|
| 16 |
+
two modes is what happens to the credit after it leaves the atomizer. Here
|
| 17 |
+
nothing is shrunk and nothing is pooled: a PCGrad projection is applied in
|
| 18 |
+
coefficient space, and its sum over each chunk becomes the credit the loss
|
| 19 |
+
consumes. No renormalization follows, because a projection is meant to be
|
| 20 |
+
measured, and rescaling it would erase the size of what it did.
|
| 21 |
+
"""
|
| 22 |
+
if student_logits is None or student_labels is None:
|
| 23 |
+
raise RuntimeError(
|
| 24 |
+
"mp_opd align mode requires the student logits and labels"
|
| 25 |
+
)
|
| 26 |
+
if student_hidden is None:
|
| 27 |
+
raise RuntimeError(
|
| 28 |
+
"mp_opd align mode requires the student LM-head hidden states"
|
| 29 |
+
)
|
| 30 |
+
if credits.student_token_nll is None:
|
| 31 |
+
raise RuntimeError(
|
| 32 |
+
"mp_opd align mode requires per-token student NLL"
|
| 33 |
+
)
|
| 34 |
+
if credits.student_token_nll.numel() != student_logits.shape[0]:
|
| 35 |
+
raise ValueError(
|
| 36 |
+
"per-token student NLL and student logits disagree on token count"
|
| 37 |
+
)
|
| 38 |
+
if self.grass_chunk_source == "xtoken" and (
|
| 39 |
+
student_ids is None or teacher_ids is None
|
| 40 |
+
):
|
| 41 |
+
raise RuntimeError(
|
| 42 |
+
"mp_opd grass_chunk_source='xtoken' needs the sampled student and "
|
| 43 |
+
"teacher token ids for the alignment"
|
| 44 |
+
)
|
| 45 |
+
|
| 46 |
+
atom_ranges = tuple((atom.student_start, atom.student_end) for atom in atoms)
|
| 47 |
+
covered = atom_ranges[-1][1]
|
| 48 |
+
assignment = self._grass_chunk_assignment(atoms, student_ids, teacher_ids)
|
| 49 |
+
partition, _noncontiguous = chunk_partition(assignment)
|
| 50 |
+
local_precision = (torch.autocast(device_type=student_logits.device.type, enabled=False)
|
| 51 |
+
if self.mode == "align_new" else contextlib.nullcontext())
|
| 52 |
+
with local_precision:
|
| 53 |
+
head = chunk_head_gram(
|
| 54 |
+
student_logits[:covered],
|
| 55 |
+
student_hidden[:covered],
|
| 56 |
+
student_labels[:covered],
|
| 57 |
+
atom_ranges,
|
| 58 |
+
partition,
|
| 59 |
+
selected_log_prob=-credits.student_token_nll[:covered],
|
| 60 |
+
softcap=self.grass_softcap,
|
| 61 |
+
head_bias=self.grass_head_bias,
|
| 62 |
+
diagonal_only=self.grass_geometry == "diag",
|
| 63 |
+
)
|
| 64 |
+
stats = align_chunks(
|
| 65 |
+
credits.rate,
|
| 66 |
+
head.gram,
|
| 67 |
+
partition,
|
| 68 |
+
eps_h=self.align_eps_h,
|
| 69 |
+
)
|
| 70 |
+
effective = stats["rate"].to(credits.rate.dtype)
|
| 71 |
+
metrics = align_metrics(credits.rate, effective, head.gram, stats)
|
| 72 |
+
metrics.update(
|
| 73 |
+
{
|
| 74 |
+
"mp_opd_align_exact_geometry": credits.rate.new_tensor(
|
| 75 |
+
float(self.grass_geometry == "exact_head")
|
| 76 |
+
),
|
| 77 |
+
"mp_opd_align_gram_symmetry_error": credits.rate.new_tensor(
|
| 78 |
+
head.symmetry_error
|
| 79 |
+
),
|
| 80 |
+
"mp_opd_align_atom_token_count": credits.rate.new_tensor(
|
| 81 |
+
float(head.token_count)
|
| 82 |
+
),
|
| 83 |
+
"mp_opd_align_softcap": credits.rate.new_tensor(
|
| 84 |
+
float(self.grass_softcap or 0.0)
|
| 85 |
+
),
|
| 86 |
+
"mp_opd_align_head_bias": credits.rate.new_tensor(
|
| 87 |
+
float(self.grass_head_bias)
|
| 88 |
+
),
|
| 89 |
+
"mp_opd_align_chunk_source": credits.rate.new_tensor(
|
| 90 |
+
float(self.grass_chunk_source == "xtoken")
|
| 91 |
+
),
|
| 92 |
+
}
|
| 93 |
+
)
|
| 94 |
+
if self.mode == "align_new":
|
| 95 |
+
# The block Gram above is valid for LOCAL surgery only. These two
|
| 96 |
+
# explicit summed head updates include ALL cross-chunk terms.
|
| 97 |
+
pair = head_pair_geometry(
|
| 98 |
+
student_logits[:covered], student_hidden[:covered],
|
| 99 |
+
student_labels[:covered], atom_ranges, credits.rate, effective,
|
| 100 |
+
selected_log_prob=-credits.student_token_nll[:covered],
|
| 101 |
+
softcap=self.grass_softcap, head_bias=self.grass_head_bias,
|
| 102 |
+
)
|
| 103 |
+
effective, global_metrics = align_new_guard(
|
| 104 |
+
credits.rate, effective, pair.first_norm2, pair.second_norm2, pair.dot,
|
| 105 |
+
min_descent_fraction=self.align_new_min_descent_fraction,
|
| 106 |
+
eps_g=self.trust_eps_g,
|
| 107 |
+
)
|
| 108 |
+
# Old metric names imply a total update. For the new mode explicitly
|
| 109 |
+
# tag every old diagnostic as the LOCAL candidate, then add the true
|
| 110 |
+
# full-response geometry under its own namespace.
|
| 111 |
+
metrics = {k.replace("mp_opd_align_", "mp_opd_align_new_local_"): v
|
| 112 |
+
for k, v in metrics.items()}
|
| 113 |
+
metrics.update(global_metrics)
|
| 114 |
+
return soft_partition_loss(credits.current_nll, effective), metrics
|
tests/fixtures/align_new/_grass_chunk_assignment.py.txt
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
def _grass_chunk_assignment(self, atoms, student_ids, teacher_ids):
|
| 2 |
+
"""Map this response's atoms onto the upstream alignment chunks.
|
| 3 |
+
|
| 4 |
+
With ``source='xtoken'`` the aligner is run on the same loss-masked token
|
| 5 |
+
axis the atomizer used, so a chunk boundary and an atom boundary live in
|
| 6 |
+
the same coordinates and their disagreement is directly measurable.
|
| 7 |
+
"""
|
| 8 |
+
n = len(atoms)
|
| 9 |
+
if self.grass_chunk_source == "run":
|
| 10 |
+
return run_chunk_assignment(n, self.grass_chunk_run_length)
|
| 11 |
+
alignment = self.grass_aligner.align(
|
| 12 |
+
torch.tensor([list(student_ids)], dtype=torch.long),
|
| 13 |
+
torch.tensor([list(teacher_ids)], dtype=torch.long),
|
| 14 |
+
)
|
| 15 |
+
token_chunk_ids = alignment.student_chunk_id[0].tolist()
|
| 16 |
+
atom_ranges = tuple((atom.student_start, atom.student_end) for atom in atoms)
|
| 17 |
+
return atom_chunk_ids_from_tokens(
|
| 18 |
+
token_chunk_ids, atom_ranges, policy=self.grass_chunk_straddle
|
| 19 |
+
)
|
tests/fixtures/align_new/_mp_opd_align_new.py.txt
ADDED
|
@@ -0,0 +1,58 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""ALIGN-NEW: local ALIGN candidate, guarded by the full-response head update."""
|
| 2 |
+
from __future__ import annotations
|
| 3 |
+
|
| 4 |
+
import math
|
| 5 |
+
import torch
|
| 6 |
+
|
| 7 |
+
from ._mp_opd_head_pair import checked_geometry
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
def align_new_guard(rate, candidate, first_norm2, second_norm2, dot, *,
|
| 11 |
+
min_descent_fraction=.1, eps_g=1e-12):
|
| 12 |
+
"""Largest beta in [0,1] retaining rho of Atomic first-order descent.
|
| 13 |
+
|
| 14 |
+
g(beta) = (1-beta) g_atomic + beta g_candidate. Require
|
| 15 |
+
<g_atomic,g(beta)> >= rho ||g_atomic||^2, with rho fixed in the recipe.
|
| 16 |
+
No descent is defined for a vanished anchor: return Atomic and flag it.
|
| 17 |
+
Scalars are detached; there is no gradient through the geometry/guard.
|
| 18 |
+
"""
|
| 19 |
+
rho, eps = float(min_descent_fraction), float(eps_g)
|
| 20 |
+
if not (math.isfinite(rho) and 0 <= rho <= 1):
|
| 21 |
+
raise ValueError("ALIGN-NEW minimum descent fraction must be in [0,1]")
|
| 22 |
+
if not (math.isfinite(eps) and eps >= 0):
|
| 23 |
+
raise ValueError("ALIGN-NEW eps_g must be finite and nonnegative")
|
| 24 |
+
if rate.ndim != 1 or candidate.shape != rate.shape:
|
| 25 |
+
raise ValueError("ALIGN-NEW rates must have the same flat shape")
|
| 26 |
+
if not all(bool(torch.isfinite(t).all()) for t in (rate, candidate)):
|
| 27 |
+
raise FloatingPointError("non-finite ALIGN-NEW credit")
|
| 28 |
+
pair = checked_geometry(first_norm2, second_norm2, dot)
|
| 29 |
+
a, b, d = pair.first_norm2, pair.second_norm2, pair.dot
|
| 30 |
+
no_anchor = a <= eps
|
| 31 |
+
beta = 0.0 if no_anchor else (min(1., (1-rho)*a/(a-d)) if d < rho*a else 1.)
|
| 32 |
+
# Move infinitesimally into the feasible interval to absorb coefficient
|
| 33 |
+
# rounding, while retaining the largest feasible beta to numerical precision.
|
| 34 |
+
if 0 < beta < 1:
|
| 35 |
+
beta *= 1 - 1e-6
|
| 36 |
+
effective = rate.detach() + beta * (candidate.detach().to(rate.dtype) - rate.detach())
|
| 37 |
+
if not bool(torch.isfinite(effective).all()):
|
| 38 |
+
raise FloatingPointError("non-finite guarded ALIGN-NEW credit")
|
| 39 |
+
post_dot = (1-beta)*a + beta*d
|
| 40 |
+
post2 = max((1-beta)**2*a + beta**2*b + 2*beta*(1-beta)*d, 0.)
|
| 41 |
+
cosine = post_dot / math.sqrt(a*post2) if a > 0 and post2 > 0 else (1. if a == post2 == 0 else 0.)
|
| 42 |
+
candidate_cos = d / math.sqrt(a*b) if a > 0 and b > 0 else 0.
|
| 43 |
+
def m(value):
|
| 44 |
+
return rate.detach().new_tensor(float(value))
|
| 45 |
+
metrics = {
|
| 46 |
+
"mp_opd_align_new_beta": m(beta),
|
| 47 |
+
"mp_opd_align_new_guard_active_fraction": m(beta < 1),
|
| 48 |
+
"mp_opd_align_new_no_anchor_fraction": m(no_anchor),
|
| 49 |
+
"mp_opd_align_new_min_descent_fraction": m(rho),
|
| 50 |
+
"mp_opd_align_new_atomic_head_norm2": m(a),
|
| 51 |
+
"mp_opd_align_new_candidate_head_norm2": m(b),
|
| 52 |
+
"mp_opd_align_new_candidate_atomic_dot": m(d),
|
| 53 |
+
"mp_opd_align_new_candidate_head_update_cosine": m(max(-1., min(candidate_cos, 1.))),
|
| 54 |
+
"mp_opd_align_new_head_update_cosine": m(max(-1., min(cosine, 1.))),
|
| 55 |
+
"mp_opd_align_new_head_update_norm_ratio": m(math.sqrt(post2/a) if a > 0 else 0.),
|
| 56 |
+
"mp_opd_align_new_retained_descent_fraction": m(post_dot/a if a > 0 else 0.),
|
| 57 |
+
}
|
| 58 |
+
return effective, metrics
|
tests/fixtures/align_new/_mp_opd_head_pair.py.txt
ADDED
|
@@ -0,0 +1,110 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Full-response output-edge head geometry, including cross-chunk terms.
|
| 2 |
+
|
| 3 |
+
Two coefficient-weighted head updates are reduced in vocabulary tiles. No
|
| 4 |
+
global atom Gram or persistent [vocab, hidden] update is allocated. This is the
|
| 5 |
+
LM-head output contribution, not a full-Transformer (or tied-input) gradient.
|
| 6 |
+
"""
|
| 7 |
+
from __future__ import annotations
|
| 8 |
+
|
| 9 |
+
import math
|
| 10 |
+
from dataclasses import dataclass
|
| 11 |
+
|
| 12 |
+
import torch
|
| 13 |
+
|
| 14 |
+
from ._mp_opd_grass_span import _exact_fp32, _log_partition_from_selected
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
@dataclass(frozen=True)
|
| 18 |
+
class HeadPairGeometry:
|
| 19 |
+
first_norm2: float
|
| 20 |
+
second_norm2: float
|
| 21 |
+
dot: float
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
def checked_geometry(first_norm2, second_norm2, dot) -> HeadPairGeometry:
|
| 25 |
+
values = tuple(float(v) for v in (first_norm2, second_norm2, dot))
|
| 26 |
+
if not all(math.isfinite(v) for v in values):
|
| 27 |
+
raise FloatingPointError("head pair geometry must be finite")
|
| 28 |
+
a, b, d = values
|
| 29 |
+
if a < 0 or b < 0:
|
| 30 |
+
raise ValueError("head pair squared norms must be nonnegative")
|
| 31 |
+
bound = math.sqrt(a) * math.sqrt(b)
|
| 32 |
+
if abs(d) > bound * (1 + 1e-6) + 1e-300:
|
| 33 |
+
raise ValueError("head pair geometry violates Cauchy-Schwarz")
|
| 34 |
+
return HeadPairGeometry(a, b, max(-bound, min(d, bound)))
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
@torch.no_grad()
|
| 38 |
+
def head_pair_geometry(logits, hidden, labels, ranges, first, second, *,
|
| 39 |
+
selected_log_prob, softcap=None, head_bias=False,
|
| 40 |
+
vocab_chunk=4096) -> HeadPairGeometry:
|
| 41 |
+
"""Read <sum first_i G_i, sum second_i G_i> in exact fp32 head geometry.
|
| 42 |
+
|
| 43 |
+
Each atom loss is a TOKEN SUM. Thus an atom's coefficient is repeated on
|
| 44 |
+
every token, with no division by atom length. Bias and final softcap use
|
| 45 |
+
precisely the same output-edge derivative as the training loss.
|
| 46 |
+
"""
|
| 47 |
+
ranges = tuple(ranges)
|
| 48 |
+
if (not ranges or ranges[0][0] != 0 or
|
| 49 |
+
any(b <= a for a, b in ranges) or
|
| 50 |
+
any(ranges[i][1] != ranges[i+1][0] for i in range(len(ranges)-1))):
|
| 51 |
+
raise ValueError("head pair ranges must tile a nonempty covered prefix")
|
| 52 |
+
covered = ranges[-1][1]
|
| 53 |
+
if logits.ndim != 2 or hidden.ndim != 2 or labels.ndim != 1:
|
| 54 |
+
raise ValueError("head pair expects token-by-vocab/hidden and flat labels")
|
| 55 |
+
if min(logits.shape[0], hidden.shape[0], labels.numel(), selected_log_prob.numel()) < covered:
|
| 56 |
+
raise ValueError("head pair input is shorter than covered prefix")
|
| 57 |
+
if first.ndim != 1 or second.shape != first.shape or first.numel() != len(ranges):
|
| 58 |
+
raise ValueError("head pair coefficients must match atom count")
|
| 59 |
+
if labels.dtype not in (torch.int32, torch.int64):
|
| 60 |
+
raise ValueError("head pair labels must be integer token IDs")
|
| 61 |
+
if any(t.device != logits.device for t in (hidden, labels, first, second, selected_log_prob)):
|
| 62 |
+
raise ValueError("head pair inputs must share one device")
|
| 63 |
+
vocab, width = logits.shape[1], hidden.shape[1]
|
| 64 |
+
if vocab < 1 or width < 1 or int(vocab_chunk) < 1:
|
| 65 |
+
raise ValueError("head pair dimensions and vocabulary tile must be positive")
|
| 66 |
+
if softcap is not None and not (math.isfinite(float(softcap)) and float(softcap) > 0):
|
| 67 |
+
raise ValueError("head pair softcap must be finite and positive")
|
| 68 |
+
logits, hidden, labels = logits[:covered], hidden[:covered], labels[:covered]
|
| 69 |
+
selected_log_prob = selected_log_prob[:covered]
|
| 70 |
+
if not all(bool(torch.isfinite(t).all()) for t in (logits, hidden, first, second, selected_log_prob)):
|
| 71 |
+
raise FloatingPointError("non-finite head pair input")
|
| 72 |
+
if bool(((labels < 0) | (labels >= vocab)).any()):
|
| 73 |
+
raise ValueError("head pair label outside vocabulary")
|
| 74 |
+
tile = min(int(vocab_chunk), vocab)
|
| 75 |
+
# Conservative transient bound: logits/probabilities/Jacobian tiles, two
|
| 76 |
+
# head tiles including fp64 reduction temporaries, and weighted hidden states.
|
| 77 |
+
need = 16 * covered * tile + 48 * tile * width + 16 * covered * width
|
| 78 |
+
if logits.is_cuda and need > torch.cuda.mem_get_info(logits.device)[0] * .5:
|
| 79 |
+
raise ValueError("head pair vocabulary tiles exceed memory guard")
|
| 80 |
+
lengths = torch.tensor([b-a for a, b in ranges], device=logits.device)
|
| 81 |
+
qa = first.detach().float().repeat_interleave(lengths)
|
| 82 |
+
qb = second.detach().float().repeat_interleave(lengths)
|
| 83 |
+
ha = hidden.detach().float() * qa[:, None]
|
| 84 |
+
hb = hidden.detach().float() * qb[:, None]
|
| 85 |
+
logz = _log_partition_from_selected(logits, labels, selected_log_prob).float()
|
| 86 |
+
a2 = b2 = dot = 0.0
|
| 87 |
+
# Force exact fp32 irrespective of outer autocast; a bf16 matmul would make
|
| 88 |
+
# this guard measure a different update from the stated fp32 head proxy.
|
| 89 |
+
with _exact_fp32(), torch.autocast(device_type=logits.device.type, enabled=False):
|
| 90 |
+
for low in range(0, vocab, tile):
|
| 91 |
+
high = min(low+tile, vocab)
|
| 92 |
+
z = logits[:, low:high].detach().float()
|
| 93 |
+
delta = torch.exp(z - logz[:, None])
|
| 94 |
+
jac = None
|
| 95 |
+
if softcap is not None:
|
| 96 |
+
jac = (1 - (z / float(softcap)).clamp(-1, 1).square()).clamp_min(0)
|
| 97 |
+
delta *= jac
|
| 98 |
+
rows = ((labels >= low) & (labels < high)).nonzero().flatten()
|
| 99 |
+
cols = labels[rows] - low
|
| 100 |
+
delta[rows, cols] -= 1.0 if jac is None else jac[rows, cols]
|
| 101 |
+
ga, gb = (delta.T @ ha).double(), (delta.T @ hb).double()
|
| 102 |
+
a2 += float(ga.square().sum())
|
| 103 |
+
b2 += float(gb.square().sum())
|
| 104 |
+
dot += float((ga*gb).sum())
|
| 105 |
+
if head_bias:
|
| 106 |
+
ba, bb = (qa @ delta).double(), (qb @ delta).double()
|
| 107 |
+
a2 += float(ba.square().sum())
|
| 108 |
+
b2 += float(bb.square().sum())
|
| 109 |
+
dot += float((ba*bb).sum())
|
| 110 |
+
return checked_geometry(a2, b2, dot)
|
tests/test_align_new.py
ADDED
|
@@ -0,0 +1,245 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""ALIGN-NEW donor-composition parity and independent output-head geometry checks."""
|
| 2 |
+
import contextlib
|
| 3 |
+
import copy
|
| 4 |
+
import hashlib
|
| 5 |
+
import importlib.machinery
|
| 6 |
+
import importlib.util
|
| 7 |
+
import json
|
| 8 |
+
import pathlib
|
| 9 |
+
import sys
|
| 10 |
+
import types
|
| 11 |
+
import unittest
|
| 12 |
+
|
| 13 |
+
import test_mp_opd_port as fixtures
|
| 14 |
+
|
| 15 |
+
ROOT = pathlib.Path(__file__).resolve().parents[1]
|
| 16 |
+
if fixtures.HAS_TORCH:
|
| 17 |
+
import torch
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
def original_scope():
|
| 21 |
+
"""Original guard/head-pair copies and donor method bodies, no adapter calls.
|
| 22 |
+
|
| 23 |
+
Existing v1 Gram/credit/chunk helpers are frozen, source-hash checked dependencies.
|
| 24 |
+
"""
|
| 25 |
+
from kdflow.algorithms._mp_opd_port import _mp_opd_grass_span as span
|
| 26 |
+
from kdflow.algorithms._mp_opd_port import _mp_opd_grass_chunk as chunk
|
| 27 |
+
from kdflow.algorithms._mp_opd_port import _mp_opd_align as align
|
| 28 |
+
from kdflow.algorithms._mp_opd_port._mp_opd_credit import soft_partition_loss
|
| 29 |
+
root = ROOT / 'tests/fixtures/align_new'
|
| 30 |
+
package = types.ModuleType('_align_original')
|
| 31 |
+
package.__path__ = [str(root)]
|
| 32 |
+
sys.modules['_align_original'] = package
|
| 33 |
+
sys.modules['_align_original._mp_opd_grass_span'] = span
|
| 34 |
+
for name in ('_mp_opd_head_pair', '_mp_opd_align_new'):
|
| 35 |
+
full = '_align_original.' + name
|
| 36 |
+
loader = importlib.machinery.SourceFileLoader(full, str(root / (name + '.py.txt')))
|
| 37 |
+
spec = importlib.util.spec_from_loader(full, loader)
|
| 38 |
+
module = importlib.util.module_from_spec(spec)
|
| 39 |
+
sys.modules[full] = module
|
| 40 |
+
loader.exec_module(module)
|
| 41 |
+
pair = sys.modules['_align_original._mp_opd_head_pair']
|
| 42 |
+
guard = sys.modules['_align_original._mp_opd_align_new']
|
| 43 |
+
scope = dict(torch=torch, contextlib=contextlib, chunk_head_gram=chunk.chunk_head_gram,
|
| 44 |
+
chunk_partition=chunk.chunk_partition, align_chunks=align.align_chunks,
|
| 45 |
+
align_metrics=align.align_metrics, head_pair_geometry=pair.head_pair_geometry,
|
| 46 |
+
align_new_guard=guard.align_new_guard, soft_partition_loss=soft_partition_loss,
|
| 47 |
+
run_chunk_assignment=chunk.run_chunk_assignment,
|
| 48 |
+
atom_chunk_ids_from_tokens=chunk.atom_chunk_ids_from_tokens)
|
| 49 |
+
for name in ('_align_loss', '_grass_chunk_assignment'):
|
| 50 |
+
exec(compile((root / (name + '.py.txt')).read_text(), 'donor_' + name, 'exec'), scope)
|
| 51 |
+
return scope
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
@unittest.skipUnless(fixtures.HAS_TORCH, 'requires existing PyTorch; no bootstrap')
|
| 55 |
+
class AlignNewTests(unittest.TestCase):
|
| 56 |
+
errors = []
|
| 57 |
+
|
| 58 |
+
@classmethod
|
| 59 |
+
def tearDownClass(cls):
|
| 60 |
+
if cls.errors:
|
| 61 |
+
print('ALIGN_NEW_PARITY ' + json.dumps({'cases': len(cls.errors),
|
| 62 |
+
'max_loss_abs_error': max(v[0] for v in cls.errors),
|
| 63 |
+
'max_gradient_abs_error': max(v[1] for v in cls.errors)}))
|
| 64 |
+
|
| 65 |
+
def setup_pair(self, *, source='run', cap=None, rho=.1):
|
| 66 |
+
from kdflow.algorithms.mp_opd_align_new import AlignNewMPOnPolicyKD
|
| 67 |
+
helper = fixtures.MPPortTests()
|
| 68 |
+
mp, student, teacher = helper.fixture(source=source, cap=cap)
|
| 69 |
+
mp.kd.mp_opd_align_new_min_descent_fraction = rho
|
| 70 |
+
mp.kd.mp_opd_align_new_eps_g = 1e-12
|
| 71 |
+
algo = AlignNewMPOnPolicyKD(mp.strategy, student, teacher, fixtures.Tokenizer(), fixtures.Tokenizer())
|
| 72 |
+
scope = original_scope()
|
| 73 |
+
reference = types.SimpleNamespace(mode='align_new', grass_chunk_source=source,
|
| 74 |
+
grass_chunk_run_length=mp.kd.mp_opd_chunk_run_length,
|
| 75 |
+
grass_chunk_straddle=mp.kd.mp_opd_chunk_straddle, grass_aligner=algo.aligner,
|
| 76 |
+
grass_geometry='exact_head', grass_softcap=cap, grass_head_bias=False,
|
| 77 |
+
align_eps_h=mp.kd.mp_opd_align_eps_h, align_new_min_descent_fraction=rho, trust_eps_g=1e-12)
|
| 78 |
+
reference._grass_chunk_assignment = types.MethodType(scope['_grass_chunk_assignment'], reference)
|
| 79 |
+
return helper, algo, student, teacher, scope, reference
|
| 80 |
+
|
| 81 |
+
def reference_loss(self, algo, student, teacher, batch, scope, reference):
|
| 82 |
+
from kdflow.algorithms._mp_opd_port._mp_opd_credit import build_atom_credits
|
| 83 |
+
sm, tm = batch['stu_loss_mask'], batch['tea_loss_mask']
|
| 84 |
+
sh = student(batch['stu_input_ids'])['hidden_states'][-1][sm]
|
| 85 |
+
th = batch['teacher_hiddens']
|
| 86 |
+
sy, ty = batch['stu_input_ids'].roll(-1, 1), batch['tea_input_ids'].roll(-1, 1)
|
| 87 |
+
total, so, to = sh.sum() * 0.0, 0, 0
|
| 88 |
+
for row in range(sm.shape[0]):
|
| 89 |
+
sid, tid = sy[row][sm[row]].tolist(), ty[row][tm[row]].tolist()
|
| 90 |
+
atoms = algo.atomizer.atomize(sid, tid, sample_id='original')
|
| 91 |
+
self.assertTrue(atoms.valid)
|
| 92 |
+
ns, nt = atoms.covered_student_events, atoms.covered_teacher_events
|
| 93 |
+
sl = student.model.lm_head(sh[so:so+ns], skip=False)
|
| 94 |
+
tl = teacher(th[to:to+nt]).detach()
|
| 95 |
+
if algo.student_cap is not None:
|
| 96 |
+
sl = torch.tanh(sl / algo.student_cap) * algo.student_cap
|
| 97 |
+
tl = torch.tanh(tl / algo.teacher_cap) * algo.teacher_cap
|
| 98 |
+
labels, tea_labels = sy[row][sm[row]][:ns], ty[row][tm[row]][:nt]
|
| 99 |
+
credits = build_atom_credits(atoms.atoms, sl, labels, tl, tea_labels)
|
| 100 |
+
loss, _ = scope['_align_loss'](reference, credits, atoms.atoms,
|
| 101 |
+
student_logits=sl, student_labels=labels, student_hidden=sh[so:so+ns],
|
| 102 |
+
student_ids=sid[:ns], teacher_ids=tid[:nt])
|
| 103 |
+
total = total + loss
|
| 104 |
+
so, to = so + len(sid), to + len(tid)
|
| 105 |
+
return total / batch['avg_micro_batch_token_num']
|
| 106 |
+
|
| 107 |
+
def test_full_adapter_loss_and_gradients_match_original_composition(self):
|
| 108 |
+
for source, cap, rho in [('run', None, .1), ('run', 3., .1), ('xtoken', None, .1),
|
| 109 |
+
('xtoken', 3., .1), ('run', None, 0.), ('run', None, 1.)]:
|
| 110 |
+
with self.subTest(source=source, cap=cap, rho=rho):
|
| 111 |
+
helper, algo, student, teacher, scope, reference = self.setup_pair(source=source, cap=cap, rho=rho)
|
| 112 |
+
rs, rt = copy.deepcopy(student), copy.deepcopy(teacher)
|
| 113 |
+
batch = helper.batch(copies=2)
|
| 114 |
+
actual = algo.training_step(batch)['train/loss']
|
| 115 |
+
expected = self.reference_loss(algo, rs, rt, batch, scope, reference)
|
| 116 |
+
ga = torch.autograd.grad(actual, tuple(student.parameters()))
|
| 117 |
+
gb = torch.autograd.grad(expected, tuple(rs.parameters()))
|
| 118 |
+
torch.testing.assert_close(actual, expected, atol=1e-6, rtol=1e-5)
|
| 119 |
+
for a, b in zip(ga, gb):
|
| 120 |
+
torch.testing.assert_close(a, b, atol=1e-6, rtol=1e-5)
|
| 121 |
+
self.errors.append((float((actual-expected).detach().abs()),
|
| 122 |
+
max(float((a-b).abs().max()) for a, b in zip(ga, gb))))
|
| 123 |
+
teacher_grads = torch.autograd.grad(algo.training_step(batch)['train/loss'],
|
| 124 |
+
(*teacher.parameters(), batch['teacher_hiddens']), allow_unused=True)
|
| 125 |
+
self.assertTrue(all(g is None for g in teacher_grads))
|
| 126 |
+
|
| 127 |
+
def test_active_global_guard_matches_original_loss_and_head_gradient(self):
|
| 128 |
+
from kdflow.algorithms._mp_opd_port._mp_opd_credit import AtomCreditTensors
|
| 129 |
+
_, algo, student, _, scope, ref = self.setup_pair()
|
| 130 |
+
ref.grass_chunk_run_length = 2
|
| 131 |
+
with torch.no_grad():
|
| 132 |
+
student.model.lm_head.weight.zero_()
|
| 133 |
+
hidden = torch.tensor([[1.,0.,0.,0.],[-1.,1.,0.,0.],[-.4,-1.,0.,0.]], requires_grad=True)
|
| 134 |
+
logits = student.model.lm_head(hidden, skip=False)
|
| 135 |
+
labels = torch.ones(3, dtype=torch.long)
|
| 136 |
+
nll = -logits.log_softmax(-1)[:, 1]
|
| 137 |
+
rate = torch.ones(3)
|
| 138 |
+
credits = AtomCreditTensors(-nll.detach()+1, -nll.detach(), rate, rate, rate, nll, nll)
|
| 139 |
+
atoms = tuple(types.SimpleNamespace(student_start=i, student_end=i+1) for i in range(3))
|
| 140 |
+
effective, metrics = algo._effective_rates(credits, atoms, ((0,2),(2,3)), hidden, labels)
|
| 141 |
+
actual = (effective*nll).sum()
|
| 142 |
+
expected, donor_metrics = scope['_align_loss'](ref, credits, atoms, student_logits=logits,
|
| 143 |
+
student_labels=labels, student_hidden=hidden, student_ids=[1]*3, teacher_ids=[1]*3)
|
| 144 |
+
torch.testing.assert_close(actual, expected)
|
| 145 |
+
ga = torch.autograd.grad(actual, student.model.lm_head.weight, retain_graph=True)[0]
|
| 146 |
+
gb = torch.autograd.grad(expected, student.model.lm_head.weight)[0]
|
| 147 |
+
torch.testing.assert_close(ga, gb)
|
| 148 |
+
beta = float(metrics['mp_opd_align_new_beta'])
|
| 149 |
+
self.assertGreater(beta, 0)
|
| 150 |
+
self.assertLess(beta, 1)
|
| 151 |
+
self.assertGreaterEqual(float(metrics['mp_opd_align_new_retained_descent_fraction']), .1-1e-6)
|
| 152 |
+
torch.testing.assert_close(metrics['mp_opd_align_new_beta'], donor_metrics['mp_opd_align_new_beta'])
|
| 153 |
+
self.errors.append((float((actual-expected).detach().abs()), float((ga-gb).abs().max())))
|
| 154 |
+
|
| 155 |
+
def test_full_head_pair_includes_cross_chunk_terms_bias_and_softcap(self):
|
| 156 |
+
from kdflow.algorithms._mp_opd_port._mp_opd_head_pair import head_pair_geometry
|
| 157 |
+
torch.manual_seed(123)
|
| 158 |
+
hidden = torch.randn(5,3)
|
| 159 |
+
weight = torch.randn(7,3, requires_grad=True)
|
| 160 |
+
bias = torch.randn(7, requires_grad=True)
|
| 161 |
+
labels = torch.tensor([0,1,2,3,4])
|
| 162 |
+
ranges = ((0,2),(2,3),(3,5))
|
| 163 |
+
first, second = torch.tensor([1.,-2.,.3]), torch.tensor([-.4,1.,2.])
|
| 164 |
+
raw = hidden @ weight.T + bias
|
| 165 |
+
logits = torch.tanh(raw / 2.) * 2.
|
| 166 |
+
lp = logits.log_softmax(-1).gather(1, labels[:,None]).squeeze(1)
|
| 167 |
+
sums = torch.stack([-lp[a:b].sum() for a,b in ranges])
|
| 168 |
+
ga = torch.cat([g.flatten() for g in torch.autograd.grad((first*sums).sum(), (weight,bias), retain_graph=True)])
|
| 169 |
+
gb = torch.cat([g.flatten() for g in torch.autograd.grad((second*sums).sum(), (weight,bias))])
|
| 170 |
+
expected = (float(ga.double().square().sum()), float(gb.double().square().sum()), float((ga.double()*gb.double()).sum()))
|
| 171 |
+
for tile in (1,3,4096):
|
| 172 |
+
pair = head_pair_geometry(logits.detach(), hidden, labels, ranges, first, second,
|
| 173 |
+
selected_log_prob=lp.detach(), softcap=2., head_bias=True, vocab_chunk=tile)
|
| 174 |
+
torch.testing.assert_close(torch.tensor([pair.first_norm2,pair.second_norm2,pair.dot]), torch.tensor(expected), atol=1e-5, rtol=1e-5)
|
| 175 |
+
|
| 176 |
+
def test_microbatch_invariance_and_no_state_collision_with_v1(self):
|
| 177 |
+
helper, algo, student, _, _, _ = self.setup_pair()
|
| 178 |
+
full, one = helper.batch(2), helper.batch()
|
| 179 |
+
one['avg_micro_batch_token_num'] *= 2
|
| 180 |
+
a = algo.training_step(full)['train/loss']
|
| 181 |
+
b = algo.training_step(one)['train/loss'] + algo.training_step(one)['train/loss']
|
| 182 |
+
torch.testing.assert_close(a,b)
|
| 183 |
+
for x,y in zip(torch.autograd.grad(a, tuple(student.parameters())), torch.autograd.grad(b, tuple(student.parameters()))):
|
| 184 |
+
torch.testing.assert_close(x,y)
|
| 185 |
+
old, _, _ = helper.fixture('mp_opd_align')
|
| 186 |
+
with self.assertRaises(ValueError):
|
| 187 |
+
algo.load_state_dict(old.state_dict())
|
| 188 |
+
|
| 189 |
+
def test_ambient_autocast_uses_same_credit_logits_as_original(self):
|
| 190 |
+
from kdflow.algorithms._mp_opd_port._mp_opd_credit import build_atom_credits
|
| 191 |
+
helper, algo, student, teacher, scope, ref = self.setup_pair()
|
| 192 |
+
batch = helper.batch()
|
| 193 |
+
sy = batch['stu_input_ids'].roll(-1,1)[batch['stu_loss_mask']]
|
| 194 |
+
ty = batch['tea_input_ids'].roll(-1,1)[batch['tea_loss_mask']]
|
| 195 |
+
atomized = algo.atomizer.atomize(sy.tolist(), ty.tolist(), sample_id='autocast')
|
| 196 |
+
ns,nt = atomized.covered_student_events,atomized.covered_teacher_events
|
| 197 |
+
hidden = student(batch['stu_input_ids'])['hidden_states'][-1][batch['stu_loss_mask']][:ns]
|
| 198 |
+
with torch.autocast(device_type='cpu',dtype=torch.bfloat16):
|
| 199 |
+
logits = student.model.lm_head(hidden,skip=False)
|
| 200 |
+
teacher_logits = teacher(batch['teacher_hiddens'][:nt])
|
| 201 |
+
credits = build_atom_credits(atomized.atoms,logits,sy[:ns],teacher_logits,ty[:nt])
|
| 202 |
+
partition,_,_ = algo._partition(atomized.atoms,sy[:ns].tolist(),ty[:nt].tolist())
|
| 203 |
+
effective,metrics = algo._effective_rates(credits,atomized.atoms,partition,hidden,sy[:ns])
|
| 204 |
+
actual = (credits.current_nll*effective).sum()
|
| 205 |
+
expected,donor_metrics = scope['_align_loss'](ref,credits,atomized.atoms,
|
| 206 |
+
student_logits=logits,student_labels=sy[:ns],student_hidden=hidden,
|
| 207 |
+
student_ids=sy[:ns].tolist(),teacher_ids=ty[:nt].tolist())
|
| 208 |
+
torch.testing.assert_close(actual,expected,atol=1e-6,rtol=1e-5)
|
| 209 |
+
for key in ('atomic_head_norm2','candidate_head_norm2','candidate_atomic_dot','beta'):
|
| 210 |
+
name = 'mp_opd_align_new_'+key
|
| 211 |
+
torch.testing.assert_close(metrics[name],donor_metrics[name],atol=1e-5,rtol=1e-5)
|
| 212 |
+
|
| 213 |
+
def test_guard_edge_cases_and_invalid_geometry_fail_closed(self):
|
| 214 |
+
from kdflow.algorithms._mp_opd_port._mp_opd_align_new import align_new_guard
|
| 215 |
+
rate, candidate = torch.tensor([1.,2.]), torch.tensor([3.,4.])
|
| 216 |
+
for a,b,d in [(0.,1.,0.), (1.,1.,1.), (1.,1.,-1.)]:
|
| 217 |
+
actual, metrics = align_new_guard(rate,candidate,a,b,d)
|
| 218 |
+
expected, ref_metrics = original_scope()['align_new_guard'](rate,candidate,a,b,d)
|
| 219 |
+
torch.testing.assert_close(actual,expected)
|
| 220 |
+
torch.testing.assert_close(metrics['mp_opd_align_new_beta'],ref_metrics['mp_opd_align_new_beta'])
|
| 221 |
+
for a,b,d in [(1.,1.,2.), (-1.,1.,0.), (float('nan'),1.,0.)]:
|
| 222 |
+
with self.assertRaises((ValueError,FloatingPointError)):
|
| 223 |
+
align_new_guard(rate,candidate,a,b,d)
|
| 224 |
+
|
| 225 |
+
def test_argument_guards_and_registry(self):
|
| 226 |
+
from kdflow.arguments.distillation_args import DistillationArguments
|
| 227 |
+
from kdflow.algorithms import ALGO_DICT
|
| 228 |
+
opts = dict(kd_algorithm='mp_opd_align_new', kd_ratio=1., kd_loss_fn='rkl')
|
| 229 |
+
self.assertIn('mp_opd_align_new', ALGO_DICT)
|
| 230 |
+
self.assertEqual(DistillationArguments(**opts).mp_opd_align_new_min_descent_fraction, .1)
|
| 231 |
+
for kwargs in ({'mp_opd_grass_geometry':'diag'}, {'mp_opd_align_new_eps_g':-1},
|
| 232 |
+
{'mp_opd_align_new_min_descent_fraction':1.1}, {'mp_opd_align_new_eps_g':float('nan')}):
|
| 233 |
+
with self.assertRaises(ValueError):
|
| 234 |
+
DistillationArguments(**opts, **kwargs)
|
| 235 |
+
|
| 236 |
+
|
| 237 |
+
class AlignNewSourceTests(unittest.TestCase):
|
| 238 |
+
def test_source_hashes_and_unchanged_existing_algorithms(self):
|
| 239 |
+
manifest = json.loads((ROOT / 'docs/ALIGN_NEW_SOURCE.json').read_text())
|
| 240 |
+
for row in manifest['files'] + manifest['unchanged_existing_files']:
|
| 241 |
+
self.assertEqual(hashlib.sha256((ROOT/row['path']).read_bytes()).hexdigest(), row['sha256'])
|
| 242 |
+
|
| 243 |
+
|
| 244 |
+
if __name__ == '__main__':
|
| 245 |
+
unittest.main()
|
tests/test_single_b200.py
CHANGED
|
@@ -87,9 +87,20 @@ class LauncherTests(unittest.TestCase):
|
|
| 87 |
for path in (ROOT / 'kdflow/arguments').glob('*_args.py'):
|
| 88 |
fields |= {node.target.id for node in ast.walk(ast.parse(path.read_text()))
|
| 89 |
if isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name)}
|
| 90 |
-
for name in ['mp-atomic', 'mp-fixed5', 'mp-random2-5', 'mp-grass-chunk', 'mp-align']:
|
| 91 |
self.assertFalse(set(self.plan('--algorithm', name)['flags']) - fields)
|
| 92 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 93 |
def test_dry_run_creates_no_files(self):
|
| 94 |
with tempfile.TemporaryDirectory() as folder:
|
| 95 |
destination = pathlib.Path(folder) / 'absent'
|
|
|
|
| 87 |
for path in (ROOT / 'kdflow/arguments').glob('*_args.py'):
|
| 88 |
fields |= {node.target.id for node in ast.walk(ast.parse(path.read_text()))
|
| 89 |
if isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name)}
|
| 90 |
+
for name in ['mp-atomic', 'mp-fixed5', 'mp-random2-5', 'mp-grass-chunk', 'mp-align', 'mp-align-new']:
|
| 91 |
self.assertFalse(set(self.plan('--algorithm', name)['flags']) - fields)
|
| 92 |
|
| 93 |
+
def test_align_new_preset_and_exact_head_guard(self):
|
| 94 |
+
flags = self.plan('--algorithm', 'mp-align-new')['flags']
|
| 95 |
+
self.assertEqual(flags['kd_algorithm'], 'mp_opd_align_new')
|
| 96 |
+
self.assertEqual(flags['mp_opd_align_new_min_descent_fraction'], .1)
|
| 97 |
+
self.assertEqual(flags['mp_opd_align_new_eps_g'], 1e-12)
|
| 98 |
+
self.assertEqual(flags['mp_opd_chunk_source'], 'xtoken')
|
| 99 |
+
for key in ('train_batch_size','rollout_batch_size','max_steps','generate_max_len','eval_steps','eval_math_n_samples'):
|
| 100 |
+
self.assertEqual(flags[key], self.plan()['flags'][key])
|
| 101 |
+
with self.assertRaisesRegex(ValueError, 'exact_head'):
|
| 102 |
+
self.plan('--algorithm','mp-align-new','--mp-geometry','diag')
|
| 103 |
+
|
| 104 |
def test_dry_run_creates_no_files(self):
|
| 105 |
with tempfile.TemporaryDirectory() as folder:
|
| 106 |
destination = pathlib.Path(folder) / 'absent'
|
tools/launch_single_b200.py
CHANGED
|
@@ -27,7 +27,7 @@ def parser():
|
|
| 27 |
help='Installed prefix; default CTOPD_RUNTIME_PREFIX or /opt/ctopd-b200')
|
| 28 |
p.add_argument('--algorithm', choices=['strict-top16', 'strict-full', 'simct', 'mp-atomic',
|
| 29 |
'mp-fixed1', 'mp-fixed2', 'mp-fixed3', 'mp-fixed4', 'mp-fixed5',
|
| 30 |
-
'mp-random1-2', 'mp-random2-5', 'mp-grass-chunk', 'mp-align'], default='strict-top16')
|
| 31 |
p.add_argument('--mp-chunk-source', choices=['xtoken', 'run'], default='xtoken')
|
| 32 |
p.add_argument('--mp-chunk-run-length', type=int, default=2)
|
| 33 |
p.add_argument('--mp-geometry', choices=['exact_head', 'diag'], default='exact_head')
|
|
@@ -42,6 +42,8 @@ def parser():
|
|
| 42 |
|
| 43 |
|
| 44 |
def build_plan(args):
|
|
|
|
|
|
|
| 45 |
if args.concurrency < 1 or args.eval_timeout < 0 or args.mp_chunk_run_length < 1:
|
| 46 |
raise ValueError('Concurrency must be positive and eval timeout nonnegative')
|
| 47 |
run_id = args.run_id or (datetime.datetime.now(datetime.timezone.utc).strftime('%Y%m%dT%H%M%S%fZ')
|
|
@@ -92,10 +94,12 @@ def build_plan(args):
|
|
| 92 |
low, high = map(int, name[len('random'):].split('-'))
|
| 93 |
flags.update(mp_opd_random_min_length=low, mp_opd_random_max_length=high,
|
| 94 |
mp_opd_random_seed=args.seed)
|
| 95 |
-
if family in {'grass_chunk', 'align'}:
|
| 96 |
flags.update(mp_opd_chunk_source=args.mp_chunk_source,
|
| 97 |
mp_opd_chunk_run_length=args.mp_chunk_run_length,
|
| 98 |
mp_opd_grass_geometry=args.mp_geometry)
|
|
|
|
|
|
|
| 99 |
command = ['bash', str(runtime / 'python-b200.sh'), '-u', '-m', 'kdflow.cli.train_kd_on_policy']
|
| 100 |
for key, value in flags.items():
|
| 101 |
command.extend(['--' + key, str(value)])
|
|
|
|
| 27 |
help='Installed prefix; default CTOPD_RUNTIME_PREFIX or /opt/ctopd-b200')
|
| 28 |
p.add_argument('--algorithm', choices=['strict-top16', 'strict-full', 'simct', 'mp-atomic',
|
| 29 |
'mp-fixed1', 'mp-fixed2', 'mp-fixed3', 'mp-fixed4', 'mp-fixed5',
|
| 30 |
+
'mp-random1-2', 'mp-random2-5', 'mp-grass-chunk', 'mp-align', 'mp-align-new'], default='strict-top16')
|
| 31 |
p.add_argument('--mp-chunk-source', choices=['xtoken', 'run'], default='xtoken')
|
| 32 |
p.add_argument('--mp-chunk-run-length', type=int, default=2)
|
| 33 |
p.add_argument('--mp-geometry', choices=['exact_head', 'diag'], default='exact_head')
|
|
|
|
| 42 |
|
| 43 |
|
| 44 |
def build_plan(args):
|
| 45 |
+
if args.algorithm == 'mp-align-new' and args.mp_geometry != 'exact_head':
|
| 46 |
+
raise ValueError('ALIGN-NEW requires exact_head geometry')
|
| 47 |
if args.concurrency < 1 or args.eval_timeout < 0 or args.mp_chunk_run_length < 1:
|
| 48 |
raise ValueError('Concurrency must be positive and eval timeout nonnegative')
|
| 49 |
run_id = args.run_id or (datetime.datetime.now(datetime.timezone.utc).strftime('%Y%m%dT%H%M%S%fZ')
|
|
|
|
| 94 |
low, high = map(int, name[len('random'):].split('-'))
|
| 95 |
flags.update(mp_opd_random_min_length=low, mp_opd_random_max_length=high,
|
| 96 |
mp_opd_random_seed=args.seed)
|
| 97 |
+
if family in {'grass_chunk', 'align', 'align_new'}:
|
| 98 |
flags.update(mp_opd_chunk_source=args.mp_chunk_source,
|
| 99 |
mp_opd_chunk_run_length=args.mp_chunk_run_length,
|
| 100 |
mp_opd_grass_geometry=args.mp_geometry)
|
| 101 |
+
if family == 'align_new':
|
| 102 |
+
flags.update(mp_opd_align_new_min_descent_fraction=0.1, mp_opd_align_new_eps_g=1e-12)
|
| 103 |
command = ['bash', str(runtime / 'python-b200.sh'), '-u', '-m', 'kdflow.cli.train_kd_on_policy']
|
| 104 |
for key, value in flags.items():
|
| 105 |
command.extend(['--' + key, str(value)])
|