codemaivanngu commited on
Commit
43eefa6
·
unverified ·
1 Parent(s): 87e4791

feat(simct): add isolated author baseline with loss gradient parity

Browse files
README.md CHANGED
@@ -17,6 +17,10 @@ GRASS-Chunk and ALIGN as separate modes. The existing Strict algorithm sources
17
  remain unchanged. The new modes use sampled atom credits rather than Strict KL;
18
  they have CPU verification, with B200/full training qualification still pending.
19
 
 
 
 
 
20
  ## Original project documentation
21
 
22
  This repository provides the implementation and example training scripts for `byte_align_ctkd` and `topk_byte_align_ctkd`.
 
17
  remain unchanged. The new modes use sampled atom credits rather than Strict KL;
18
  they have CPU verification, with B200/full training qualification still pending.
19
 
20
+ A separate [author-code SimCT baseline](docs/SIMCT.md) uses `--algorithm simct`
21
+ in the same launcher. It preserves the pinned released implementation, with
22
+ original-source loss/gradient parity tests and an explicit paper/code distinction.
23
+
24
  ## Original project documentation
25
 
26
  This repository provides the implementation and example training scripts for `byte_align_ctkd` and `topk_byte_align_ctkd`.
docs/SIMCT.md ADDED
@@ -0,0 +1,100 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Author-code SimCT in the common CTOPD pipeline
2
+
3
+ Select `simct` to run the author's released `span_ctkd` objective. This is separate
4
+ from Strict and the MP-OPD modes: `mp-atomic`/`mp-fixed1` are not SimCT.
5
+
6
+ ## Source and exact scope
7
+
8
+ The source is [sunjie279/SimCT-](https://github.com/sunjie279/SimCT-/tree/cf0f33a0e6c967d4b74ea32b2dba12be01b73b9e),
9
+ commit `cf0f33a0e6c967d4b74ea32b2dba12be01b73b9e`. The author's released
10
+ `scripts/ctopd/qwen25_gemma2_span_mix10k_lr5e-7.sh` selects `span_ctkd` with unit
11
+ RKL. Locally the public mode name is `simct`; the donor registration is isolated
12
+ and does not register a second legacy alias. The vendor class differs only by
13
+ relocating the registration import. Original source fixtures, SHA-256 hashes and
14
+ the MIT notice are retained; see [SIMCT_SOURCE.json](SIMCT_SOURCE.json).
15
+
16
+ The adapter changes the execution interface from the author's full student
17
+ logits to CTOPD hidden states plus the student LM head, preserves student final
18
+ softcapping, and translates metrics into CTOPD's `train/*` / `distill/*` names.
19
+ The author's teacher uses the raw linear head, which is preserved here. It does
20
+ not inherit the MP-OPD adapter. Per-response alignment, shared-vocabulary mapping,
21
+ span construction, EOS inclusion, unmatched suffix handling, virtual vocabulary
22
+ and optional teacher-mass masking all use the pinned author's functions.
23
+
24
+ The adapter processes one response at a time and checkpoints the student head.
25
+ Empty alignment gives a connected zero loss. Nonfinite loss, invalid hidden row
26
+ counts and invalid token normalizers fail explicitly. Debug token/logit dumps
27
+ are suppressed. Unit RKL, no CE mixing, on-policy KD, one student rank and ring
28
+ size 1 are the supported contract. Other configurations fail before launch.
29
+
30
+ ## Important distinction: released code versus paper equations
31
+
32
+ The [paper's equations 7-8](https://arxiv.org/html/2605.07711v1#S3.SS3) describe
33
+ length-normalized continuation **log-probability** scores. The released class
34
+ `_build_virtual_vocab_logits` instead averages selected **raw logits** for each
35
+ span and appends that value to the raw shared-vocabulary logits. Its comments
36
+ claim a geometric-mean probability interpretation, but position-dependent
37
+ full-vocabulary normalizers are absent from that computation. These two formulas
38
+ are generally different, including under additive logit shifts at later span
39
+ positions. This port preserves the released code; it does not silently replace
40
+ it with a reconstructed paper objective.
41
+
42
+ The released alignment is greedy over independently decoded token strings. It
43
+ is not the MP-OPD byte-checked atomizer, and may leave an unmatched suffix
44
+ unsupervised. These semantics are retained for source parity. This port is not
45
+ proof of paper/code equivalence, nor of parity with a different KDFlow revision
46
+ whose algorithm is also named `simct`.
47
+
48
+ ## Launch
49
+
50
+ Inside this checkout on an allocated company node:
51
+
52
+ ```bash
53
+ git pull --ff-only
54
+ python3 tools/launch_single_b200.py --algorithm simct \
55
+ --runtime /opt/ctopd-b200-u22 --dry-run
56
+ ```
57
+
58
+ The standard library dry-run uses no GPU and creates no run. Remove `--dry-run`
59
+ only when ready for the real allocated single-B200 workload, with the assigned
60
+ GPU exposed through `CUDA_VISIBLE_DEVICES` and all models/data present. Use a new
61
+ run ID. No runtime rebuild or new dependency is needed for this source port.
62
+
63
+ The preset changes only the algorithm and adds `simct_gh_mask_threshold=0`
64
+ (disabled, as in the released author launchers). The adapter maps this isolated
65
+ flag to the author's optional `span_gh_mask_threshold` semantics. Direct KDFlow
66
+ CLI users may set `--simct_gh_mask_threshold` to a positive finite number;
67
+ zero disables it. Current launch presets do not enable it.
68
+
69
+ The common CTOPD recipe retains Qwen-to-Llama, the existing student initialization,
70
+ 20k prompt source, optimizer batch 512, 100 steps, generation limit 8192, optimizer
71
+ and evaluation settings. It does not add SimCT's teacher-generated 10k SFT warmup
72
+ or its original model pairs. Compare all methods from the same checkpoint under
73
+ this common recipe; label the result a common-pipeline comparison, not a
74
+ reproduction of SimCT's published table. The launcher currently evaluates MATH500
75
+ only, not the paper's entire benchmark suite.
76
+
77
+ ## Verification
78
+
79
+ Using an existing compatible CPU PyTorch environment:
80
+
81
+ ```bash
82
+ TORCH_COMPILE_DISABLE=1 OMP_NUM_THREADS=1 python -m unittest discover -s tests -v
83
+ ```
84
+
85
+ The oracle runs the original saved author class and original RKL numerical bodies,
86
+ without importing the adapter or CTOPD's RKL. Only import/decorator wiring is
87
+ removed to execute those original bodies in the test process. Compare mixed
88
+ spans, strict-only responses, optional G(h) masks, softcapping, loss, every student
89
+ gradient, and alignment/mask diagnostics. An independent hand construction checks
90
+ virtual logits and RKL gradients. Additional checks cover teacher stop-gradient,
91
+ response boundaries, shared microbatch normalization, empty alignment, argument
92
+ validation, launcher settings and unchanged existing numerical source hashes.
93
+
94
+ The [CPU validation receipt](SIMCT_VALIDATION.json) records the 56-test pass and
95
+ six original-source comparisons: maximum absolute loss error `0.0`, maximum
96
+ absolute student-gradient error `3.725290298461914e-09` in FP32 fixtures.
97
+
98
+ CPU checks do not qualify target Torch/CUDA, B200/FSDP integration, full-response
99
+ memory/throughput or downstream accuracy. Per-response virtual-vocabulary logits
100
+ can still be large. No GPU training is launched by the port or tests.
docs/SIMCT_SOURCE.json ADDED
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "source_repository": "https://github.com/sunjie279/SimCT-",
3
+ "source_commit": "cf0f33a0e6c967d4b74ea32b2dba12be01b73b9e",
4
+ "author_launcher_algorithm": "span_ctkd",
5
+ "local_algorithm": "simct",
6
+ "files": [
7
+ {
8
+ "sha256": "a83bc75c4f297c7f13e9eac809647c8c9dc228b9a3fdf90eb413d30bfc94795a",
9
+ "path": "tests/fixtures/simct_upstream/span_ctkd.py.txt"
10
+ },
11
+ {
12
+ "sha256": "abca69fdbe4f7d90a7ca990300189147230cce8f20d589c934d88f0b63c4edf6",
13
+ "path": "tests/fixtures/simct_upstream/reverse_kl_div.py.txt"
14
+ },
15
+ {
16
+ "sha256": "64bd78c0a991dcaa8e438c0d96dd61d3aacabf136c5194816c0e9e349f407e49",
17
+ "path": "kdflow/algorithms/_simct_upstream/span_ctkd.py"
18
+ },
19
+ {
20
+ "sha256": "c9f18bd09f6b760b2ceb9826447d56fd72b272ee318fb2eb48be20a68210b3be",
21
+ "path": "kdflow/algorithms/_simct_upstream/LICENSE.MIT.txt"
22
+ }
23
+ ],
24
+ "unchanged_existing_files": [
25
+ {
26
+ "sha256": "fdbd99c55aeac4d061527158bdfc82a93333a42bf347eceaa134d34f5ef05952",
27
+ "path": "kdflow/algorithms/simple_ctkd.py"
28
+ },
29
+ {
30
+ "sha256": "5c6647c6fb8064ffd5f7b1ed971168cd1f3392caee62e2d75916d41931828e18",
31
+ "path": "kdflow/algorithms/byte_align_ctkd.py"
32
+ },
33
+ {
34
+ "sha256": "298925da4ce973b0f2917e5ba5ad2adbc4862d7c1a73515c5c86514ac0d22ddb",
35
+ "path": "kdflow/algorithms/topk_byte_align_ctkd.py"
36
+ },
37
+ {
38
+ "sha256": "b2387d7f36502dea16c28f4391af4bc97a7e77f367d35dea2867c744e91277a6",
39
+ "path": "kdflow/algorithms/mp_opd.py"
40
+ },
41
+ {
42
+ "sha256": "abca69fdbe4f7d90a7ca990300189147230cce8f20d589c934d88f0b63c4edf6",
43
+ "path": "kdflow/loss/reverse_kl_div.py"
44
+ },
45
+ {
46
+ "sha256": "151c81614f97b908465a952f595abe1f4dc5c0d17cea86431a7068c381125cc5",
47
+ "path": "kdflow/loss/chunked_loss.py"
48
+ },
49
+ {
50
+ "sha256": "c77cd962cb0430cd97ace5312077e847398a03da3ce1473062fd2c5c9808498a",
51
+ "path": "kdflow/token_alignment.py"
52
+ },
53
+ {
54
+ "sha256": "97a8495d68da45c05aa87ba8d87fe6db6dd520cb15de4352cb2ea0e927ddb076",
55
+ "path": "kdflow/algorithms/_mp_opd_port/_mp_opd_grass_chunk.py"
56
+ },
57
+ {
58
+ "sha256": "c6c0286c74215a8d30888c15b2bb5820ab77f54a45afd832db99ca6eb81cf1e0",
59
+ "path": "kdflow/algorithms/_mp_opd_port/_mp_opd_align.py"
60
+ }
61
+ ],
62
+ "changes": "Only register_algorithm import relocated in vendored class. Adapter uses hidden states and raw heads, per-response execution, CTOPD metric names, unit-RKL/no-CE contract, no token debug logging. See SIMCT.md for paper versus code distinction."
63
+ }
docs/SIMCT_VALIDATION.json ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "date": "2026-10-19",
3
+ "source_commit": "cf0f33a0e6c967d4b74ea32b2dba12be01b73b9e",
4
+ "platform": "WSL Linux, CPU only",
5
+ "python": "3.12.3",
6
+ "torch": "2.14.0+cpu",
7
+ "command": "TORCH_COMPILE_DISABLE=1 OMP_NUM_THREADS=1 python -m unittest discover -s tests -v",
8
+ "full_suite": {
9
+ "passed": 56,
10
+ "skipped": 0,
11
+ "elapsed_seconds": 91.714
12
+ },
13
+ "original_source_parity": {
14
+ "cases": 6,
15
+ "max_loss_abs_error": 0.0,
16
+ "max_student_gradient_abs_error": 3.725290298461914e-09,
17
+ "atol": 1e-6,
18
+ "rtol": 1e-5,
19
+ "dtype": "float32",
20
+ "coverage": "Mixed spans with two responses, strict-only, student softcap, G(h) disabled and three positive thresholds",
21
+ "oracle": "Pinned original class and original RKL numerical bodies; only imports/decorators removed for isolated execution"
22
+ },
23
+ "additional_checks": [
24
+ "Independent hand-constructed virtual logits, RKL and gradients",
25
+ "No teacher parameter or hidden-state gradient",
26
+ "Response boundaries and shared microbatch normalization",
27
+ "Empty alignment, invalid hidden rows, arguments and topology guards",
28
+ "Existing Strict/MP source hashes and numerical regression tests",
29
+ "SimCT dry-run retains batch512, steps100, generation8192 and common optimizer/evaluation settings",
30
+ "Staged diff whitespace check excluding the verbatim original RKL fixture, which retains three upstream trailing-whitespace lines",
31
+ "Source hashes rechecked after renaming the MIT notice to enforce LF through existing .gitattributes"
32
+ ],
33
+ "not_qualified": [
34
+ "Paper-equation equivalence: released code averages raw logits rather than log-probabilities",
35
+ "Parity with other KDFlow implementations named simct",
36
+ "Target Torch 2.11 CUDA, B200/FSDP execution and full-size memory/throughput",
37
+ "Full training, published scores and the original SFT warmup recipe"
38
+ ]
39
+ }
docs/SINGLE_B200.md CHANGED
@@ -4,6 +4,8 @@
4
 
5
  Separate experimental MP-OPD presets (Atomic, Fixed1-5, Random1-2/2-5, GRASS-Chunk and ALIGN) are documented in [MP_OPD.md](MP_OPD.md). They retain the workload below but change the objective; they are not Strict reproduction modes. Their CPU checks do not qualify B200/FSDP execution or full-training memory requirements.
6
 
 
 
7
  ## Preserved training and evaluation settings
8
 
9
  | Setting | Value |
 
4
 
5
  Separate experimental MP-OPD presets (Atomic, Fixed1-5, Random1-2/2-5, GRASS-Chunk and ALIGN) are documented in [MP_OPD.md](MP_OPD.md). They retain the workload below but change the objective; they are not Strict reproduction modes. Their CPU checks do not qualify B200/FSDP execution or full-training memory requirements.
6
 
7
+ The separate [SimCT author-code preset](SIMCT.md), `--algorithm simct`, uses the same common workload. It adds no SFT warmup; it is not the original SimCT end-to-end recipe.
8
+
9
  ## Preserved training and evaluation settings
10
 
11
  | Setting | Value |
kdflow/algorithms/__init__.py CHANGED
@@ -13,6 +13,7 @@ def register_algorithm(name):
13
  from kdflow.algorithms.simple_ctkd import SimpleCrossTokenizerKD
14
  from kdflow.algorithms.byte_align_ctkd import ByteAlignCrossTokenizerKD
15
  from kdflow.algorithms.topk_byte_align_ctkd import TopKByteAlignCrossTokenizerKD
 
16
  from kdflow.algorithms.mp_opd import (
17
  AtomicMPOnPolicyKD, FixedMPOnPolicyKD, RandomMPOnPolicyKD,
18
  GrassChunkMPOnPolicyKD, AlignMPOnPolicyKD,
 
13
  from kdflow.algorithms.simple_ctkd import SimpleCrossTokenizerKD
14
  from kdflow.algorithms.byte_align_ctkd import ByteAlignCrossTokenizerKD
15
  from kdflow.algorithms.topk_byte_align_ctkd import TopKByteAlignCrossTokenizerKD
16
+ from kdflow.algorithms.simct import SimCT
17
  from kdflow.algorithms.mp_opd import (
18
  AtomicMPOnPolicyKD, FixedMPOnPolicyKD, RandomMPOnPolicyKD,
19
  GrassChunkMPOnPolicyKD, AlignMPOnPolicyKD,
kdflow/algorithms/_simct_upstream/LICENSE.MIT.txt ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Songming Zhang
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
kdflow/algorithms/_simct_upstream/__init__.py ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ """Pinned author's SimCT code; never register its legacy name in KDFlow."""
2
+
3
+
4
+ def register_algorithm(_name):
5
+ return lambda cls: cls
kdflow/algorithms/_simct_upstream/span_ctkd.py ADDED
@@ -0,0 +1,508 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import torch
2
+ import torch.nn.functional as F
3
+
4
+ from kdflow.loss import build_loss_fn
5
+ from . import register_algorithm
6
+ from kdflow.loss.cross_entropy import compute_cross_entropy
7
+ from kdflow.utils.logging_utils import init_logger
8
+
9
+ logger = init_logger(__name__)
10
+
11
+
12
+ @register_algorithm("span_ctkd")
13
+ class SpanCrossTokenizerKD:
14
+ """Span-based Cross-Tokenizer Knowledge Distillation.
15
+
16
+ Improvement over SimpleCrossTokenizerKD: instead of discarding positions where
17
+ student and teacher tokens do not align 1:1, we group misaligned tokens into
18
+ "spans" and compute their logits as the arithmetic mean of the constituent
19
+ token logits (which corresponds to geometric mean after softmax).
20
+
21
+ These spans are appended to the shared (overlap) vocabulary to form a
22
+ "virtual common vocabulary". RKL alignment is then performed on this
23
+ extended vocabulary for every aligned position (both 1:1 tokens and spans).
24
+
25
+ For span segments, the overlap logits are taken from the **first** token
26
+ position only (not averaged across positions), because the span is treated
27
+ as a single virtual token occupying one position in the sequence.
28
+ """
29
+
30
+ def __init__(
31
+ self,
32
+ strategy,
33
+ student_model,
34
+ teacher_lm_head,
35
+ student_tokenizer,
36
+ teacher_tokenizer,
37
+ **kwargs,
38
+ ):
39
+ self.strategy = strategy
40
+ self.args = strategy.args
41
+ self.student = student_model
42
+ self.teacher_lm_head = teacher_lm_head
43
+ self.student_tokenizer = student_tokenizer
44
+ self.teacher_tokenizer = teacher_tokenizer
45
+ self.student_overlap_token_ids, self.teacher_overlap_token_ids = self._find_overlap_tokens()
46
+ self.loss_fn = build_loss_fn(self.args.kd.kd_loss_fn, self.args)
47
+ self._debug_count = 0 # Counter for debug logging
48
+ self._span_debug_count = 0 # Counter for span logits debug logging
49
+
50
+ # ------------------------------------------------------------------
51
+ # Overlap vocabulary (copied from simple_ctkd)
52
+ # ------------------------------------------------------------------
53
+ def _find_overlap_tokens(self):
54
+ """Find the overlap tokens between student and teacher tokenizer."""
55
+ student_vocab = {k.replace("Ġ", "▁"): v for k, v in self.student_tokenizer.get_vocab().items()}
56
+ teacher_vocab = {k.replace("Ġ", "▁"): v for k, v in self.teacher_tokenizer.get_vocab().items()}
57
+ overlap_tokens = set(student_vocab.keys()) & set(teacher_vocab.keys())
58
+ student_ids = [student_vocab[token] for token in overlap_tokens]
59
+ teacher_ids = [teacher_vocab[token] for token in overlap_tokens]
60
+ stu_eos, tea_eos = self.student_tokenizer.eos_token_id, self.teacher_tokenizer.eos_token_id
61
+ if stu_eos not in student_ids or tea_eos not in teacher_ids:
62
+ student_ids.append(stu_eos)
63
+ teacher_ids.append(tea_eos)
64
+ device = self.teacher_lm_head.weight.device
65
+ logger.info(f"[SpanCTKD] Num of overlap_tokens between student & teacher: {len(student_ids)}")
66
+ return (
67
+ torch.tensor(student_ids, dtype=torch.long, device=device),
68
+ torch.tensor(teacher_ids, dtype=torch.long, device=device),
69
+ )
70
+
71
+ # ------------------------------------------------------------------
72
+ # Sequence alignment with span identification
73
+ # (adapted from ALM._compute_chunk_alignment)
74
+ # ------------------------------------------------------------------
75
+ def _align_sequences_with_spans(self, tea_label_ids, stu_label_ids):
76
+ """Align teacher and student label tokens and identify spans.
77
+
78
+ Operates on **label ids** (i.e. input_ids shifted left by one position,
79
+ already filtered by loss_mask) so that alignment is consistent with the
80
+ next-token-prediction semantics of the logits.
81
+
82
+ Uses ``tokenizer.decode([tid])`` per token (O(n)) to correctly handle
83
+ all tokenizer conventions including newlines.
84
+
85
+ Args:
86
+ tea_label_ids: 1-D list of teacher label token ids (loss_mask positions).
87
+ stu_label_ids: 1-D list of student label token ids (loss_mask positions).
88
+
89
+ Returns:
90
+ segments: list of (tea_start, tea_end, stu_start, stu_end) where
91
+ indices are positions within the label id sequences. Each tuple
92
+ represents an aligned segment: teacher tokens ``[tea_start, tea_end)``
93
+ and student tokens ``[stu_start, stu_end)`` encode the same text.
94
+ When both ranges have length 1 it is a normal 1:1 alignment;
95
+ otherwise it is a span.
96
+ tea_label_ids_list: the teacher label ids as a Python list.
97
+ stu_label_ids_list: the student label ids as a Python list.
98
+ """
99
+ if len(tea_label_ids) == 0 or len(stu_label_ids) == 0:
100
+ return [], [], []
101
+
102
+ tea_ids_list = tea_label_ids if isinstance(tea_label_ids, list) else tea_label_ids.cpu().tolist()
103
+ stu_ids_list = stu_label_ids if isinstance(stu_label_ids, list) else stu_label_ids.cpu().tolist()
104
+
105
+ # Convert token ids to text strings for alignment.
106
+ # Use tokenizer.decode([tid]) per token to correctly handle all
107
+ # tokenizer conventions (e.g. Qwen uses Ċ for \n internally, which
108
+ # would NOT be normalized by simple strip of ▁/Ġ prefixes).
109
+ tea_token_texts = [self.teacher_tokenizer.decode([tid]) for tid in tea_ids_list]
110
+ stu_token_texts = [self.student_tokenizer.decode([tid]) for tid in stu_ids_list]
111
+
112
+ # Greedy cumulative-text alignment (same algorithm as simple_ctkd)
113
+ tea_eos = self.teacher_tokenizer.eos_token
114
+ stu_eos = self.student_tokenizer.eos_token
115
+
116
+ i, j = 0, 0
117
+ boundaries = []
118
+ history_tea = ""
119
+ history_stu = ""
120
+
121
+ while i < len(tea_token_texts) and j < len(stu_token_texts):
122
+ is_eos_match = (tea_token_texts[i] == tea_eos and stu_token_texts[j] == stu_eos)
123
+ if history_tea == history_stu and (
124
+ tea_token_texts[i] == stu_token_texts[j] or is_eos_match
125
+ ):
126
+ boundaries.append((i, j))
127
+ history_tea += tea_token_texts[i]
128
+ history_stu += stu_token_texts[j]
129
+ i += 1
130
+ j += 1
131
+ elif len(history_tea) > len(history_stu):
132
+ history_stu += stu_token_texts[j]
133
+ j += 1
134
+ elif len(history_tea) < len(history_stu):
135
+ history_tea += tea_token_texts[i]
136
+ i += 1
137
+ else:
138
+ history_tea += tea_token_texts[i]
139
+ history_stu += stu_token_texts[j]
140
+ i += 1
141
+ j += 1
142
+
143
+ if len(boundaries) == 0:
144
+ if self._debug_count < 3:
145
+ self._debug_count += 1
146
+ logger.warning(
147
+ f"[SpanCTKD DEBUG] No boundaries found! "
148
+ f"tea_token_texts({len(tea_token_texts)}): {tea_token_texts[:10]}... "
149
+ f"stu_token_texts({len(stu_token_texts)}): {stu_token_texts[:10]}..."
150
+ )
151
+ return [], tea_ids_list, stu_ids_list
152
+
153
+ if self._debug_count < 3:
154
+ self._debug_count += 1
155
+ logger.info(
156
+ f"[SpanCTKD DEBUG] Found {len(boundaries)} boundaries! "
157
+ f"tea_texts({len(tea_token_texts)}): {tea_token_texts[:5]}... "
158
+ f"stu_texts({len(stu_token_texts)}): {stu_token_texts[:5]}..."
159
+ )
160
+
161
+ # Convert boundaries to aligned segments (local indices within response)
162
+ segments = []
163
+ for idx in range(len(boundaries)):
164
+ if idx == 0:
165
+ local_tea_start, local_stu_start = 0, 0
166
+ else:
167
+ local_tea_start = boundaries[idx - 1][0] + 1
168
+ local_stu_start = boundaries[idx - 1][1] + 1
169
+ local_tea_end = boundaries[idx][0] + 1
170
+ local_stu_end = boundaries[idx][1] + 1
171
+ segments.append((local_tea_start, local_tea_end, local_stu_start, local_stu_end))
172
+
173
+ return segments, tea_ids_list, stu_ids_list
174
+
175
+ # ------------------------------------------------------------------
176
+ # Build virtual common vocabulary logits for a single sample
177
+ # ------------------------------------------------------------------
178
+ def _build_virtual_vocab_logits(
179
+ self,
180
+ segments,
181
+ stu_logits_aligned,
182
+ tea_logits_aligned,
183
+ stu_label_ids_list,
184
+ tea_label_ids_list,
185
+ ):
186
+ """Build student and teacher logit matrices on the virtual common vocabulary.
187
+
188
+ For each aligned segment we construct a logit vector of size
189
+ ``num_overlap + num_total_spans`` where:
190
+ - the first ``num_overlap`` dims come from the base overlap vocabulary,
191
+ - the remaining ``num_total_spans`` dims correspond to span logits.
192
+
193
+ For a 1:1 aligned position the overlap-vocab logits are taken directly
194
+ from the single token; for a span the overlap-vocab logits are taken
195
+ from the **first** token position only (since the span is treated as a
196
+ single virtual token — other overlap tokens compete with it at the
197
+ first position). The span-logit dimensions are filled with ``-1e9``
198
+ (≈ 0 after softmax) except for the span that the current segment
199
+ belongs to, which gets the mean of each constituent token's logit at
200
+ its own token id (i.e. the model's average confidence in generating
201
+ this span).
202
+
203
+ Args:
204
+ segments: list of (tea_start, tea_end, stu_start, stu_end) —
205
+ indices into the label id sequences.
206
+ stu_logits_aligned: student logits at loss_mask positions
207
+ [num_stu_loss_tokens, vocab_s].
208
+ tea_logits_aligned: teacher logits at loss_mask positions
209
+ [num_tea_loss_tokens, vocab_t].
210
+ stu_label_ids_list: Python list of student label token ids.
211
+ tea_label_ids_list: Python list of teacher label token ids.
212
+
213
+ Returns:
214
+ stu_virtual_logits: [num_segments, num_overlap + num_spans]
215
+ tea_virtual_logits: [num_segments, num_overlap + num_spans]
216
+ """
217
+ num_overlap = self.student_overlap_token_ids.shape[0]
218
+ device = stu_logits_aligned.device
219
+
220
+ # Identify which segments are spans (multi-token on either side)
221
+ span_indices = [] # indices into `segments` that are spans
222
+ for seg_idx, (ts, te, ss, se) in enumerate(segments):
223
+ if (te - ts) > 1 or (se - ss) > 1:
224
+ span_indices.append(seg_idx)
225
+ num_spans = len(span_indices)
226
+ # Map from segment index to its position in the span dimensions
227
+ seg_to_span_dim = {}
228
+ for dim_idx, seg_idx in enumerate(span_indices):
229
+ seg_to_span_dim[seg_idx] = dim_idx
230
+
231
+ virtual_dim = num_overlap + num_spans
232
+
233
+ stu_rows = []
234
+ tea_rows = []
235
+
236
+ for seg_idx, (ts, te, ss, se) in enumerate(segments):
237
+ # --- Student side ---
238
+ stu_seg_logits = stu_logits_aligned[ss:se] # [num_stu_tokens, vocab_s]
239
+ # For overlap logits, always use the FIRST token position only.
240
+ # The span is a single virtual token; other overlap tokens compete
241
+ # with it at the first position (i.e. "what comes after the prefix").
242
+ stu_first_logits = stu_seg_logits[0] # [vocab_s]
243
+ stu_overlap = stu_first_logits[self.student_overlap_token_ids] # [num_overlap]
244
+
245
+ # --- Teacher side ---
246
+ tea_seg_logits = tea_logits_aligned[ts:te] # [num_tea_tokens, vocab_t]
247
+ tea_first_logits = tea_seg_logits[0] # [vocab_t]
248
+ tea_overlap = tea_first_logits[self.teacher_overlap_token_ids] # [num_overlap]
249
+
250
+ if num_spans > 0:
251
+ # Span dimensions: default to -1e9 (negligible after softmax)
252
+ stu_span_dims = torch.full((num_spans,), -1e9, device=device, dtype=stu_overlap.dtype)
253
+ tea_span_dims = torch.full((num_spans,), -1e9, device=device, dtype=tea_overlap.dtype)
254
+
255
+ if seg_idx in seg_to_span_dim:
256
+ dim_pos = seg_to_span_dim[seg_idx]
257
+ # Span logit = mean of each constituent token's logit at
258
+ # its own token id. This represents the model's average
259
+ # confidence in generating this specific span of text.
260
+ # After softmax this corresponds to the geometric mean of
261
+ # per-token probabilities (avoiding length-dependent decay).
262
+ stu_span_token_ids = stu_label_ids_list[ss:se]
263
+ stu_self_logits = torch.stack([
264
+ stu_seg_logits[k, tid]
265
+ for k, tid in enumerate(stu_span_token_ids)
266
+ ])
267
+ stu_span_dims[dim_pos] = stu_self_logits.mean()
268
+
269
+ tea_span_token_ids = tea_label_ids_list[ts:te]
270
+ tea_self_logits = torch.stack([
271
+ tea_seg_logits[k, tid]
272
+ for k, tid in enumerate(tea_span_token_ids)
273
+ ])
274
+ tea_span_dims[dim_pos] = tea_self_logits.mean()
275
+
276
+ # Debug: print one example of span logits vs overlap logits
277
+ if self._span_debug_count < 1:
278
+ self._span_debug_count += 1
279
+ logger.info(
280
+ f"\n[SpanCTKD SPAN DEBUG] === Example span logits (seg_idx={seg_idx}) ===\n"
281
+ f" Student span token ids: {stu_span_token_ids}\n"
282
+ f" Student self_logits (per token): {stu_self_logits.detach().cpu().tolist()}\n"
283
+ f" Student span_dim value (mean): {stu_span_dims[dim_pos].item():.4f}\n"
284
+ f" Teacher span token ids: {tea_span_token_ids}\n"
285
+ f" Teacher self_logits (per token): {tea_self_logits.detach().cpu().tolist()}\n"
286
+ f" Teacher span_dim value (mean): {tea_span_dims[dim_pos].item():.4f}\n"
287
+ f" ---\n"
288
+ f" Student overlap logits (first pos) stats: mean={stu_overlap.mean().item():.4f}, "
289
+ f"max={stu_overlap.max().item():.4f}, min={stu_overlap.min().item():.4f}, "
290
+ f"std={stu_overlap.std().item():.4f}\n"
291
+ f" Teacher overlap logits (first pos) stats: mean={tea_overlap.mean().item():.4f}, "
292
+ f"max={tea_overlap.max().item():.4f}, min={tea_overlap.min().item():.4f}, "
293
+ f"std={tea_overlap.std().item():.4f}\n"
294
+ f" num_overlap={num_overlap}, num_spans={num_spans}, virtual_dim={virtual_dim}\n"
295
+ f" All stu_span_dims: {stu_span_dims.detach().cpu().tolist()}\n"
296
+ f" All tea_span_dims: {tea_span_dims.detach().cpu().tolist()}"
297
+ )
298
+
299
+ stu_row = torch.cat([stu_overlap, stu_span_dims]) # [virtual_dim]
300
+ tea_row = torch.cat([tea_overlap, tea_span_dims]) # [virtual_dim]
301
+ else:
302
+ stu_row = stu_overlap
303
+ tea_row = tea_overlap
304
+
305
+ stu_rows.append(stu_row)
306
+ tea_rows.append(tea_row)
307
+
308
+ stu_virtual_logits = torch.stack(stu_rows) # [num_segments, virtual_dim]
309
+ tea_virtual_logits = torch.stack(tea_rows) # [num_segments, virtual_dim]
310
+ return stu_virtual_logits, tea_virtual_logits
311
+
312
+ # ------------------------------------------------------------------
313
+ # Re-normalization multiplier G(h) safeguard
314
+ # ------------------------------------------------------------------
315
+ @torch.no_grad()
316
+ def _compute_segment_teacher_Z(self, segments, tea_logits_aligned, tea_label_ids_list):
317
+ """Teacher mass ``Z_T(h)`` captured by the SimCT candidate space, per segment.
318
+
319
+ ``Z_T(h)`` is computed on the **full-vocabulary** teacher softmax so that
320
+ it measures the original teacher probability mass that the candidate
321
+ space (shared tokens + the span realized at this position) represents:
322
+
323
+ - shared-token mass at the first teacher position, plus
324
+ - for a span, the geometric-mean probability of the span's teacher
325
+ tokens (matching the span logit used in the virtual distribution).
326
+
327
+ The re-normalization multiplier is then ``G(h) = 1 / Z_T(h)``. A large
328
+ ``G(h)`` means the candidate space holds little teacher mass, so
329
+ re-normalizing over it would amplify a low-mass tail.
330
+
331
+ Returns:
332
+ Z: [num_segments] float tensor in (0, 1].
333
+ """
334
+ device = tea_logits_aligned.device
335
+ Z = torch.ones(len(segments), device=device, dtype=torch.float32)
336
+ for seg_idx, (ts, te, ss, se) in enumerate(segments):
337
+ tea_seg_logits = tea_logits_aligned[ts:te] # [k_t, vocab_t]
338
+ p_first = torch.softmax(tea_seg_logits[0].float(), dim=-1) # full-vocab probs
339
+ m = p_first[self.teacher_overlap_token_ids].sum() # shared-token mass
340
+ if (te - ts) > 1 or (se - ss) > 1:
341
+ span_ids = tea_label_ids_list[ts:te]
342
+ logps = torch.stack([
343
+ torch.log_softmax(tea_seg_logits[k].float(), dim=-1)[tid]
344
+ for k, tid in enumerate(span_ids)
345
+ ])
346
+ span_prob = torch.exp(logps.mean()) # geometric mean
347
+ Z[seg_idx] = (m + span_prob).clamp(max=1.0)
348
+ else:
349
+ Z[seg_idx] = m.clamp(max=1.0)
350
+ return Z
351
+
352
+ # ------------------------------------------------------------------
353
+ # Training step
354
+ # ------------------------------------------------------------------
355
+ def training_step(self, micro_batch):
356
+ """Perform one training step with span-based cross-tokenizer KD loss.
357
+
358
+ Flow:
359
+ 1. Student forward pass → full-sequence logits.
360
+ 2. Teacher logits from hidden states (loss_mask positions only).
361
+ 3. Per-sample: align sequences, identify spans, build virtual vocab
362
+ logits, compute RKL loss.
363
+ """
364
+ student_input_ids = micro_batch["stu_input_ids"]
365
+ student_attn_mask = micro_batch["stu_attn_mask"]
366
+ student_loss_mask = micro_batch["stu_loss_mask"].bool()
367
+ teacher_input_ids = micro_batch["tea_input_ids"]
368
+ teacher_attn_mask = micro_batch["tea_attn_mask"]
369
+ teacher_loss_mask = micro_batch["tea_loss_mask"].bool()
370
+ teacher_hiddens = micro_batch.get("teacher_hiddens", None)
371
+ avg_token_num = micro_batch["avg_micro_batch_token_num"]
372
+
373
+ assert teacher_hiddens is not None, "micro_batch must contain `teacher_hiddens` for KD"
374
+
375
+ mm_kwargs = {k[3:]: v for k, v in micro_batch.items() if k.startswith("mm_")}
376
+
377
+ # Student forward
378
+ output = self.student(
379
+ student_input_ids,
380
+ attention_mask=student_attn_mask,
381
+ allgather_logits=True,
382
+ ring_attn_group=self.strategy.ring_attn_group,
383
+ **mm_kwargs,
384
+ )
385
+ student_logits = output["logits"]
386
+
387
+ # Extract label ids (next-token) at loss_mask positions — consistent
388
+ # with simple_ctkd which aligns on labels, not inputs.
389
+ student_label_ids = student_input_ids.roll(shifts=-1, dims=1)
390
+ teacher_label_ids = teacher_input_ids.roll(shifts=-1, dims=1)
391
+
392
+ # Flatten student logits at loss_mask positions (same as simple_ctkd)
393
+ student_logits_flat = student_logits[student_loss_mask] # [total_stu_loss_tokens, vocab_s]
394
+
395
+ # Free full student logits early — only loss_mask positions are needed
396
+ del student_logits, output
397
+
398
+ # NOTE: `teacher_hiddens` from teacher_actor is ALREADY 2D and filtered
399
+ # by loss_mask — shape [total_tea_loss_tokens, hidden_size]. See
400
+ # teacher_actor.py where hidden states are gathered only at loss_mask
401
+ # positions and np.concatenate-d across the micro-batch. Do NOT re-index
402
+ # with teacher_loss_mask (that would be shape [B, T] and mismatch).
403
+ teacher_hiddens = teacher_hiddens.to(self.teacher_lm_head.weight)
404
+ teacher_logits_flat = self.teacher_lm_head(teacher_hiddens) # [total_tea_loss_tokens, vocab_t]
405
+ del teacher_hiddens # free after lm_head computation
406
+
407
+ # Process each sample in the batch
408
+ batch_size = student_input_ids.shape[0]
409
+ total_loss = torch.tensor(0.0, device=student_logits_flat.device, requires_grad=True)
410
+ total_aligned_tokens = 0
411
+ total_response_tokens = 0
412
+ total_kept_segments = 0
413
+ total_masked_segments = 0
414
+
415
+ # Offset trackers for flattened logits (both are 2D across batch)
416
+ tea_logits_offset = 0
417
+ stu_logits_offset = 0
418
+
419
+ for b in range(batch_size):
420
+ stu_mask = student_loss_mask[b]
421
+ tea_mask = teacher_loss_mask[b]
422
+
423
+ stu_num_loss_tokens = stu_mask.sum().item()
424
+ tea_num_loss_tokens = tea_mask.sum().item()
425
+
426
+ # Student logits for this sample (already flattened by loss_mask)
427
+ stu_logits_b = student_logits_flat[stu_logits_offset:stu_logits_offset + stu_num_loss_tokens]
428
+ stu_logits_offset += stu_num_loss_tokens
429
+
430
+ # Teacher logits for this sample (already flattened by loss_mask)
431
+ tea_logits_b = teacher_logits_flat[tea_logits_offset:tea_logits_offset + tea_num_loss_tokens]
432
+ tea_logits_offset += tea_num_loss_tokens
433
+
434
+ # Extract label ids at loss_mask positions
435
+ stu_label_ids_b = student_label_ids[b][stu_mask].cpu().tolist()
436
+ tea_label_ids_b = teacher_label_ids[b][tea_mask].cpu().tolist()
437
+
438
+ # Align on label ids (next-token) and identify spans
439
+ segments, tea_ids_list, stu_ids_list = self._align_sequences_with_spans(
440
+ tea_label_ids_b, stu_label_ids_b
441
+ )
442
+
443
+ if len(segments) == 0:
444
+ total_response_tokens += max(stu_num_loss_tokens, tea_num_loss_tokens)
445
+ continue
446
+
447
+ # Build virtual vocab logits and compute loss.
448
+ # stu_logits_b and tea_logits_b are already [num_loss_tokens, vocab]
449
+ # and segments index directly into them.
450
+ stu_virtual, tea_virtual = self._build_virtual_vocab_logits(
451
+ segments, stu_logits_b, tea_logits_b,
452
+ stu_ids_list, tea_ids_list,
453
+ )
454
+
455
+ assert stu_virtual.shape == tea_virtual.shape, \
456
+ f"Virtual logit shape mismatch: student {stu_virtual.shape} vs teacher {tea_virtual.shape}"
457
+
458
+ # Compute RKL loss on virtual common vocabulary
459
+ sample_loss = self.loss_fn(
460
+ stu_virtual,
461
+ tea_virtual.detach(),
462
+ reduction="none",
463
+ )
464
+
465
+ # Re-normalization safeguard: drop positions whose multiplier
466
+ # G(h) = 1 / Z_T(h) exceeds the threshold (teacher captured mass
467
+ # Z_T(h) < 1/threshold). These low-mass positions would otherwise
468
+ # let re-normalization amplify a weak/noisy teacher tail, so they
469
+ # contribute no distillation gradient.
470
+ gh_threshold = getattr(self.args.kd, "span_gh_mask_threshold", 0.0)
471
+ if gh_threshold and gh_threshold > 0:
472
+ Z = self._compute_segment_teacher_Z(segments, tea_logits_b, tea_ids_list)
473
+ keep = (Z >= (1.0 / gh_threshold)).to(sample_loss.dtype) # G(h) <= threshold
474
+ total_kept_segments += int(keep.sum().item())
475
+ total_masked_segments += int((keep.numel() - keep.sum()).item())
476
+ sample_loss = sample_loss * keep
477
+
478
+ total_loss = total_loss + sample_loss.sum()
479
+
480
+ # Count aligned tokens (sum of all segment lengths on student side)
481
+ for ts, te, ss, se in segments:
482
+ total_aligned_tokens += max(te - ts, se - ss)
483
+ total_response_tokens += max(stu_num_loss_tokens, tea_num_loss_tokens)
484
+
485
+ # Normalize
486
+ kd_loss = total_loss / avg_token_num
487
+
488
+ align_ratio = torch.tensor(
489
+ total_aligned_tokens / max(total_response_tokens, 1),
490
+ device=student_logits_flat.device,
491
+ )
492
+
493
+ loss_info = {"loss": kd_loss, "kd_loss": kd_loss, "align_ratio": align_ratio}
494
+
495
+ gh_masked_total = total_kept_segments + total_masked_segments
496
+ loss_info["gh_mask_ratio"] = torch.tensor(
497
+ total_masked_segments / max(gh_masked_total, 1),
498
+ device=student_logits_flat.device,
499
+ )
500
+
501
+ if self.args.kd.kd_ratio < 1:
502
+ ce_label_ids = student_label_ids[student_loss_mask]
503
+ ce_loss = compute_cross_entropy(student_logits_flat, ce_label_ids, reduction="sum") / avg_token_num
504
+ loss = (1 - self.args.kd.kd_ratio) * ce_loss + self.args.kd.kd_ratio * kd_loss
505
+ loss_info["loss"] = loss
506
+ loss_info["ce_loss"] = ce_loss
507
+
508
+ return loss_info
kdflow/algorithms/simct.py ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Author-code SimCT baseline, integrated with CTOPD's hidden-state interface."""
2
+ import torch
3
+ from torch.utils.checkpoint import checkpoint
4
+
5
+ from . import register_algorithm
6
+
7
+
8
+ @register_algorithm('simct')
9
+ class SimCT:
10
+ def __init__(self, strategy, student_model, teacher_lm_head,
11
+ student_tokenizer, teacher_tokenizer, **kwargs):
12
+ # Lazy: selecting Strict/MP never imports the author's implementation.
13
+ from ._simct_upstream.span_ctkd import SpanCrossTokenizerKD
14
+ self.args = strategy.args
15
+ if self.args.scenario != 'on_policy_kd':
16
+ raise ValueError('SimCT baseline supports on-policy KD only')
17
+ if (self.args.kd.kd_ratio != 1 or self.args.kd.kd_temperature != 1
18
+ or self.args.kd.kd_loss_fn != 'rkl'):
19
+ raise ValueError('SimCT baseline requires kd_ratio=1, kd_temperature=1, kd_loss_fn=rkl')
20
+ if (self.args.train.num_nodes * self.args.train.num_gpus_per_node != 1
21
+ or self.args.model.ring_attn_size != 1):
22
+ raise ValueError('SimCT baseline initially requires one student rank and ring_attn_size=1')
23
+ self.student, self.teacher_head = student_model, teacher_lm_head
24
+ self.upstream = SpanCrossTokenizerKD(strategy, student_model, teacher_lm_head,
25
+ student_tokenizer, teacher_tokenizer, **kwargs)
26
+ # Preserve calculations but suppress upstream logs of response token IDs/logits.
27
+ self.upstream._debug_count, self.upstream._span_debug_count = 3, 1
28
+ config = getattr(student_model.model.config, 'text_config', None) or student_model.model.config
29
+ self.softcap = getattr(config, 'final_logit_softcapping', None)
30
+ if self.softcap is not None and self.softcap <= 0:
31
+ raise ValueError('final_logit_softcapping must be positive')
32
+
33
+ def _student_logits(self, hidden):
34
+ # Match the author's actual HF student output, including final softcap.
35
+ logits = self.student.model.lm_head(hidden, skip=False)
36
+ return logits if self.softcap is None else torch.tanh(logits / self.softcap) * self.softcap
37
+
38
+ def training_step(self, micro_batch):
39
+ sm, tm = micro_batch['stu_loss_mask'].bool(), micro_batch['tea_loss_mask'].bool()
40
+ if sm.shape[0] != tm.shape[0]:
41
+ raise ValueError('SimCT response counts differ')
42
+ output = self.student(micro_batch['stu_input_ids'], attention_mask=micro_batch['stu_attn_mask'],
43
+ allgather_logits=True, ring_attn_group=None,
44
+ **(micro_batch.get('stu_multi_modal_inputs') or {}))
45
+ hidden = output['hidden_states'][-1][sm]
46
+ th = micro_batch['teacher_hiddens']
47
+ if isinstance(th, list):
48
+ th = torch.cat(th, 0)
49
+ if th.shape[0] != int(tm.sum()):
50
+ raise ValueError('SimCT teacher hidden rows do not match loss mask')
51
+ th = th.to(self.teacher_head.weight)
52
+ sy = micro_batch['stu_input_ids'].roll(-1, 1)
53
+ ty = micro_batch['tea_input_ids'].roll(-1, 1)
54
+ total = hidden.sum() * 0.0
55
+ aligned = response_tokens = dropped = kept = 0
56
+ so = to = 0
57
+ for row in range(sm.shape[0]):
58
+ sid, tid = sy[row][sm[row]].tolist(), ty[row][tm[row]].tolist()
59
+ sh, teacher_hidden = hidden[so:so + len(sid)], th[to:to + len(tid)]
60
+ so, to = so + len(sid), to + len(tid)
61
+ response_tokens += max(len(sid), len(tid))
62
+ segments, tid, sid = self.upstream._align_sequences_with_spans(tid, sid)
63
+ if not segments:
64
+ continue
65
+ # One response at a time; preserve the author's virtual-vocabulary math.
66
+ # Target memory/throughput is not qualified by the CPU parity tests.
67
+ sl = checkpoint(self._student_logits, sh, use_reentrant=False) if torch.is_grad_enabled() else self._student_logits(sh)
68
+ with torch.no_grad():
69
+ tl = self.teacher_head(teacher_hidden) # author's teacher head has no softcap
70
+ sv, tv = self.upstream._build_virtual_vocab_logits(segments, sl, tl, sid, tid)
71
+ losses = self.upstream.loss_fn(sv, tv.detach(), reduction='none')
72
+ threshold = self.args.kd.simct_gh_mask_threshold
73
+ if threshold > 0:
74
+ mass = self.upstream._compute_segment_teacher_Z(segments, tl, tid)
75
+ keep = mass >= 1 / threshold
76
+ kept += int(keep.sum())
77
+ dropped += keep.numel() - int(keep.sum())
78
+ losses = losses * keep.to(losses.dtype)
79
+ total = total + losses.sum()
80
+ aligned += sum(max(te - ts, se - ss) for ts, te, ss, se in segments)
81
+ denominator = micro_batch['avg_micro_batch_token_num']
82
+ if float(denominator) <= 0:
83
+ raise ValueError('SimCT requires positive shared token normalization')
84
+ loss = total / denominator
85
+ if not bool(torch.isfinite(loss)):
86
+ raise FloatingPointError('SimCT produced non-finite loss')
87
+ return {'train/loss': loss, 'train/kd_loss': loss,
88
+ 'distill/alignment_ratio': loss.detach().new_tensor(aligned / max(response_tokens, 1)),
89
+ 'distill/simct_gh_mask_ratio': loss.detach().new_tensor(dropped / max(dropped + kept, 1))}
kdflow/arguments/distillation_args.py CHANGED
@@ -23,7 +23,7 @@ class DistillationArguments:
23
  default="byte_align_ctkd",
24
  metadata={
25
  "help": "KD algorithm for each training step.",
26
- "choices": ["simple_ctkd", "byte_align_ctkd", "topk_byte_align_ctkd", *MP_OPD_ALGORITHMS],
27
  }
28
  )
29
  kd_loss_fn: str = field(
@@ -111,6 +111,9 @@ class DistillationArguments:
111
  metadata={"help": "Top-k Ranking for Hierarchical Ranking Loss."}
112
  )
113
 
 
 
 
114
  # Opt-in MP-OPD namespace. None of these fields changes a Strict mode.
115
  mp_opd_fixed_span_length: int = field(default=2)
116
  mp_opd_random_min_length: int = field(default=1)
@@ -131,12 +134,17 @@ class DistillationArguments:
131
  "This release supports one teacher model per run; "
132
  "use --teacher_name_or_path instead of --multi_teacher_config."
133
  )
134
- supported_algorithms = {"simple_ctkd", "byte_align_ctkd", "topk_byte_align_ctkd", *MP_OPD_ALGORITHMS}
135
  if self.kd_algorithm not in supported_algorithms:
136
  raise ValueError(
137
  f"kd_algorithm must be one of {sorted(supported_algorithms)}, "
138
  f"got {self.kd_algorithm!r}."
139
  )
 
 
 
 
 
140
  if self.kd_algorithm in MP_OPD_ALGORITHMS:
141
  if self.kd_ratio != 1.0 or self.kd_temperature != 1.0 or self.kd_loss_fn != 'rkl':
142
  raise ValueError('MP-OPD requires kd_ratio=1, kd_temperature=1, kd_loss_fn=rkl')
 
23
  default="byte_align_ctkd",
24
  metadata={
25
  "help": "KD algorithm for each training step.",
26
+ "choices": ["simple_ctkd", "byte_align_ctkd", "topk_byte_align_ctkd", "simct", *MP_OPD_ALGORITHMS],
27
  }
28
  )
29
  kd_loss_fn: str = field(
 
111
  metadata={"help": "Top-k Ranking for Hierarchical Ranking Loss."}
112
  )
113
 
114
+ simct_gh_mask_threshold: float = field(default=0.0,
115
+ metadata={'help': 'Author SimCT optional G(h) mask; 0 disables it as in released launchers.'})
116
+
117
  # Opt-in MP-OPD namespace. None of these fields changes a Strict mode.
118
  mp_opd_fixed_span_length: int = field(default=2)
119
  mp_opd_random_min_length: int = field(default=1)
 
134
  "This release supports one teacher model per run; "
135
  "use --teacher_name_or_path instead of --multi_teacher_config."
136
  )
137
+ supported_algorithms = {"simple_ctkd", "byte_align_ctkd", "topk_byte_align_ctkd", "simct", *MP_OPD_ALGORITHMS}
138
  if self.kd_algorithm not in supported_algorithms:
139
  raise ValueError(
140
  f"kd_algorithm must be one of {sorted(supported_algorithms)}, "
141
  f"got {self.kd_algorithm!r}."
142
  )
143
+ if self.kd_algorithm == 'simct':
144
+ if self.kd_ratio != 1 or self.kd_temperature != 1 or self.kd_loss_fn != 'rkl':
145
+ raise ValueError('SimCT baseline requires kd_ratio=1, kd_temperature=1, kd_loss_fn=rkl')
146
+ if not math.isfinite(self.simct_gh_mask_threshold) or self.simct_gh_mask_threshold < 0:
147
+ raise ValueError('SimCT G(h) threshold must be finite and nonnegative')
148
  if self.kd_algorithm in MP_OPD_ALGORITHMS:
149
  if self.kd_ratio != 1.0 or self.kd_temperature != 1.0 or self.kd_loss_fn != 'rkl':
150
  raise ValueError('MP-OPD requires kd_ratio=1, kd_temperature=1, kd_loss_fn=rkl')
kdflow/cli/train_kd_on_policy.py CHANGED
@@ -22,6 +22,10 @@ from kdflow.utils.utils import get_tokenizer, load_custom_eval_fn
22
 
23
 
24
  def train(args):
 
 
 
 
25
  if args.kd.kd_algorithm in MP_OPD_ALGORITHMS and (
26
  args.train.num_nodes * args.train.num_gpus_per_node != 1
27
  or args.model.ring_attn_size != 1
@@ -91,11 +95,12 @@ def train(args):
91
  "simple_ctkd",
92
  "byte_align_ctkd",
93
  "topk_byte_align_ctkd",
 
94
  *MP_OPD_ALGORITHMS,
95
  }:
96
  raise ValueError(
97
  "Cross-tokenizer distillation supports simple_ctkd, "
98
- "byte_align_ctkd, topk_byte_align_ctkd, or an opt-in mp_opd_* mode."
99
  )
100
 
101
  environment_rollout = bool(args.rollout.alfworld_config)
 
22
 
23
 
24
  def train(args):
25
+ if args.kd.kd_algorithm == 'simct' and (
26
+ args.train.num_nodes * args.train.num_gpus_per_node != 1 or args.model.ring_attn_size != 1
27
+ ):
28
+ raise ValueError('SimCT baseline initially requires one student rank and ring_attn_size=1')
29
  if args.kd.kd_algorithm in MP_OPD_ALGORITHMS and (
30
  args.train.num_nodes * args.train.num_gpus_per_node != 1
31
  or args.model.ring_attn_size != 1
 
95
  "simple_ctkd",
96
  "byte_align_ctkd",
97
  "topk_byte_align_ctkd",
98
+ "simct",
99
  *MP_OPD_ALGORITHMS,
100
  }:
101
  raise ValueError(
102
  "Cross-tokenizer distillation supports simple_ctkd, "
103
+ "byte_align_ctkd, topk_byte_align_ctkd, simct, or an opt-in mp_opd_* mode."
104
  )
105
 
106
  environment_rollout = bool(args.rollout.alfworld_config)
tests/fixtures/simct_upstream/reverse_kl_div.py.txt ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import torch
2
+
3
+ from kdflow.loss import register_loss
4
+
5
+
6
+ @register_loss("rkl")
7
+ @torch.compile()
8
+ def compute_reverse_kl_div(
9
+ student_logits,
10
+ teacher_logits,
11
+ temperature=1.0,
12
+ reduction="none",
13
+ **kwargs
14
+ ):
15
+ student_logits = student_logits / temperature
16
+ teacher_logits = teacher_logits / temperature
17
+ student_log_probs = torch.log_softmax(student_logits, -1, dtype=torch.float32)
18
+ teacher_log_probs = torch.log_softmax(teacher_logits, -1, dtype=torch.float32)
19
+ student_probs = student_log_probs.exp()
20
+ rkl_div = (student_probs * (student_log_probs - teacher_log_probs)).sum(-1)
21
+
22
+ if reduction == "mean":
23
+ rkl_div = rkl_div.mean()
24
+ elif reduction == "sum":
25
+ rkl_div = rkl_div.sum()
26
+
27
+ return rkl_div
tests/fixtures/simct_upstream/span_ctkd.py.txt ADDED
@@ -0,0 +1,508 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import torch
2
+ import torch.nn.functional as F
3
+
4
+ from kdflow.loss import build_loss_fn
5
+ from kdflow.algorithms import register_algorithm
6
+ from kdflow.loss.cross_entropy import compute_cross_entropy
7
+ from kdflow.utils.logging_utils import init_logger
8
+
9
+ logger = init_logger(__name__)
10
+
11
+
12
+ @register_algorithm("span_ctkd")
13
+ class SpanCrossTokenizerKD:
14
+ """Span-based Cross-Tokenizer Knowledge Distillation.
15
+
16
+ Improvement over SimpleCrossTokenizerKD: instead of discarding positions where
17
+ student and teacher tokens do not align 1:1, we group misaligned tokens into
18
+ "spans" and compute their logits as the arithmetic mean of the constituent
19
+ token logits (which corresponds to geometric mean after softmax).
20
+
21
+ These spans are appended to the shared (overlap) vocabulary to form a
22
+ "virtual common vocabulary". RKL alignment is then performed on this
23
+ extended vocabulary for every aligned position (both 1:1 tokens and spans).
24
+
25
+ For span segments, the overlap logits are taken from the **first** token
26
+ position only (not averaged across positions), because the span is treated
27
+ as a single virtual token occupying one position in the sequence.
28
+ """
29
+
30
+ def __init__(
31
+ self,
32
+ strategy,
33
+ student_model,
34
+ teacher_lm_head,
35
+ student_tokenizer,
36
+ teacher_tokenizer,
37
+ **kwargs,
38
+ ):
39
+ self.strategy = strategy
40
+ self.args = strategy.args
41
+ self.student = student_model
42
+ self.teacher_lm_head = teacher_lm_head
43
+ self.student_tokenizer = student_tokenizer
44
+ self.teacher_tokenizer = teacher_tokenizer
45
+ self.student_overlap_token_ids, self.teacher_overlap_token_ids = self._find_overlap_tokens()
46
+ self.loss_fn = build_loss_fn(self.args.kd.kd_loss_fn, self.args)
47
+ self._debug_count = 0 # Counter for debug logging
48
+ self._span_debug_count = 0 # Counter for span logits debug logging
49
+
50
+ # ------------------------------------------------------------------
51
+ # Overlap vocabulary (copied from simple_ctkd)
52
+ # ------------------------------------------------------------------
53
+ def _find_overlap_tokens(self):
54
+ """Find the overlap tokens between student and teacher tokenizer."""
55
+ student_vocab = {k.replace("Ġ", "▁"): v for k, v in self.student_tokenizer.get_vocab().items()}
56
+ teacher_vocab = {k.replace("Ġ", "▁"): v for k, v in self.teacher_tokenizer.get_vocab().items()}
57
+ overlap_tokens = set(student_vocab.keys()) & set(teacher_vocab.keys())
58
+ student_ids = [student_vocab[token] for token in overlap_tokens]
59
+ teacher_ids = [teacher_vocab[token] for token in overlap_tokens]
60
+ stu_eos, tea_eos = self.student_tokenizer.eos_token_id, self.teacher_tokenizer.eos_token_id
61
+ if stu_eos not in student_ids or tea_eos not in teacher_ids:
62
+ student_ids.append(stu_eos)
63
+ teacher_ids.append(tea_eos)
64
+ device = self.teacher_lm_head.weight.device
65
+ logger.info(f"[SpanCTKD] Num of overlap_tokens between student & teacher: {len(student_ids)}")
66
+ return (
67
+ torch.tensor(student_ids, dtype=torch.long, device=device),
68
+ torch.tensor(teacher_ids, dtype=torch.long, device=device),
69
+ )
70
+
71
+ # ------------------------------------------------------------------
72
+ # Sequence alignment with span identification
73
+ # (adapted from ALM._compute_chunk_alignment)
74
+ # ------------------------------------------------------------------
75
+ def _align_sequences_with_spans(self, tea_label_ids, stu_label_ids):
76
+ """Align teacher and student label tokens and identify spans.
77
+
78
+ Operates on **label ids** (i.e. input_ids shifted left by one position,
79
+ already filtered by loss_mask) so that alignment is consistent with the
80
+ next-token-prediction semantics of the logits.
81
+
82
+ Uses ``tokenizer.decode([tid])`` per token (O(n)) to correctly handle
83
+ all tokenizer conventions including newlines.
84
+
85
+ Args:
86
+ tea_label_ids: 1-D list of teacher label token ids (loss_mask positions).
87
+ stu_label_ids: 1-D list of student label token ids (loss_mask positions).
88
+
89
+ Returns:
90
+ segments: list of (tea_start, tea_end, stu_start, stu_end) where
91
+ indices are positions within the label id sequences. Each tuple
92
+ represents an aligned segment: teacher tokens ``[tea_start, tea_end)``
93
+ and student tokens ``[stu_start, stu_end)`` encode the same text.
94
+ When both ranges have length 1 it is a normal 1:1 alignment;
95
+ otherwise it is a span.
96
+ tea_label_ids_list: the teacher label ids as a Python list.
97
+ stu_label_ids_list: the student label ids as a Python list.
98
+ """
99
+ if len(tea_label_ids) == 0 or len(stu_label_ids) == 0:
100
+ return [], [], []
101
+
102
+ tea_ids_list = tea_label_ids if isinstance(tea_label_ids, list) else tea_label_ids.cpu().tolist()
103
+ stu_ids_list = stu_label_ids if isinstance(stu_label_ids, list) else stu_label_ids.cpu().tolist()
104
+
105
+ # Convert token ids to text strings for alignment.
106
+ # Use tokenizer.decode([tid]) per token to correctly handle all
107
+ # tokenizer conventions (e.g. Qwen uses Ċ for \n internally, which
108
+ # would NOT be normalized by simple strip of ▁/Ġ prefixes).
109
+ tea_token_texts = [self.teacher_tokenizer.decode([tid]) for tid in tea_ids_list]
110
+ stu_token_texts = [self.student_tokenizer.decode([tid]) for tid in stu_ids_list]
111
+
112
+ # Greedy cumulative-text alignment (same algorithm as simple_ctkd)
113
+ tea_eos = self.teacher_tokenizer.eos_token
114
+ stu_eos = self.student_tokenizer.eos_token
115
+
116
+ i, j = 0, 0
117
+ boundaries = []
118
+ history_tea = ""
119
+ history_stu = ""
120
+
121
+ while i < len(tea_token_texts) and j < len(stu_token_texts):
122
+ is_eos_match = (tea_token_texts[i] == tea_eos and stu_token_texts[j] == stu_eos)
123
+ if history_tea == history_stu and (
124
+ tea_token_texts[i] == stu_token_texts[j] or is_eos_match
125
+ ):
126
+ boundaries.append((i, j))
127
+ history_tea += tea_token_texts[i]
128
+ history_stu += stu_token_texts[j]
129
+ i += 1
130
+ j += 1
131
+ elif len(history_tea) > len(history_stu):
132
+ history_stu += stu_token_texts[j]
133
+ j += 1
134
+ elif len(history_tea) < len(history_stu):
135
+ history_tea += tea_token_texts[i]
136
+ i += 1
137
+ else:
138
+ history_tea += tea_token_texts[i]
139
+ history_stu += stu_token_texts[j]
140
+ i += 1
141
+ j += 1
142
+
143
+ if len(boundaries) == 0:
144
+ if self._debug_count < 3:
145
+ self._debug_count += 1
146
+ logger.warning(
147
+ f"[SpanCTKD DEBUG] No boundaries found! "
148
+ f"tea_token_texts({len(tea_token_texts)}): {tea_token_texts[:10]}... "
149
+ f"stu_token_texts({len(stu_token_texts)}): {stu_token_texts[:10]}..."
150
+ )
151
+ return [], tea_ids_list, stu_ids_list
152
+
153
+ if self._debug_count < 3:
154
+ self._debug_count += 1
155
+ logger.info(
156
+ f"[SpanCTKD DEBUG] Found {len(boundaries)} boundaries! "
157
+ f"tea_texts({len(tea_token_texts)}): {tea_token_texts[:5]}... "
158
+ f"stu_texts({len(stu_token_texts)}): {stu_token_texts[:5]}..."
159
+ )
160
+
161
+ # Convert boundaries to aligned segments (local indices within response)
162
+ segments = []
163
+ for idx in range(len(boundaries)):
164
+ if idx == 0:
165
+ local_tea_start, local_stu_start = 0, 0
166
+ else:
167
+ local_tea_start = boundaries[idx - 1][0] + 1
168
+ local_stu_start = boundaries[idx - 1][1] + 1
169
+ local_tea_end = boundaries[idx][0] + 1
170
+ local_stu_end = boundaries[idx][1] + 1
171
+ segments.append((local_tea_start, local_tea_end, local_stu_start, local_stu_end))
172
+
173
+ return segments, tea_ids_list, stu_ids_list
174
+
175
+ # ------------------------------------------------------------------
176
+ # Build virtual common vocabulary logits for a single sample
177
+ # ------------------------------------------------------------------
178
+ def _build_virtual_vocab_logits(
179
+ self,
180
+ segments,
181
+ stu_logits_aligned,
182
+ tea_logits_aligned,
183
+ stu_label_ids_list,
184
+ tea_label_ids_list,
185
+ ):
186
+ """Build student and teacher logit matrices on the virtual common vocabulary.
187
+
188
+ For each aligned segment we construct a logit vector of size
189
+ ``num_overlap + num_total_spans`` where:
190
+ - the first ``num_overlap`` dims come from the base overlap vocabulary,
191
+ - the remaining ``num_total_spans`` dims correspond to span logits.
192
+
193
+ For a 1:1 aligned position the overlap-vocab logits are taken directly
194
+ from the single token; for a span the overlap-vocab logits are taken
195
+ from the **first** token position only (since the span is treated as a
196
+ single virtual token — other overlap tokens compete with it at the
197
+ first position). The span-logit dimensions are filled with ``-1e9``
198
+ (≈ 0 after softmax) except for the span that the current segment
199
+ belongs to, which gets the mean of each constituent token's logit at
200
+ its own token id (i.e. the model's average confidence in generating
201
+ this span).
202
+
203
+ Args:
204
+ segments: list of (tea_start, tea_end, stu_start, stu_end) —
205
+ indices into the label id sequences.
206
+ stu_logits_aligned: student logits at loss_mask positions
207
+ [num_stu_loss_tokens, vocab_s].
208
+ tea_logits_aligned: teacher logits at loss_mask positions
209
+ [num_tea_loss_tokens, vocab_t].
210
+ stu_label_ids_list: Python list of student label token ids.
211
+ tea_label_ids_list: Python list of teacher label token ids.
212
+
213
+ Returns:
214
+ stu_virtual_logits: [num_segments, num_overlap + num_spans]
215
+ tea_virtual_logits: [num_segments, num_overlap + num_spans]
216
+ """
217
+ num_overlap = self.student_overlap_token_ids.shape[0]
218
+ device = stu_logits_aligned.device
219
+
220
+ # Identify which segments are spans (multi-token on either side)
221
+ span_indices = [] # indices into `segments` that are spans
222
+ for seg_idx, (ts, te, ss, se) in enumerate(segments):
223
+ if (te - ts) > 1 or (se - ss) > 1:
224
+ span_indices.append(seg_idx)
225
+ num_spans = len(span_indices)
226
+ # Map from segment index to its position in the span dimensions
227
+ seg_to_span_dim = {}
228
+ for dim_idx, seg_idx in enumerate(span_indices):
229
+ seg_to_span_dim[seg_idx] = dim_idx
230
+
231
+ virtual_dim = num_overlap + num_spans
232
+
233
+ stu_rows = []
234
+ tea_rows = []
235
+
236
+ for seg_idx, (ts, te, ss, se) in enumerate(segments):
237
+ # --- Student side ---
238
+ stu_seg_logits = stu_logits_aligned[ss:se] # [num_stu_tokens, vocab_s]
239
+ # For overlap logits, always use the FIRST token position only.
240
+ # The span is a single virtual token; other overlap tokens compete
241
+ # with it at the first position (i.e. "what comes after the prefix").
242
+ stu_first_logits = stu_seg_logits[0] # [vocab_s]
243
+ stu_overlap = stu_first_logits[self.student_overlap_token_ids] # [num_overlap]
244
+
245
+ # --- Teacher side ---
246
+ tea_seg_logits = tea_logits_aligned[ts:te] # [num_tea_tokens, vocab_t]
247
+ tea_first_logits = tea_seg_logits[0] # [vocab_t]
248
+ tea_overlap = tea_first_logits[self.teacher_overlap_token_ids] # [num_overlap]
249
+
250
+ if num_spans > 0:
251
+ # Span dimensions: default to -1e9 (negligible after softmax)
252
+ stu_span_dims = torch.full((num_spans,), -1e9, device=device, dtype=stu_overlap.dtype)
253
+ tea_span_dims = torch.full((num_spans,), -1e9, device=device, dtype=tea_overlap.dtype)
254
+
255
+ if seg_idx in seg_to_span_dim:
256
+ dim_pos = seg_to_span_dim[seg_idx]
257
+ # Span logit = mean of each constituent token's logit at
258
+ # its own token id. This represents the model's average
259
+ # confidence in generating this specific span of text.
260
+ # After softmax this corresponds to the geometric mean of
261
+ # per-token probabilities (avoiding length-dependent decay).
262
+ stu_span_token_ids = stu_label_ids_list[ss:se]
263
+ stu_self_logits = torch.stack([
264
+ stu_seg_logits[k, tid]
265
+ for k, tid in enumerate(stu_span_token_ids)
266
+ ])
267
+ stu_span_dims[dim_pos] = stu_self_logits.mean()
268
+
269
+ tea_span_token_ids = tea_label_ids_list[ts:te]
270
+ tea_self_logits = torch.stack([
271
+ tea_seg_logits[k, tid]
272
+ for k, tid in enumerate(tea_span_token_ids)
273
+ ])
274
+ tea_span_dims[dim_pos] = tea_self_logits.mean()
275
+
276
+ # Debug: print one example of span logits vs overlap logits
277
+ if self._span_debug_count < 1:
278
+ self._span_debug_count += 1
279
+ logger.info(
280
+ f"\n[SpanCTKD SPAN DEBUG] === Example span logits (seg_idx={seg_idx}) ===\n"
281
+ f" Student span token ids: {stu_span_token_ids}\n"
282
+ f" Student self_logits (per token): {stu_self_logits.detach().cpu().tolist()}\n"
283
+ f" Student span_dim value (mean): {stu_span_dims[dim_pos].item():.4f}\n"
284
+ f" Teacher span token ids: {tea_span_token_ids}\n"
285
+ f" Teacher self_logits (per token): {tea_self_logits.detach().cpu().tolist()}\n"
286
+ f" Teacher span_dim value (mean): {tea_span_dims[dim_pos].item():.4f}\n"
287
+ f" ---\n"
288
+ f" Student overlap logits (first pos) stats: mean={stu_overlap.mean().item():.4f}, "
289
+ f"max={stu_overlap.max().item():.4f}, min={stu_overlap.min().item():.4f}, "
290
+ f"std={stu_overlap.std().item():.4f}\n"
291
+ f" Teacher overlap logits (first pos) stats: mean={tea_overlap.mean().item():.4f}, "
292
+ f"max={tea_overlap.max().item():.4f}, min={tea_overlap.min().item():.4f}, "
293
+ f"std={tea_overlap.std().item():.4f}\n"
294
+ f" num_overlap={num_overlap}, num_spans={num_spans}, virtual_dim={virtual_dim}\n"
295
+ f" All stu_span_dims: {stu_span_dims.detach().cpu().tolist()}\n"
296
+ f" All tea_span_dims: {tea_span_dims.detach().cpu().tolist()}"
297
+ )
298
+
299
+ stu_row = torch.cat([stu_overlap, stu_span_dims]) # [virtual_dim]
300
+ tea_row = torch.cat([tea_overlap, tea_span_dims]) # [virtual_dim]
301
+ else:
302
+ stu_row = stu_overlap
303
+ tea_row = tea_overlap
304
+
305
+ stu_rows.append(stu_row)
306
+ tea_rows.append(tea_row)
307
+
308
+ stu_virtual_logits = torch.stack(stu_rows) # [num_segments, virtual_dim]
309
+ tea_virtual_logits = torch.stack(tea_rows) # [num_segments, virtual_dim]
310
+ return stu_virtual_logits, tea_virtual_logits
311
+
312
+ # ------------------------------------------------------------------
313
+ # Re-normalization multiplier G(h) safeguard
314
+ # ------------------------------------------------------------------
315
+ @torch.no_grad()
316
+ def _compute_segment_teacher_Z(self, segments, tea_logits_aligned, tea_label_ids_list):
317
+ """Teacher mass ``Z_T(h)`` captured by the SimCT candidate space, per segment.
318
+
319
+ ``Z_T(h)`` is computed on the **full-vocabulary** teacher softmax so that
320
+ it measures the original teacher probability mass that the candidate
321
+ space (shared tokens + the span realized at this position) represents:
322
+
323
+ - shared-token mass at the first teacher position, plus
324
+ - for a span, the geometric-mean probability of the span's teacher
325
+ tokens (matching the span logit used in the virtual distribution).
326
+
327
+ The re-normalization multiplier is then ``G(h) = 1 / Z_T(h)``. A large
328
+ ``G(h)`` means the candidate space holds little teacher mass, so
329
+ re-normalizing over it would amplify a low-mass tail.
330
+
331
+ Returns:
332
+ Z: [num_segments] float tensor in (0, 1].
333
+ """
334
+ device = tea_logits_aligned.device
335
+ Z = torch.ones(len(segments), device=device, dtype=torch.float32)
336
+ for seg_idx, (ts, te, ss, se) in enumerate(segments):
337
+ tea_seg_logits = tea_logits_aligned[ts:te] # [k_t, vocab_t]
338
+ p_first = torch.softmax(tea_seg_logits[0].float(), dim=-1) # full-vocab probs
339
+ m = p_first[self.teacher_overlap_token_ids].sum() # shared-token mass
340
+ if (te - ts) > 1 or (se - ss) > 1:
341
+ span_ids = tea_label_ids_list[ts:te]
342
+ logps = torch.stack([
343
+ torch.log_softmax(tea_seg_logits[k].float(), dim=-1)[tid]
344
+ for k, tid in enumerate(span_ids)
345
+ ])
346
+ span_prob = torch.exp(logps.mean()) # geometric mean
347
+ Z[seg_idx] = (m + span_prob).clamp(max=1.0)
348
+ else:
349
+ Z[seg_idx] = m.clamp(max=1.0)
350
+ return Z
351
+
352
+ # ------------------------------------------------------------------
353
+ # Training step
354
+ # ------------------------------------------------------------------
355
+ def training_step(self, micro_batch):
356
+ """Perform one training step with span-based cross-tokenizer KD loss.
357
+
358
+ Flow:
359
+ 1. Student forward pass → full-sequence logits.
360
+ 2. Teacher logits from hidden states (loss_mask positions only).
361
+ 3. Per-sample: align sequences, identify spans, build virtual vocab
362
+ logits, compute RKL loss.
363
+ """
364
+ student_input_ids = micro_batch["stu_input_ids"]
365
+ student_attn_mask = micro_batch["stu_attn_mask"]
366
+ student_loss_mask = micro_batch["stu_loss_mask"].bool()
367
+ teacher_input_ids = micro_batch["tea_input_ids"]
368
+ teacher_attn_mask = micro_batch["tea_attn_mask"]
369
+ teacher_loss_mask = micro_batch["tea_loss_mask"].bool()
370
+ teacher_hiddens = micro_batch.get("teacher_hiddens", None)
371
+ avg_token_num = micro_batch["avg_micro_batch_token_num"]
372
+
373
+ assert teacher_hiddens is not None, "micro_batch must contain `teacher_hiddens` for KD"
374
+
375
+ mm_kwargs = {k[3:]: v for k, v in micro_batch.items() if k.startswith("mm_")}
376
+
377
+ # Student forward
378
+ output = self.student(
379
+ student_input_ids,
380
+ attention_mask=student_attn_mask,
381
+ allgather_logits=True,
382
+ ring_attn_group=self.strategy.ring_attn_group,
383
+ **mm_kwargs,
384
+ )
385
+ student_logits = output["logits"]
386
+
387
+ # Extract label ids (next-token) at loss_mask positions — consistent
388
+ # with simple_ctkd which aligns on labels, not inputs.
389
+ student_label_ids = student_input_ids.roll(shifts=-1, dims=1)
390
+ teacher_label_ids = teacher_input_ids.roll(shifts=-1, dims=1)
391
+
392
+ # Flatten student logits at loss_mask positions (same as simple_ctkd)
393
+ student_logits_flat = student_logits[student_loss_mask] # [total_stu_loss_tokens, vocab_s]
394
+
395
+ # Free full student logits early — only loss_mask positions are needed
396
+ del student_logits, output
397
+
398
+ # NOTE: `teacher_hiddens` from teacher_actor is ALREADY 2D and filtered
399
+ # by loss_mask — shape [total_tea_loss_tokens, hidden_size]. See
400
+ # teacher_actor.py where hidden states are gathered only at loss_mask
401
+ # positions and np.concatenate-d across the micro-batch. Do NOT re-index
402
+ # with teacher_loss_mask (that would be shape [B, T] and mismatch).
403
+ teacher_hiddens = teacher_hiddens.to(self.teacher_lm_head.weight)
404
+ teacher_logits_flat = self.teacher_lm_head(teacher_hiddens) # [total_tea_loss_tokens, vocab_t]
405
+ del teacher_hiddens # free after lm_head computation
406
+
407
+ # Process each sample in the batch
408
+ batch_size = student_input_ids.shape[0]
409
+ total_loss = torch.tensor(0.0, device=student_logits_flat.device, requires_grad=True)
410
+ total_aligned_tokens = 0
411
+ total_response_tokens = 0
412
+ total_kept_segments = 0
413
+ total_masked_segments = 0
414
+
415
+ # Offset trackers for flattened logits (both are 2D across batch)
416
+ tea_logits_offset = 0
417
+ stu_logits_offset = 0
418
+
419
+ for b in range(batch_size):
420
+ stu_mask = student_loss_mask[b]
421
+ tea_mask = teacher_loss_mask[b]
422
+
423
+ stu_num_loss_tokens = stu_mask.sum().item()
424
+ tea_num_loss_tokens = tea_mask.sum().item()
425
+
426
+ # Student logits for this sample (already flattened by loss_mask)
427
+ stu_logits_b = student_logits_flat[stu_logits_offset:stu_logits_offset + stu_num_loss_tokens]
428
+ stu_logits_offset += stu_num_loss_tokens
429
+
430
+ # Teacher logits for this sample (already flattened by loss_mask)
431
+ tea_logits_b = teacher_logits_flat[tea_logits_offset:tea_logits_offset + tea_num_loss_tokens]
432
+ tea_logits_offset += tea_num_loss_tokens
433
+
434
+ # Extract label ids at loss_mask positions
435
+ stu_label_ids_b = student_label_ids[b][stu_mask].cpu().tolist()
436
+ tea_label_ids_b = teacher_label_ids[b][tea_mask].cpu().tolist()
437
+
438
+ # Align on label ids (next-token) and identify spans
439
+ segments, tea_ids_list, stu_ids_list = self._align_sequences_with_spans(
440
+ tea_label_ids_b, stu_label_ids_b
441
+ )
442
+
443
+ if len(segments) == 0:
444
+ total_response_tokens += max(stu_num_loss_tokens, tea_num_loss_tokens)
445
+ continue
446
+
447
+ # Build virtual vocab logits and compute loss.
448
+ # stu_logits_b and tea_logits_b are already [num_loss_tokens, vocab]
449
+ # and segments index directly into them.
450
+ stu_virtual, tea_virtual = self._build_virtual_vocab_logits(
451
+ segments, stu_logits_b, tea_logits_b,
452
+ stu_ids_list, tea_ids_list,
453
+ )
454
+
455
+ assert stu_virtual.shape == tea_virtual.shape, \
456
+ f"Virtual logit shape mismatch: student {stu_virtual.shape} vs teacher {tea_virtual.shape}"
457
+
458
+ # Compute RKL loss on virtual common vocabulary
459
+ sample_loss = self.loss_fn(
460
+ stu_virtual,
461
+ tea_virtual.detach(),
462
+ reduction="none",
463
+ )
464
+
465
+ # Re-normalization safeguard: drop positions whose multiplier
466
+ # G(h) = 1 / Z_T(h) exceeds the threshold (teacher captured mass
467
+ # Z_T(h) < 1/threshold). These low-mass positions would otherwise
468
+ # let re-normalization amplify a weak/noisy teacher tail, so they
469
+ # contribute no distillation gradient.
470
+ gh_threshold = getattr(self.args.kd, "span_gh_mask_threshold", 0.0)
471
+ if gh_threshold and gh_threshold > 0:
472
+ Z = self._compute_segment_teacher_Z(segments, tea_logits_b, tea_ids_list)
473
+ keep = (Z >= (1.0 / gh_threshold)).to(sample_loss.dtype) # G(h) <= threshold
474
+ total_kept_segments += int(keep.sum().item())
475
+ total_masked_segments += int((keep.numel() - keep.sum()).item())
476
+ sample_loss = sample_loss * keep
477
+
478
+ total_loss = total_loss + sample_loss.sum()
479
+
480
+ # Count aligned tokens (sum of all segment lengths on student side)
481
+ for ts, te, ss, se in segments:
482
+ total_aligned_tokens += max(te - ts, se - ss)
483
+ total_response_tokens += max(stu_num_loss_tokens, tea_num_loss_tokens)
484
+
485
+ # Normalize
486
+ kd_loss = total_loss / avg_token_num
487
+
488
+ align_ratio = torch.tensor(
489
+ total_aligned_tokens / max(total_response_tokens, 1),
490
+ device=student_logits_flat.device,
491
+ )
492
+
493
+ loss_info = {"loss": kd_loss, "kd_loss": kd_loss, "align_ratio": align_ratio}
494
+
495
+ gh_masked_total = total_kept_segments + total_masked_segments
496
+ loss_info["gh_mask_ratio"] = torch.tensor(
497
+ total_masked_segments / max(gh_masked_total, 1),
498
+ device=student_logits_flat.device,
499
+ )
500
+
501
+ if self.args.kd.kd_ratio < 1:
502
+ ce_label_ids = student_label_ids[student_loss_mask]
503
+ ce_loss = compute_cross_entropy(student_logits_flat, ce_label_ids, reduction="sum") / avg_token_num
504
+ loss = (1 - self.args.kd.kd_ratio) * ce_loss + self.args.kd.kd_ratio * kd_loss
505
+ loss_info["loss"] = loss
506
+ loss_info["ce_loss"] = ce_loss
507
+
508
+ return loss_info
tests/test_simct.py ADDED
@@ -0,0 +1,197 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Author-source oracle parity. Requires existing torch; never installs packages."""
2
+ import ast
3
+ import copy
4
+ import hashlib
5
+ import json
6
+ import logging
7
+ import pathlib
8
+ import types
9
+ import unittest
10
+
11
+ import test_mp_opd_port as fixtures
12
+
13
+ ROOT = pathlib.Path(__file__).resolve().parents[1]
14
+ if fixtures.HAS_TORCH:
15
+ import torch
16
+
17
+
18
+ def original_oracle_class():
19
+ """Execute original numerical bodies, stripping only imports/decorators.
20
+
21
+ The oracle does not import the port, adapter, or CTOPD RKL implementation.
22
+ """
23
+ source = ROOT / 'tests/fixtures/simct_upstream'
24
+ rkl_tree = ast.parse((source / 'reverse_kl_div.py.txt').read_text())
25
+ rkl = next(n for n in rkl_tree.body if isinstance(n, ast.FunctionDef))
26
+ rkl.decorator_list = []
27
+ scope = {'torch': torch}
28
+ exec(compile(ast.Module(body=[rkl], type_ignores=[]), 'original_rkl', 'exec'), scope)
29
+ cls_tree = ast.parse((source / 'span_ctkd.py.txt').read_text())
30
+ cls = next(n for n in cls_tree.body if isinstance(n, ast.ClassDef))
31
+ cls.decorator_list = []
32
+ scope.update(build_loss_fn=lambda *_: scope['compute_reverse_kl_div'],
33
+ logger=logging.getLogger('simct-oracle'))
34
+ exec(compile(ast.Module(body=[cls], type_ignores=[]), 'original_simct', 'exec'), scope)
35
+ return scope['SpanCrossTokenizerKD']
36
+
37
+
38
+ @unittest.skipUnless(fixtures.HAS_TORCH, 'requires existing PyTorch; no bootstrap')
39
+ class SimCTTests(unittest.TestCase):
40
+ parity_errors = []
41
+
42
+ @classmethod
43
+ def tearDownClass(cls):
44
+ if cls.parity_errors:
45
+ print('SIMCT_PARITY ' + json.dumps({
46
+ 'cases': len(cls.parity_errors),
47
+ 'max_loss_abs_error': max(x[0] for x in cls.parity_errors),
48
+ 'max_student_gradient_abs_error': max(x[1] for x in cls.parity_errors)}))
49
+
50
+ def pair(self, *, threshold=0.0, cap=None):
51
+ from kdflow.algorithms.simct import SimCT
52
+ helper = fixtures.MPPortTests()
53
+ mp, student, teacher = helper.fixture(cap=cap)
54
+ args = mp.args
55
+ args.kd.kd_algorithm = 'simct'
56
+ args.kd.simct_gh_mask_threshold = args.kd.span_gh_mask_threshold = threshold
57
+ actual = SimCT(mp.strategy, student, teacher, fixtures.Tokenizer(), fixtures.Tokenizer())
58
+ ref_student, ref_teacher = copy.deepcopy(student), copy.deepcopy(teacher)
59
+
60
+ class OriginalHFOutput:
61
+ def __call__(self, ids, **kwargs):
62
+ hidden = ref_student(ids, **kwargs)['hidden_states'][-1]
63
+ logits = ref_student.model.lm_head(hidden, skip=False)
64
+ if cap is not None:
65
+ logits = torch.tanh(logits / cap) * cap
66
+ return {'logits': logits}
67
+
68
+ reference = original_oracle_class()(mp.strategy, OriginalHFOutput(), ref_teacher,
69
+ fixtures.Tokenizer(), fixtures.Tokenizer())
70
+ reference._debug_count, reference._span_debug_count = 3, 1
71
+ # Set iteration ordering is identical within this process. Assert explicitly.
72
+ self.assertTrue(torch.equal(actual.upstream.student_overlap_token_ids,
73
+ reference.student_overlap_token_ids))
74
+ return helper, actual, reference, student, ref_student, teacher, ref_teacher
75
+
76
+ def assert_pair(self, *, threshold=0.0, cap=None, copies=1, strict=False):
77
+ helper, actual, reference, student, rs, teacher, rt = self.pair(threshold=threshold, cap=cap)
78
+ batch = helper.batch(copies)
79
+ if strict:
80
+ batch['tea_input_ids'] = batch['stu_input_ids'].clone()
81
+ batch['tea_loss_mask'] = batch['stu_loss_mask'].clone()
82
+ batch['tea_attn_mask'] = batch['stu_attn_mask'].clone()
83
+ batch['teacher_hiddens'] = torch.randn(int(batch['tea_loss_mask'].sum()), 4, requires_grad=True)
84
+ a, b = actual.training_step(batch), reference.training_step(batch)
85
+ torch.testing.assert_close(a['train/loss'], b['loss'], atol=1e-6, rtol=1e-5)
86
+ torch.testing.assert_close(a['distill/alignment_ratio'], b['align_ratio'])
87
+ torch.testing.assert_close(a['distill/simct_gh_mask_ratio'], b['gh_mask_ratio'])
88
+ ga = torch.autograd.grad(a['train/loss'], tuple(student.parameters()))
89
+ gb = torch.autograd.grad(b['loss'], tuple(rs.parameters()))
90
+ for x, y in zip(ga, gb):
91
+ torch.testing.assert_close(x, y, atol=1e-6, rtol=1e-5)
92
+ self.parity_errors.append((float((a['train/loss'] - b['loss']).detach().abs()),
93
+ max(float((x - y).abs().max()) for x, y in zip(ga, gb))))
94
+ self.assertTrue(all(p.grad is None for p in teacher.parameters()))
95
+ self.assertTrue(all(p.grad is None for p in rt.parameters()))
96
+ # Assert teacher/hidden stop-gradient with autograd too, not only .grad buffers.
97
+ grads = torch.autograd.grad(actual.training_step(batch)['train/loss'],
98
+ (*teacher.parameters(), batch['teacher_hiddens']), allow_unused=True)
99
+ self.assertTrue(all(g is None for g in grads))
100
+
101
+ def test_mixed_spans_loss_and_all_student_gradients_match_original(self):
102
+ self.assert_pair(copies=2)
103
+
104
+ def test_strict_only_loss_and_gradients_match_original(self):
105
+ self.assert_pair(strict=True)
106
+
107
+ def test_softcap_matches_original_hf_student_output(self):
108
+ self.assert_pair(cap=2.5)
109
+
110
+ def test_optional_teacher_mass_mask_matches_original(self):
111
+ for threshold in (0.5, 2.0, 100.0):
112
+ with self.subTest(threshold=threshold):
113
+ self.assert_pair(threshold=threshold)
114
+
115
+ def test_virtual_logits_and_derivatives_match_independent_hand_calculation(self):
116
+ _, algo, _, _, _, _, _ = self.pair()
117
+ # First segment mismatches: teacher 'ab' vs student 'a','b'; next is EOS.
118
+ segments = [(0, 1, 0, 2), (1, 2, 2, 3)]
119
+ sl = torch.randn(3, 8, requires_grad=True)
120
+ tl = torch.randn(2, 8)
121
+ sv, tv = algo.upstream._build_virtual_vocab_logits(segments, sl, tl, [1, 2, 0], [4, 0])
122
+ ids, tids = algo.upstream.student_overlap_token_ids, algo.upstream.teacher_overlap_token_ids
123
+ manual_s = torch.stack([torch.cat([sl[0, ids], ((sl[0, 1] + sl[1, 2]) / 2)[None]]),
124
+ torch.cat([sl[2, ids], sl.new_tensor([-1e9])])])
125
+ manual_t = torch.stack([torch.cat([tl[0, tids], tl[0, 4][None]]),
126
+ torch.cat([tl[1, tids], tl.new_tensor([-1e9])])])
127
+ torch.testing.assert_close(sv, manual_s)
128
+ torch.testing.assert_close(tv, manual_t)
129
+ p, q = manual_s.log_softmax(-1), manual_t.log_softmax(-1)
130
+ expected = (p.exp() * (p - q)).sum()
131
+ result = algo.upstream.loss_fn(sv, tv, reduction='sum')
132
+ torch.testing.assert_close(result, expected)
133
+ torch.testing.assert_close(torch.autograd.grad(result, sl)[0],
134
+ torch.autograd.grad(expected, sl)[0])
135
+
136
+ def test_response_boundaries_and_microbatch_normalizer(self):
137
+ helper, algo, _, student, _, _, _ = self.pair()
138
+ full = helper.batch(copies=2)
139
+ one = helper.batch()
140
+ one['avg_micro_batch_token_num'] = full['avg_micro_batch_token_num']
141
+ whole = algo.training_step(full)['train/loss']
142
+ split = sum(algo.training_step(one)['train/loss'] for _ in range(2))
143
+ torch.testing.assert_close(whole, split)
144
+ for a, b in zip(torch.autograd.grad(whole, tuple(student.parameters())),
145
+ torch.autograd.grad(split, tuple(student.parameters()))):
146
+ torch.testing.assert_close(a, b)
147
+
148
+ def test_no_alignment_returns_connected_zero_and_bad_shapes_fail(self):
149
+ helper, algo, _, student, _, _, _ = self.pair()
150
+ batch = helper.batch()
151
+ batch['tea_input_ids'][:] = 5
152
+ loss = algo.training_step(batch)['train/loss']
153
+ self.assertEqual(float(loss.detach()), 0.0)
154
+ gradients = torch.autograd.grad(loss, tuple(student.parameters()), allow_unused=True)
155
+ self.assertTrue(all(g is None or not bool(g.any()) for g in gradients))
156
+ batch['teacher_hiddens'] = batch['teacher_hiddens'][:-1]
157
+ with self.assertRaisesRegex(ValueError, 'hidden rows'):
158
+ algo.training_step(batch)
159
+
160
+ def test_mode_registration_arguments_and_no_legacy_alias(self):
161
+ from kdflow.algorithms import ALGO_DICT
162
+ from kdflow.arguments.distillation_args import DistillationArguments
163
+ self.assertIn('simct', ALGO_DICT)
164
+ self.assertNotIn('span_ctkd', ALGO_DICT)
165
+ options = dict(kd_algorithm='simct', kd_ratio=1, kd_loss_fn='rkl')
166
+ self.assertEqual(DistillationArguments(**options).simct_gh_mask_threshold, 0)
167
+ for value in (-1, float('nan'), float('inf')):
168
+ with self.assertRaises(ValueError):
169
+ DistillationArguments(**options, simct_gh_mask_threshold=value)
170
+ with self.assertRaises(ValueError):
171
+ DistillationArguments(kd_algorithm='simct')
172
+
173
+ def test_unsupported_topology_and_non_on_policy_fail(self):
174
+ from kdflow.algorithms.simct import SimCT
175
+ _, algo, _, student, _, teacher, _ = self.pair()
176
+ algo.args.train.num_gpus_per_node = 2
177
+ with self.assertRaisesRegex(ValueError, 'one student rank'):
178
+ SimCT(algo.upstream.strategy, student, teacher, fixtures.Tokenizer(), fixtures.Tokenizer())
179
+ algo.args.train.num_gpus_per_node = 1
180
+ algo.args.scenario = 'off_policy_kd'
181
+ with self.assertRaisesRegex(ValueError, 'on-policy'):
182
+ SimCT(algo.upstream.strategy, student, teacher, fixtures.Tokenizer(), fixtures.Tokenizer())
183
+
184
+
185
+ class SimCTSourceTests(unittest.TestCase):
186
+ def test_pinned_sources_and_frozen_existing_modes(self):
187
+ manifest = json.loads((ROOT / 'docs/SIMCT_SOURCE.json').read_text())
188
+ for record in manifest['files'] + manifest['unchanged_existing_files']:
189
+ self.assertEqual(hashlib.sha256((ROOT / record['path']).read_bytes()).hexdigest(), record['sha256'])
190
+ raw = (ROOT / 'tests/fixtures/simct_upstream/span_ctkd.py.txt').read_bytes()
191
+ port = (ROOT / 'kdflow/algorithms/_simct_upstream/span_ctkd.py').read_bytes()
192
+ self.assertEqual(raw.replace(b'from kdflow.algorithms import register_algorithm',
193
+ b'from . import register_algorithm'), port)
194
+
195
+
196
+ if __name__ == '__main__':
197
+ unittest.main()
tests/test_single_b200.py CHANGED
@@ -55,6 +55,14 @@ class LauncherTests(unittest.TestCase):
55
  self.assertEqual(plan['flags']['train_batch_size'], 512)
56
  self.assertEqual(plan['flags']['kd_algorithm'], 'byte_align_ctkd')
57
 
 
 
 
 
 
 
 
 
58
  def test_mp_mode_presets_preserve_full_workload(self):
59
  baseline = self.plan()['flags']
60
  variants = ['mp-atomic', *['mp-fixed' + str(n) for n in range(1, 6)],
 
55
  self.assertEqual(plan['flags']['train_batch_size'], 512)
56
  self.assertEqual(plan['flags']['kd_algorithm'], 'byte_align_ctkd')
57
 
58
+ def test_simct_preset_changes_only_algorithm_and_adds_disabled_guard(self):
59
+ baseline = self.plan('--algorithm', 'strict-full')['flags']
60
+ flags = self.plan('--algorithm', 'simct')['flags']
61
+ self.assertEqual(flags['kd_algorithm'], 'simct')
62
+ self.assertEqual(flags['simct_gh_mask_threshold'], 0)
63
+ self.assertEqual({k: v for k, v in flags.items() if k not in {'kd_algorithm', 'simct_gh_mask_threshold'}},
64
+ {k: v for k, v in baseline.items() if k != 'kd_algorithm'})
65
+
66
  def test_mp_mode_presets_preserve_full_workload(self):
67
  baseline = self.plan()['flags']
68
  variants = ['mp-atomic', *['mp-fixed' + str(n) for n in range(1, 6)],
tools/launch_single_b200.py CHANGED
@@ -25,7 +25,7 @@ def parser():
25
  default=pathlib.Path('/workspace/storage-shared/nlp/tungks/ctopd_all'))
26
  p.add_argument('--runtime', type=pathlib.Path,
27
  help='Installed prefix; default CTOPD_RUNTIME_PREFIX or /opt/ctopd-b200')
28
- p.add_argument('--algorithm', choices=['strict-top16', 'strict-full', 'mp-atomic',
29
  'mp-fixed1', 'mp-fixed2', 'mp-fixed3', 'mp-fixed4', 'mp-fixed5',
30
  'mp-random1-2', 'mp-random2-5', 'mp-grass-chunk', 'mp-align'], default='strict-top16')
31
  p.add_argument('--mp-chunk-source', choices=['xtoken', 'run'], default='xtoken')
@@ -80,6 +80,8 @@ def build_plan(args):
80
  'teacher_mem_fraction_static': 0.6, 'teacher_offload_tags': 'all',
81
  'chunked_loss_size': 2048, 'use_wandb': False,
82
  }
 
 
83
  if args.algorithm.startswith('mp-'):
84
  name = args.algorithm[3:]
85
  family = 'fixed' if name.startswith('fixed') else 'random' if name.startswith('random') else name.replace('-', '_')
@@ -101,7 +103,9 @@ def build_plan(args):
101
  'source': str(SOURCE), 'flags': flags, 'command': command,
102
  'gradient_accumulation_microsteps': 512 // args.micro_batch,
103
  'expected_runtime_archive_sha256': ARCHIVE_SHA,
104
- 'qualification': ('experimental MP-OPD source port; no B200/full-training qualification'
 
 
105
  if args.algorithm.startswith('mp-') else
106
  'unvalidated single-B200 full workload; no metric-match claim'),
107
  'evaluation_scope': 'MATH500 accuracy over 32 responses/problem; not the full paper benchmark suite'}
 
25
  default=pathlib.Path('/workspace/storage-shared/nlp/tungks/ctopd_all'))
26
  p.add_argument('--runtime', type=pathlib.Path,
27
  help='Installed prefix; default CTOPD_RUNTIME_PREFIX or /opt/ctopd-b200')
28
+ p.add_argument('--algorithm', choices=['strict-top16', 'strict-full', 'simct', 'mp-atomic',
29
  'mp-fixed1', 'mp-fixed2', 'mp-fixed3', 'mp-fixed4', 'mp-fixed5',
30
  'mp-random1-2', 'mp-random2-5', 'mp-grass-chunk', 'mp-align'], default='strict-top16')
31
  p.add_argument('--mp-chunk-source', choices=['xtoken', 'run'], default='xtoken')
 
80
  'teacher_mem_fraction_static': 0.6, 'teacher_offload_tags': 'all',
81
  'chunked_loss_size': 2048, 'use_wandb': False,
82
  }
83
+ if args.algorithm == 'simct':
84
+ flags.update(kd_algorithm='simct', simct_gh_mask_threshold=0.0)
85
  if args.algorithm.startswith('mp-'):
86
  name = args.algorithm[3:]
87
  family = 'fixed' if name.startswith('fixed') else 'random' if name.startswith('random') else name.replace('-', '_')
 
103
  'source': str(SOURCE), 'flags': flags, 'command': command,
104
  'gradient_accumulation_microsteps': 512 // args.micro_batch,
105
  'expected_runtime_archive_sha256': ARCHIVE_SHA,
106
+ 'qualification': ('author-code SimCT port; no B200/full-training qualification'
107
+ if args.algorithm == 'simct' else
108
+ 'experimental MP-OPD source port; no B200/full-training qualification'
109
  if args.algorithm.startswith('mp-') else
110
  'unvalidated single-B200 full workload; no metric-match claim'),
111
  'evaluation_scope': 'MATH500 accuracy over 32 responses/problem; not the full paper benchmark suite'}