codemaivanngu commited on
Commit
0ce2d91
·
unverified ·
1 Parent(s): cfd7f24

feat(repro): add full-workload single-B200 launcher

Browse files
README.md CHANGED
@@ -8,6 +8,8 @@ Start editing directly in `kdflow/algorithms/`, `kdflow/token_alignment.py`, `kd
8
 
9
  Use the existing [qualified B200 runtime on Hugging Face](https://huggingface.co/datasets/codemaivanngu/cross-tokenizer-opd-b200-runtime) with a source overlay rather than reinstalling dependencies after each code edit. See [B200 development instructions](docs/RUNTIME_B200.md).
10
 
 
 
11
  ## Original project documentation
12
 
13
  This repository provides the implementation and example training scripts for `byte_align_ctkd` and `topk_byte_align_ctkd`.
 
8
 
9
  Use the existing [qualified B200 runtime on Hugging Face](https://huggingface.co/datasets/codemaivanngu/cross-tokenizer-opd-b200-runtime) with a source overlay rather than reinstalling dependencies after each code edit. See [B200 development instructions](docs/RUNTIME_B200.md).
10
 
11
+ For a full-workload, one-run-per-B200 adaptation, use [the single-B200 launcher](docs/SINGLE_B200.md). It keeps 512-response rollout/optimizer batches, 8,192 generated tokens, 100 training iterations and 32-sample MATH500 evaluation. There is no reduced-workload mode. Single-B200 full training and paper metric agreement remain unvalidated.
12
+
13
  ## Original project documentation
14
 
15
  This repository provides the implementation and example training scripts for `byte_align_ctkd` and `topk_byte_align_ctkd`.
docs/SINGLE_B200.md ADDED
@@ -0,0 +1,91 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # One full Qwen -> Llama run per B200
2
+
3
+ `tools/launch_single_b200.py` prepares the original workload for one exclusively allocated B200. It has no toy/canary workload and does not install packages or download models/data. `--dry-run` prints configuration only; a normal invocation starts the real 100-iteration run, including full MATH500 evaluation at step 0 and every 10 iterations.
4
+
5
+ ## Preserved training and evaluation settings
6
+
7
+ | Setting | Value |
8
+ | --- | --- |
9
+ | Teacher | Qwen2.5-7B-Instruct, full checkpoint, no quantization |
10
+ | Student | Llama-3.2-3B-Instruct, full parameter training, no extra SFT |
11
+ | Algorithm | Strict top-16 by default; `--algorithm strict-full` selects the full shared-vocabulary baseline |
12
+ | KD | Reverse KL, weight 1, temperature 1 |
13
+ | Global rollout / optimizer batch | 512 / 512 responses |
14
+ | Responses per prompt | 1 |
15
+ | Training iterations | 100 |
16
+ | Prompt / generated / combined maximum tokens | 2,048 / 8,192 / 10,240 |
17
+ | Sampling | Temperature 1, top-p 0.95 |
18
+ | Optimizer | AdamW, LR 1e-6 constant, betas (0.9, 0.98), weight decay 0, grad clip 1 |
19
+ | Computation | BF16, gradient checkpointing, packed samples, loss chunk 2,048 |
20
+ | Checkpoint / MATH500 evaluation interval | 10 / 10 iterations, plus initial evaluation |
21
+ | MATH500 sampling | 32 responses per problem, 8,192 generated tokens, same temperature/top-p |
22
+
23
+ The launcher explicitly uses seed 42, matching the original launcher's code default. This does not establish the seed used for the paper's reported tables. Model commits and dataset hashes used by the original paper were not supplied in the imported launchers; pin and record the revisions actually downloaded for this reproduction.
24
+
25
+ ## Changed execution topology and limits
26
+
27
+ - One node / one B200; one teacher engine with TP=DP=PP=1; one student rollout engine with TP=1; student FSDP2 world size 1.
28
+ - The original launcher uses 8 GPUs, teacher DP=4 and TP=2, and 8 rollout engines. Those GPU counts are not portable to one B200 by changing only `num_gpus_per_node`.
29
+ - Teacher, rollout and student share the GPU in successive phases using the existing sleep/offload implementation. This is full-weight training, not LoRA or a smaller model.
30
+ - Micro-batch remains 4 by default, giving 128 gradient-accumulation microsteps per update. `--micro-batch 1` or `2` changes execution granularity while retaining global batch 512. Global batch is never reduced.
31
+ - Generation/evaluation request concurrency defaults to 8 instead of the source defaults of 512/256. This caps simultaneous requests, not the number of samples. Preprocessing uses 8 CPU workers instead of 32. The evaluation request timeout is disabled by default (`--eval-timeout 0`) to avoid imposing the original 2,400-second request limit on slower single-GPU execution.
32
+ - The launcher starts a fresh local Ray instance, keeps output directories separate and never attaches to an unrelated Ray cluster. Local hostname/IP and loopback connections are exempted from the inherited HTTP proxy for Ray/SGLang; external network proxy settings are retained.
33
+
34
+ The inspected code divides each micro-batch's summed loss by average response-token count, divides backward loss by gradient accumulation, and averages distributed gradients. On fixed responses, this preserves the batch token-weighted objective across these world/micro-batch sizes in exact arithmetic. A standard-library scalar regression exercises the actual preparation/backward methods with uneven response lengths for (world, micro) = (8,4), (1,4), (1,1). This checks normalization algebra; it does not test CUDA tensors, FSDP gradient parity or training trajectories.
35
+
36
+ One B200 therefore defines an **unvalidated topology adaptation**, not a bitwise reproduction of eight H20 GPUs. Different random-number consumption, request scheduling, tensor-parallel arithmetic and BF16 reduction order can change sampled responses and final scores. The full workload must still be run and compared with the paper before claiming agreement. No runtime or throughput measurement of this full configuration is available yet.
37
+
38
+ CPU RAM and `/dev/shm` also matter: the teacher forwards a whole rollout batch and retains response hidden states in CPU/Ray memory. One teacher DP engine concentrates work previously split over four. Existing CPU weight/optimizer offload adds host-memory pressure. Lowering request concurrency alone does not bound the retained 512-response hidden-state memory. Do not infer host-memory sufficiency from B200 VRAM alone.
39
+
40
+ ## Required files
41
+
42
+ With `ROOT=/workspace/storage-shared/nlp/tungks/ctopd_all`, prepare:
43
+
44
+ ```text
45
+ $ROOT/runtime/python-b200.sh
46
+ $ROOT/models/Qwen2.5-7B-Instruct/ # complete official HF snapshot + tokenizer
47
+ $ROOT/models/Llama-3.2-3B-Instruct/ # complete official HF snapshot + tokenizer
48
+ $ROOT/data/Train/math_code_20k/train.parquet
49
+ $ROOT/data/math500/test.jsonl
50
+ ```
51
+
52
+ Use `--runtime /actual/installed/prefix` if the runtime was installed elsewhere. The runtime tar.gz alone is not an installed prefix. Models/data must be downloaded directly on the runtime host; the Llama HF token must belong to an account with approved model access.
53
+
54
+ ## Print the complete plan, then run it unchanged
55
+
56
+ From this checkout on the company host:
57
+
58
+ ```bash
59
+ python3 tools/launch_single_b200.py --run-id qwen-llama-top16-s42 --dry-run
60
+ ```
61
+
62
+ Inside a pod exposing exactly the single B200 allocated to this run:
63
+
64
+ ```bash
65
+ set -euo pipefail
66
+ python3 tools/launch_single_b200.py --run-id qwen-llama-top16-s42
67
+ ```
68
+
69
+ On a shared multi-GPU host, set `CUDA_VISIBLE_DEVICES` to the **allocated GPU UUID** for the command; do not select an arbitrary idle device. The launcher checks that the runtime sees exactly one B200 before starting training. Visibility is not proof of allocation: obtain that allocation first. Separate pods are preferable for independent runs.
70
+
71
+ The launcher's preflight checks model/tokenizer files, all referenced safetensors shards, GPU visibility, the 20k-row source training pool and all 500 MATH500 problems. It then records source commit, effective arguments, data SHA-256 and preflight observations in `$ROOT/runs/<run-id>/launch.json`. It refuses a dirty source checkout, an existing output directory, or implicit resume from an unrelated checkpoint. Checkpoints/results go under that run's `checkpoints/`. Secrets and the full process environment are not recorded. It does not verify all model tensor hashes or independently requalify the installed runtime; the archive hash in the plan is the expected identity.
72
+
73
+ To collect the full shared-vocabulary comparison, allocate a different GPU/run and use a distinct run-id:
74
+
75
+ ```bash
76
+ python3 tools/launch_single_b200.py --algorithm strict-full --run-id qwen-llama-full-s42
77
+ ```
78
+
79
+ These are two independent complete runs; never run both simultaneously on the same allocated GPU. Each invocation also performs the initial full MATH500 evaluation before the first train iteration; lack of optimizer logs during that evaluation does not mean training is stalled.
80
+
81
+ ## Compare the right quantities
82
+
83
+ For MATH500, compare `eval/accuracy * 100` with the paper's accuracy over 32 samples, **not** `eval/pass@32`. Inspect the saved generation results for missing/failed requests before accepting a score. The current launcher's evaluation covers MATH500 only. Reproducing the paper's Math, Code and Full aggregate scores also needs the other six math benchmarks and all three code benchmarks, the same step-100 checkpoint, 32/8 samples respectively, and matching scoring scripts. This launcher alone does not implement that complete benchmark matrix.
84
+
85
+ ## Validation performed locally
86
+
87
+ ```bash
88
+ PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tests -p test_single_b200.py -v
89
+ ```
90
+
91
+ Six lightweight tests cover the full-workload contract, one-GPU topology, flag compatibility with the actual argument dataclasses, safe dry-run, preservation of existing runs, and token-normalization algebra. No GPU job, model download, company installation or full training was executed while preparing this launcher.
docs/UPSTREAM.md CHANGED
@@ -8,7 +8,7 @@
8
  - The source service did not expose an upstream Git commit. The local import commit starts a new history and does not claim to preserve upstream Git history.
9
  - `docs/upstream-source-manifest.json` records original byte sizes and hashes. It describes the initial import; later development commits may intentionally change those files.
10
 
11
- The initial commit preserves the imported implementation. Subsequent setup changes update documentation and line-ending attributes only. Datasets and figures were excluded from the source snapshot, and no runtime archives, model weights, credentials, account metadata or experiment outputs are included.
12
 
13
  ## License metadata
14
 
 
8
  - The source service did not expose an upstream Git commit. The local import commit starts a new history and does not claim to preserve upstream Git history.
9
  - `docs/upstream-source-manifest.json` records original byte sizes and hashes. It describes the initial import; later development commits may intentionally change those files.
10
 
11
+ The initial commit preserves the imported implementation. Local additions provide documentation, line-ending attributes, a single-B200 launcher and its offline tests. Imported `kdflow/` code and the two original training scripts remain unchanged. Datasets and figures were excluded from the source snapshot, and no runtime archives, model weights, credentials, account metadata or experiment outputs are included.
12
 
13
  ## License metadata
14
 
tests/test_single_b200.py ADDED
@@ -0,0 +1,120 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Offline checks for the real-workload launcher; no ML imports or GPU required."""
2
+ import ast
3
+ import importlib.util
4
+ import json
5
+ import pathlib
6
+ import subprocess
7
+ import sys
8
+ import tempfile
9
+ import types
10
+ import unittest
11
+
12
+ ROOT = pathlib.Path(__file__).resolve().parents[1]
13
+ SPEC = importlib.util.spec_from_file_location('launcher', ROOT / 'tools/launch_single_b200.py')
14
+ launcher = importlib.util.module_from_spec(SPEC)
15
+ SPEC.loader.exec_module(launcher)
16
+
17
+
18
+ def method_without_ml_imports(relative, name):
19
+ tree = ast.parse((ROOT / relative).read_text())
20
+ method = next(node for node in ast.walk(tree) if isinstance(node, ast.FunctionDef) and node.name == name)
21
+ module = ast.Module(body=[ast.ImportFrom(module='__future__', names=[ast.alias(name='annotations')], level=0), method], type_ignores=[])
22
+ scope = {}
23
+ exec(compile(ast.fix_missing_locations(module), relative, 'exec'), scope)
24
+ return scope[name]
25
+
26
+
27
+ class LauncherTests(unittest.TestCase):
28
+ def plan(self, *arguments):
29
+ return launcher.build_plan(launcher.parser().parse_args(['--run-id', 'test-real', *arguments]))
30
+
31
+ def test_real_workload_and_one_gpu_topology(self):
32
+ plan = self.plan()
33
+ flags = plan['flags']
34
+ for key, value in {'train_batch_size': 512, 'rollout_batch_size': 512, 'max_steps': 100,
35
+ 'generate_max_len': 8192, 'prompt_max_len': 2048, 'max_len': 10240,
36
+ 'micro_train_batch_size': 4, 'ctkd_topk': 16, 'eval_math_n_samples': 32,
37
+ 'eval_math_max_tokens': 8192, 'eval_steps': 10, 'save_steps': 10}.items():
38
+ self.assertEqual(flags[key], value, key)
39
+ for key in ['num_nodes', 'num_gpus_per_node', 'teacher_dp_size', 'teacher_tp_size',
40
+ 'teacher_pp_size', 'rollout_num_engines', 'rollout_tp_size']:
41
+ self.assertEqual(flags[key], 1, key)
42
+ self.assertEqual(plan['gradient_accumulation_microsteps'], 128)
43
+
44
+ def test_flags_exist_in_actual_argument_dataclasses(self):
45
+ fields = set()
46
+ for path in (ROOT / 'kdflow/arguments').glob('*_args.py'):
47
+ for node in ast.walk(ast.parse(path.read_text())):
48
+ if isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name):
49
+ fields.add(node.target.id)
50
+ self.assertFalse(set(self.plan()['flags']) - fields)
51
+
52
+ def test_micro_batch_changes_preserve_global_workload_and_baseline(self):
53
+ plan = self.plan('--micro-batch', '1', '--algorithm', 'strict-full')
54
+ self.assertEqual(plan['gradient_accumulation_microsteps'], 512)
55
+ self.assertEqual(plan['flags']['train_batch_size'], 512)
56
+ self.assertEqual(plan['flags']['kd_algorithm'], 'byte_align_ctkd')
57
+
58
+ def test_dry_run_creates_no_files(self):
59
+ with tempfile.TemporaryDirectory() as folder:
60
+ destination = pathlib.Path(folder) / 'absent'
61
+ raw = subprocess.check_output([sys.executable, str(ROOT / 'tools/launch_single_b200.py'),
62
+ '--root', str(destination), '--dry-run'], text=True)
63
+ self.assertEqual(json.loads(raw)['flags']['max_steps'], 100)
64
+ self.assertFalse(destination.exists())
65
+
66
+ def test_existing_output_and_path_traversal_are_refused(self):
67
+ with self.assertRaises(ValueError):
68
+ self.plan('--run-id', '../other-run')
69
+ with tempfile.TemporaryDirectory() as folder:
70
+ plan = self.plan('--root', folder)
71
+ for key in ['student_name_or_path', 'teacher_name_or_path']:
72
+ model = pathlib.Path(plan['flags'][key])
73
+ model.mkdir(parents=True)
74
+ for name in ['config.json', 'tokenizer_config.json', 'tokenizer.json', 'model.safetensors']:
75
+ (model / name).write_text('{}')
76
+ for path in [pathlib.Path(plan['runtime']) / 'python-b200.sh',
77
+ pathlib.Path(plan['flags']['train_dataset_path']),
78
+ pathlib.Path(plan['flags']['eval_math_dataset_path'])]:
79
+ path.parent.mkdir(parents=True, exist_ok=True)
80
+ path.touch()
81
+ pathlib.Path(plan['output']).mkdir(parents=True)
82
+ with self.assertRaises(FileExistsError):
83
+ launcher.validate_files(plan)
84
+
85
+ def test_actual_token_normalization_across_world_and_micro_batch_sizes(self):
86
+ prepare = method_without_ml_imports('kdflow/trainer/on_policy_kd_trainer.py', '_prepare_global_batches')
87
+ backward = method_without_ml_imports('kdflow/backend/fsdp/fsdp_strategy.py', 'backward')
88
+ # Unequal sequence lengths make an incorrect mean-of-means observable.
89
+ lengths = [1 + (index * 13 % 97) for index in range(512)]
90
+ gradients = [((index % 7) - 3) * length for index, length in enumerate(lengths)]
91
+ reference = sum(gradients) / sum(lengths)
92
+
93
+ class Mask:
94
+ def __init__(self, count): self.count = count
95
+ def sum(self): return self.count
96
+
97
+ class Loss:
98
+ def __init__(self, value, sink): self.value, self.sink = value, sink
99
+ def __truediv__(self, divisor): return Loss(self.value / divisor, self.sink)
100
+ def backward(self): self.sink.append(self.value)
101
+
102
+ for world, micro in [(8, 4), (1, 4), (1, 1)]:
103
+ batches = [{'stu_loss_mask': Mask(sum(lengths[i:i + micro])),
104
+ 'gradient': sum(gradients[i:i + micro])} for i in range(0, 512, micro)]
105
+ trainer = types.SimpleNamespace(args=types.SimpleNamespace(train=types.SimpleNamespace(use_dynamic_bsz=False)))
106
+ batches = prepare(trainer, batches, 512 // micro)[0]
107
+ rank_gradients = []
108
+ for rank in range(world):
109
+ strategy = types.SimpleNamespace(step=0, accumulated_gradient=512 // micro // world)
110
+ sink = []
111
+ for batch in batches[rank::world]:
112
+ loss = Loss(batch['gradient'] / batch['avg_micro_batch_token_num'], sink)
113
+ backward(strategy, loss, None, None)
114
+ self.assertEqual(strategy.step, 0)
115
+ rank_gradients.append(sum(sink))
116
+ self.assertAlmostEqual(sum(rank_gradients) / world, reference, places=12)
117
+
118
+
119
+ if __name__ == '__main__':
120
+ unittest.main()
tools/launch_single_b200.py ADDED
@@ -0,0 +1,169 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Launch the real Qwen -> Llama training configuration on one allocated B200.
3
+
4
+ Standard library only. --dry-run prints a plan without importing the runtime,
5
+ checking GPUs, writing files or starting training. There is no reduced workload.
6
+ """
7
+ import argparse
8
+ import datetime
9
+ import hashlib
10
+ import json
11
+ import os
12
+ import pathlib
13
+ import re
14
+ import shlex
15
+ import socket
16
+ import subprocess
17
+
18
+ SOURCE = pathlib.Path(__file__).resolve().parents[1]
19
+ ARCHIVE_SHA = '084c321c0b7af1cd3a36d12a595c623567d4a37525aa0eecbc3933aa7f9b15c3'
20
+
21
+
22
+ def parser():
23
+ p = argparse.ArgumentParser(description=__doc__)
24
+ p.add_argument('--root', type=pathlib.Path,
25
+ default=pathlib.Path('/workspace/storage-shared/nlp/tungks/ctopd_all'))
26
+ p.add_argument('--runtime', type=pathlib.Path, help='Installed prefix containing python-b200.sh')
27
+ p.add_argument('--algorithm', choices=['strict-top16', 'strict-full'], default='strict-top16')
28
+ p.add_argument('--seed', type=int, default=42, help='Original launcher default, not a verified paper seed')
29
+ p.add_argument('--micro-batch', type=int, choices=[1, 2, 4], default=4,
30
+ help='Original launcher uses 4; 1/2 retain global batch through accumulation')
31
+ p.add_argument('--concurrency', type=int, default=8, help='Transport concurrency; samples/batch stay unchanged')
32
+ p.add_argument('--eval-timeout', type=int, default=0, help='Seconds per evaluation request; 0 disables timeout')
33
+ p.add_argument('--run-id', help='New output directory name; existing runs are never resumed')
34
+ p.add_argument('--dry-run', action='store_true')
35
+ return p
36
+
37
+
38
+ def build_plan(args):
39
+ if args.concurrency < 1 or args.eval_timeout < 0:
40
+ raise ValueError('Concurrency must be positive and eval timeout nonnegative')
41
+ run_id = args.run_id or (datetime.datetime.now(datetime.timezone.utc).strftime('%Y%m%dT%H%M%S%fZ')
42
+ + '-' + args.algorithm + '-s' + str(args.seed))
43
+ if not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9_.-]{0,95}', run_id):
44
+ raise ValueError('run-id must be a single directory name starting with a letter or digit')
45
+ root = args.root.expanduser().resolve()
46
+ runtime = (args.runtime or root / 'runtime').expanduser().resolve()
47
+ output = root / 'runs' / run_id
48
+ flags = {
49
+ 'num_nodes': 1, 'num_gpus_per_node': 1, 'backend': 'fsdp2',
50
+ 'train_batch_size': 512, 'micro_train_batch_size': args.micro_batch,
51
+ 'learning_rate': '1e-6', 'lr_scheduler': 'constant', 'lr_warmup_ratio': 0.05,
52
+ 'adam_betas': '(0.9, 0.98)', 'weight_decay': 0, 'max_norm': 1.0,
53
+ 'num_epochs': 10, 'max_steps': 100, 'seed': args.seed,
54
+ 'save_path': str(output / 'checkpoints'), 'ckpt_path': str(output / 'unused-resume'),
55
+ 'bf16': True, 'gradient_checkpointing': True, 'enable_sleep': True,
56
+ 'use_dynamic_bsz': False,
57
+ 'student_name_or_path': str(root / 'models/Llama-3.2-3B-Instruct'),
58
+ 'teacher_name_or_path': str(root / 'models/Qwen2.5-7B-Instruct'),
59
+ 'enable_thinking': False,
60
+ 'rollout_batch_size': 512, 'rollout_num_engines': 1, 'rollout_tp_size': 1,
61
+ 'rollout_engine_concurrency': args.concurrency, 'rollout_mem_fraction_static': 0.6,
62
+ 'n_samples_per_prompt': 1, 'generate_max_len': 8192, 'temperature': 1.0, 'top_p': 0.95,
63
+ 'train_dataset_path': str(root / 'data/Train/math_code_20k/train.parquet'),
64
+ 'max_len': 10240, 'prompt_max_len': 2048, 'input_key': 'prompt',
65
+ 'label_key': 'reward_model.ground_truth', 'apply_chat_template': True,
66
+ 'preprocess_num_workers': 8, 'packing_samples': True,
67
+ 'eval_math_dataset_path': str(root / 'data/math500/test.jsonl'),
68
+ 'eval_math_max_tokens': 8192, 'eval_math_max_concurrent': args.concurrency,
69
+ 'eval_math_timeout': args.eval_timeout, 'eval_math_n_samples': 32,
70
+ 'eval_steps': 10, 'save_steps': 10, 'logging_steps': 1,
71
+ 'kd_ratio': 1.0, 'kd_loss_fn': 'rkl', 'kd_temperature': 1.0,
72
+ 'kd_algorithm': 'topk_byte_align_ctkd' if args.algorithm == 'strict-top16' else 'byte_align_ctkd',
73
+ 'ctkd_topk': 16, 'teacher_dp_size': 1, 'teacher_tp_size': 1, 'teacher_pp_size': 1,
74
+ 'teacher_mem_fraction_static': 0.6, 'teacher_offload_tags': 'all',
75
+ 'chunked_loss_size': 2048, 'use_wandb': False,
76
+ }
77
+ command = ['bash', str(runtime / 'python-b200.sh'), '-u', '-m', 'kdflow.cli.train_kd_on_policy']
78
+ for key, value in flags.items():
79
+ command.extend(['--' + key, str(value)])
80
+ return {'algorithm': args.algorithm, 'output': str(output), 'runtime': str(runtime),
81
+ 'source': str(SOURCE), 'flags': flags, 'command': command,
82
+ 'gradient_accumulation_microsteps': 512 // args.micro_batch,
83
+ 'expected_runtime_archive_sha256': ARCHIVE_SHA,
84
+ 'qualification': 'unvalidated single-B200 full workload; no metric-match claim',
85
+ 'evaluation_scope': 'MATH500 accuracy over 32 responses/problem; not the full paper benchmark suite'}
86
+
87
+
88
+ def run_environment():
89
+ env = os.environ.copy()
90
+ env.update(RAY_ADDRESS='local', HF_HUB_OFFLINE='1', TRANSFORMERS_OFFLINE='1',
91
+ PYTHONUNBUFFERED='1', PYTHONPATH=str(SOURCE))
92
+ # Company HTTP proxies must not intercept local Ray/SGLang connections.
93
+ bypass = [env.get('NO_PROXY', ''), env.get('no_proxy', ''), 'localhost', '127.0.0.1', '::1', socket.gethostname()]
94
+ try:
95
+ bypass.extend(socket.gethostbyname_ex(socket.gethostname())[2])
96
+ except OSError:
97
+ pass
98
+ env['NO_PROXY'] = env['no_proxy'] = ','.join(item for item in bypass if item)
99
+ return env
100
+
101
+
102
+ def validate_files(plan):
103
+ flags = plan['flags']
104
+ required = [pathlib.Path(plan['runtime']) / 'python-b200.sh']
105
+ required += [pathlib.Path(flags[key]) for key in ['train_dataset_path', 'eval_math_dataset_path']]
106
+ for key in ['student_name_or_path', 'teacher_name_or_path']:
107
+ model = pathlib.Path(flags[key])
108
+ required += [model / 'config.json', model / 'tokenizer_config.json', model / 'tokenizer.json']
109
+ index = model / 'model.safetensors.index.json'
110
+ if index.is_file():
111
+ shards = set(json.loads(index.read_text())['weight_map'].values())
112
+ for shard in shards:
113
+ if pathlib.PurePosixPath(shard).name != shard:
114
+ raise ValueError('Unexpected checkpoint shard path: ' + shard)
115
+ required.append(model / shard)
116
+ else:
117
+ required.append(model / 'model.safetensors')
118
+ for path in required:
119
+ if not path.is_file():
120
+ raise FileNotFoundError(path)
121
+ if pathlib.Path(plan['output']).exists():
122
+ raise FileExistsError('Choose a new run-id; refusing existing output: ' + plan['output'])
123
+
124
+
125
+ def main(argv=None):
126
+ args = parser().parse_args(argv)
127
+ plan = build_plan(args)
128
+ if args.dry_run:
129
+ print(json.dumps(plan, indent=2))
130
+ return
131
+ validate_files(plan)
132
+ env = run_environment()
133
+ commit = subprocess.check_output(['git', '-C', str(SOURCE), 'rev-parse', 'HEAD'], text=True).strip()
134
+ dirty = subprocess.check_output(['git', '-C', str(SOURCE), 'status', '--porcelain'], text=True).strip()
135
+ if dirty:
136
+ raise RuntimeError('Commit source changes before starting a recorded reproduction run')
137
+ # This validates real model/data availability and GPU visibility, not a toy training workload.
138
+ probe = '''import json, pathlib, torch, pyarrow.parquet as pq
139
+ assert torch.cuda.device_count() == 1, 'Expose exactly the one GPU allocated to this run'
140
+ name = torch.cuda.get_device_name(0)
141
+ assert 'B200' in name, 'This launcher targets a B200'
142
+ train, evaluation = __import__('sys').argv[1:]
143
+ metadata = pq.read_metadata(train)
144
+ assert metadata.num_rows == 20000, 'Expected authors source pool of 20,000 prompts before tokenizer filtering'
145
+ assert {'prompt', 'reward_model'} <= set(pq.read_schema(train).names)
146
+ rows = [json.loads(line) for line in pathlib.Path(evaluation).read_text().splitlines() if line.strip()]
147
+ assert len(rows) == 500 and all({'problem', 'answer'} <= row.keys() for row in rows)
148
+ print(json.dumps({'gpu': name, 'torch': torch.__version__, 'cuda': torch.version.cuda, 'source_rows': metadata.num_rows, 'math500_rows': len(rows)}))
149
+ '''
150
+ result = subprocess.check_output(['bash', str(pathlib.Path(plan['runtime']) / 'python-b200.sh'),
151
+ '-c', probe, plan['flags']['train_dataset_path'], plan['flags']['eval_math_dataset_path']],
152
+ env=env, text=True)
153
+ plan['source_commit'] = commit
154
+ plan['preflight'] = json.loads(result.strip().splitlines()[-1])
155
+ plan['input_sha256'] = {}
156
+ for key in ['train_dataset_path', 'eval_math_dataset_path']:
157
+ path = pathlib.Path(plan['flags'][key])
158
+ plan['input_sha256'][key] = hashlib.sha256(path.read_bytes()).hexdigest()
159
+ output = pathlib.Path(plan['output'])
160
+ output.mkdir(parents=True, exist_ok=False)
161
+ (output / 'launch.json').write_text(json.dumps(plan, indent=2) + '\n')
162
+ (output / 'command.sh').write_text(shlex.join(plan['command']) + '\n')
163
+ print('RUN_DIRECTORY=' + str(output), flush=True)
164
+ os.chdir(SOURCE)
165
+ os.execvpe(plan['command'][0], plan['command'], env)
166
+
167
+
168
+ if __name__ == '__main__':
169
+ main()