feat(repro): add full-workload single-B200 launcher
Browse files- README.md +2 -0
- docs/SINGLE_B200.md +91 -0
- docs/UPSTREAM.md +1 -1
- tests/test_single_b200.py +120 -0
- tools/launch_single_b200.py +169 -0
README.md
CHANGED
|
@@ -8,6 +8,8 @@ Start editing directly in `kdflow/algorithms/`, `kdflow/token_alignment.py`, `kd
|
|
| 8 |
|
| 9 |
Use the existing [qualified B200 runtime on Hugging Face](https://huggingface.co/datasets/codemaivanngu/cross-tokenizer-opd-b200-runtime) with a source overlay rather than reinstalling dependencies after each code edit. See [B200 development instructions](docs/RUNTIME_B200.md).
|
| 10 |
|
|
|
|
|
|
|
| 11 |
## Original project documentation
|
| 12 |
|
| 13 |
This repository provides the implementation and example training scripts for `byte_align_ctkd` and `topk_byte_align_ctkd`.
|
|
|
|
| 8 |
|
| 9 |
Use the existing [qualified B200 runtime on Hugging Face](https://huggingface.co/datasets/codemaivanngu/cross-tokenizer-opd-b200-runtime) with a source overlay rather than reinstalling dependencies after each code edit. See [B200 development instructions](docs/RUNTIME_B200.md).
|
| 10 |
|
| 11 |
+
For a full-workload, one-run-per-B200 adaptation, use [the single-B200 launcher](docs/SINGLE_B200.md). It keeps 512-response rollout/optimizer batches, 8,192 generated tokens, 100 training iterations and 32-sample MATH500 evaluation. There is no reduced-workload mode. Single-B200 full training and paper metric agreement remain unvalidated.
|
| 12 |
+
|
| 13 |
## Original project documentation
|
| 14 |
|
| 15 |
This repository provides the implementation and example training scripts for `byte_align_ctkd` and `topk_byte_align_ctkd`.
|
docs/SINGLE_B200.md
ADDED
|
@@ -0,0 +1,91 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# One full Qwen -> Llama run per B200
|
| 2 |
+
|
| 3 |
+
`tools/launch_single_b200.py` prepares the original workload for one exclusively allocated B200. It has no toy/canary workload and does not install packages or download models/data. `--dry-run` prints configuration only; a normal invocation starts the real 100-iteration run, including full MATH500 evaluation at step 0 and every 10 iterations.
|
| 4 |
+
|
| 5 |
+
## Preserved training and evaluation settings
|
| 6 |
+
|
| 7 |
+
| Setting | Value |
|
| 8 |
+
| --- | --- |
|
| 9 |
+
| Teacher | Qwen2.5-7B-Instruct, full checkpoint, no quantization |
|
| 10 |
+
| Student | Llama-3.2-3B-Instruct, full parameter training, no extra SFT |
|
| 11 |
+
| Algorithm | Strict top-16 by default; `--algorithm strict-full` selects the full shared-vocabulary baseline |
|
| 12 |
+
| KD | Reverse KL, weight 1, temperature 1 |
|
| 13 |
+
| Global rollout / optimizer batch | 512 / 512 responses |
|
| 14 |
+
| Responses per prompt | 1 |
|
| 15 |
+
| Training iterations | 100 |
|
| 16 |
+
| Prompt / generated / combined maximum tokens | 2,048 / 8,192 / 10,240 |
|
| 17 |
+
| Sampling | Temperature 1, top-p 0.95 |
|
| 18 |
+
| Optimizer | AdamW, LR 1e-6 constant, betas (0.9, 0.98), weight decay 0, grad clip 1 |
|
| 19 |
+
| Computation | BF16, gradient checkpointing, packed samples, loss chunk 2,048 |
|
| 20 |
+
| Checkpoint / MATH500 evaluation interval | 10 / 10 iterations, plus initial evaluation |
|
| 21 |
+
| MATH500 sampling | 32 responses per problem, 8,192 generated tokens, same temperature/top-p |
|
| 22 |
+
|
| 23 |
+
The launcher explicitly uses seed 42, matching the original launcher's code default. This does not establish the seed used for the paper's reported tables. Model commits and dataset hashes used by the original paper were not supplied in the imported launchers; pin and record the revisions actually downloaded for this reproduction.
|
| 24 |
+
|
| 25 |
+
## Changed execution topology and limits
|
| 26 |
+
|
| 27 |
+
- One node / one B200; one teacher engine with TP=DP=PP=1; one student rollout engine with TP=1; student FSDP2 world size 1.
|
| 28 |
+
- The original launcher uses 8 GPUs, teacher DP=4 and TP=2, and 8 rollout engines. Those GPU counts are not portable to one B200 by changing only `num_gpus_per_node`.
|
| 29 |
+
- Teacher, rollout and student share the GPU in successive phases using the existing sleep/offload implementation. This is full-weight training, not LoRA or a smaller model.
|
| 30 |
+
- Micro-batch remains 4 by default, giving 128 gradient-accumulation microsteps per update. `--micro-batch 1` or `2` changes execution granularity while retaining global batch 512. Global batch is never reduced.
|
| 31 |
+
- Generation/evaluation request concurrency defaults to 8 instead of the source defaults of 512/256. This caps simultaneous requests, not the number of samples. Preprocessing uses 8 CPU workers instead of 32. The evaluation request timeout is disabled by default (`--eval-timeout 0`) to avoid imposing the original 2,400-second request limit on slower single-GPU execution.
|
| 32 |
+
- The launcher starts a fresh local Ray instance, keeps output directories separate and never attaches to an unrelated Ray cluster. Local hostname/IP and loopback connections are exempted from the inherited HTTP proxy for Ray/SGLang; external network proxy settings are retained.
|
| 33 |
+
|
| 34 |
+
The inspected code divides each micro-batch's summed loss by average response-token count, divides backward loss by gradient accumulation, and averages distributed gradients. On fixed responses, this preserves the batch token-weighted objective across these world/micro-batch sizes in exact arithmetic. A standard-library scalar regression exercises the actual preparation/backward methods with uneven response lengths for (world, micro) = (8,4), (1,4), (1,1). This checks normalization algebra; it does not test CUDA tensors, FSDP gradient parity or training trajectories.
|
| 35 |
+
|
| 36 |
+
One B200 therefore defines an **unvalidated topology adaptation**, not a bitwise reproduction of eight H20 GPUs. Different random-number consumption, request scheduling, tensor-parallel arithmetic and BF16 reduction order can change sampled responses and final scores. The full workload must still be run and compared with the paper before claiming agreement. No runtime or throughput measurement of this full configuration is available yet.
|
| 37 |
+
|
| 38 |
+
CPU RAM and `/dev/shm` also matter: the teacher forwards a whole rollout batch and retains response hidden states in CPU/Ray memory. One teacher DP engine concentrates work previously split over four. Existing CPU weight/optimizer offload adds host-memory pressure. Lowering request concurrency alone does not bound the retained 512-response hidden-state memory. Do not infer host-memory sufficiency from B200 VRAM alone.
|
| 39 |
+
|
| 40 |
+
## Required files
|
| 41 |
+
|
| 42 |
+
With `ROOT=/workspace/storage-shared/nlp/tungks/ctopd_all`, prepare:
|
| 43 |
+
|
| 44 |
+
```text
|
| 45 |
+
$ROOT/runtime/python-b200.sh
|
| 46 |
+
$ROOT/models/Qwen2.5-7B-Instruct/ # complete official HF snapshot + tokenizer
|
| 47 |
+
$ROOT/models/Llama-3.2-3B-Instruct/ # complete official HF snapshot + tokenizer
|
| 48 |
+
$ROOT/data/Train/math_code_20k/train.parquet
|
| 49 |
+
$ROOT/data/math500/test.jsonl
|
| 50 |
+
```
|
| 51 |
+
|
| 52 |
+
Use `--runtime /actual/installed/prefix` if the runtime was installed elsewhere. The runtime tar.gz alone is not an installed prefix. Models/data must be downloaded directly on the runtime host; the Llama HF token must belong to an account with approved model access.
|
| 53 |
+
|
| 54 |
+
## Print the complete plan, then run it unchanged
|
| 55 |
+
|
| 56 |
+
From this checkout on the company host:
|
| 57 |
+
|
| 58 |
+
```bash
|
| 59 |
+
python3 tools/launch_single_b200.py --run-id qwen-llama-top16-s42 --dry-run
|
| 60 |
+
```
|
| 61 |
+
|
| 62 |
+
Inside a pod exposing exactly the single B200 allocated to this run:
|
| 63 |
+
|
| 64 |
+
```bash
|
| 65 |
+
set -euo pipefail
|
| 66 |
+
python3 tools/launch_single_b200.py --run-id qwen-llama-top16-s42
|
| 67 |
+
```
|
| 68 |
+
|
| 69 |
+
On a shared multi-GPU host, set `CUDA_VISIBLE_DEVICES` to the **allocated GPU UUID** for the command; do not select an arbitrary idle device. The launcher checks that the runtime sees exactly one B200 before starting training. Visibility is not proof of allocation: obtain that allocation first. Separate pods are preferable for independent runs.
|
| 70 |
+
|
| 71 |
+
The launcher's preflight checks model/tokenizer files, all referenced safetensors shards, GPU visibility, the 20k-row source training pool and all 500 MATH500 problems. It then records source commit, effective arguments, data SHA-256 and preflight observations in `$ROOT/runs/<run-id>/launch.json`. It refuses a dirty source checkout, an existing output directory, or implicit resume from an unrelated checkpoint. Checkpoints/results go under that run's `checkpoints/`. Secrets and the full process environment are not recorded. It does not verify all model tensor hashes or independently requalify the installed runtime; the archive hash in the plan is the expected identity.
|
| 72 |
+
|
| 73 |
+
To collect the full shared-vocabulary comparison, allocate a different GPU/run and use a distinct run-id:
|
| 74 |
+
|
| 75 |
+
```bash
|
| 76 |
+
python3 tools/launch_single_b200.py --algorithm strict-full --run-id qwen-llama-full-s42
|
| 77 |
+
```
|
| 78 |
+
|
| 79 |
+
These are two independent complete runs; never run both simultaneously on the same allocated GPU. Each invocation also performs the initial full MATH500 evaluation before the first train iteration; lack of optimizer logs during that evaluation does not mean training is stalled.
|
| 80 |
+
|
| 81 |
+
## Compare the right quantities
|
| 82 |
+
|
| 83 |
+
For MATH500, compare `eval/accuracy * 100` with the paper's accuracy over 32 samples, **not** `eval/pass@32`. Inspect the saved generation results for missing/failed requests before accepting a score. The current launcher's evaluation covers MATH500 only. Reproducing the paper's Math, Code and Full aggregate scores also needs the other six math benchmarks and all three code benchmarks, the same step-100 checkpoint, 32/8 samples respectively, and matching scoring scripts. This launcher alone does not implement that complete benchmark matrix.
|
| 84 |
+
|
| 85 |
+
## Validation performed locally
|
| 86 |
+
|
| 87 |
+
```bash
|
| 88 |
+
PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tests -p test_single_b200.py -v
|
| 89 |
+
```
|
| 90 |
+
|
| 91 |
+
Six lightweight tests cover the full-workload contract, one-GPU topology, flag compatibility with the actual argument dataclasses, safe dry-run, preservation of existing runs, and token-normalization algebra. No GPU job, model download, company installation or full training was executed while preparing this launcher.
|
docs/UPSTREAM.md
CHANGED
|
@@ -8,7 +8,7 @@
|
|
| 8 |
- The source service did not expose an upstream Git commit. The local import commit starts a new history and does not claim to preserve upstream Git history.
|
| 9 |
- `docs/upstream-source-manifest.json` records original byte sizes and hashes. It describes the initial import; later development commits may intentionally change those files.
|
| 10 |
|
| 11 |
-
The initial commit preserves the imported implementation.
|
| 12 |
|
| 13 |
## License metadata
|
| 14 |
|
|
|
|
| 8 |
- The source service did not expose an upstream Git commit. The local import commit starts a new history and does not claim to preserve upstream Git history.
|
| 9 |
- `docs/upstream-source-manifest.json` records original byte sizes and hashes. It describes the initial import; later development commits may intentionally change those files.
|
| 10 |
|
| 11 |
+
The initial commit preserves the imported implementation. Local additions provide documentation, line-ending attributes, a single-B200 launcher and its offline tests. Imported `kdflow/` code and the two original training scripts remain unchanged. Datasets and figures were excluded from the source snapshot, and no runtime archives, model weights, credentials, account metadata or experiment outputs are included.
|
| 12 |
|
| 13 |
## License metadata
|
| 14 |
|
tests/test_single_b200.py
ADDED
|
@@ -0,0 +1,120 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Offline checks for the real-workload launcher; no ML imports or GPU required."""
|
| 2 |
+
import ast
|
| 3 |
+
import importlib.util
|
| 4 |
+
import json
|
| 5 |
+
import pathlib
|
| 6 |
+
import subprocess
|
| 7 |
+
import sys
|
| 8 |
+
import tempfile
|
| 9 |
+
import types
|
| 10 |
+
import unittest
|
| 11 |
+
|
| 12 |
+
ROOT = pathlib.Path(__file__).resolve().parents[1]
|
| 13 |
+
SPEC = importlib.util.spec_from_file_location('launcher', ROOT / 'tools/launch_single_b200.py')
|
| 14 |
+
launcher = importlib.util.module_from_spec(SPEC)
|
| 15 |
+
SPEC.loader.exec_module(launcher)
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
def method_without_ml_imports(relative, name):
|
| 19 |
+
tree = ast.parse((ROOT / relative).read_text())
|
| 20 |
+
method = next(node for node in ast.walk(tree) if isinstance(node, ast.FunctionDef) and node.name == name)
|
| 21 |
+
module = ast.Module(body=[ast.ImportFrom(module='__future__', names=[ast.alias(name='annotations')], level=0), method], type_ignores=[])
|
| 22 |
+
scope = {}
|
| 23 |
+
exec(compile(ast.fix_missing_locations(module), relative, 'exec'), scope)
|
| 24 |
+
return scope[name]
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
class LauncherTests(unittest.TestCase):
|
| 28 |
+
def plan(self, *arguments):
|
| 29 |
+
return launcher.build_plan(launcher.parser().parse_args(['--run-id', 'test-real', *arguments]))
|
| 30 |
+
|
| 31 |
+
def test_real_workload_and_one_gpu_topology(self):
|
| 32 |
+
plan = self.plan()
|
| 33 |
+
flags = plan['flags']
|
| 34 |
+
for key, value in {'train_batch_size': 512, 'rollout_batch_size': 512, 'max_steps': 100,
|
| 35 |
+
'generate_max_len': 8192, 'prompt_max_len': 2048, 'max_len': 10240,
|
| 36 |
+
'micro_train_batch_size': 4, 'ctkd_topk': 16, 'eval_math_n_samples': 32,
|
| 37 |
+
'eval_math_max_tokens': 8192, 'eval_steps': 10, 'save_steps': 10}.items():
|
| 38 |
+
self.assertEqual(flags[key], value, key)
|
| 39 |
+
for key in ['num_nodes', 'num_gpus_per_node', 'teacher_dp_size', 'teacher_tp_size',
|
| 40 |
+
'teacher_pp_size', 'rollout_num_engines', 'rollout_tp_size']:
|
| 41 |
+
self.assertEqual(flags[key], 1, key)
|
| 42 |
+
self.assertEqual(plan['gradient_accumulation_microsteps'], 128)
|
| 43 |
+
|
| 44 |
+
def test_flags_exist_in_actual_argument_dataclasses(self):
|
| 45 |
+
fields = set()
|
| 46 |
+
for path in (ROOT / 'kdflow/arguments').glob('*_args.py'):
|
| 47 |
+
for node in ast.walk(ast.parse(path.read_text())):
|
| 48 |
+
if isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name):
|
| 49 |
+
fields.add(node.target.id)
|
| 50 |
+
self.assertFalse(set(self.plan()['flags']) - fields)
|
| 51 |
+
|
| 52 |
+
def test_micro_batch_changes_preserve_global_workload_and_baseline(self):
|
| 53 |
+
plan = self.plan('--micro-batch', '1', '--algorithm', 'strict-full')
|
| 54 |
+
self.assertEqual(plan['gradient_accumulation_microsteps'], 512)
|
| 55 |
+
self.assertEqual(plan['flags']['train_batch_size'], 512)
|
| 56 |
+
self.assertEqual(plan['flags']['kd_algorithm'], 'byte_align_ctkd')
|
| 57 |
+
|
| 58 |
+
def test_dry_run_creates_no_files(self):
|
| 59 |
+
with tempfile.TemporaryDirectory() as folder:
|
| 60 |
+
destination = pathlib.Path(folder) / 'absent'
|
| 61 |
+
raw = subprocess.check_output([sys.executable, str(ROOT / 'tools/launch_single_b200.py'),
|
| 62 |
+
'--root', str(destination), '--dry-run'], text=True)
|
| 63 |
+
self.assertEqual(json.loads(raw)['flags']['max_steps'], 100)
|
| 64 |
+
self.assertFalse(destination.exists())
|
| 65 |
+
|
| 66 |
+
def test_existing_output_and_path_traversal_are_refused(self):
|
| 67 |
+
with self.assertRaises(ValueError):
|
| 68 |
+
self.plan('--run-id', '../other-run')
|
| 69 |
+
with tempfile.TemporaryDirectory() as folder:
|
| 70 |
+
plan = self.plan('--root', folder)
|
| 71 |
+
for key in ['student_name_or_path', 'teacher_name_or_path']:
|
| 72 |
+
model = pathlib.Path(plan['flags'][key])
|
| 73 |
+
model.mkdir(parents=True)
|
| 74 |
+
for name in ['config.json', 'tokenizer_config.json', 'tokenizer.json', 'model.safetensors']:
|
| 75 |
+
(model / name).write_text('{}')
|
| 76 |
+
for path in [pathlib.Path(plan['runtime']) / 'python-b200.sh',
|
| 77 |
+
pathlib.Path(plan['flags']['train_dataset_path']),
|
| 78 |
+
pathlib.Path(plan['flags']['eval_math_dataset_path'])]:
|
| 79 |
+
path.parent.mkdir(parents=True, exist_ok=True)
|
| 80 |
+
path.touch()
|
| 81 |
+
pathlib.Path(plan['output']).mkdir(parents=True)
|
| 82 |
+
with self.assertRaises(FileExistsError):
|
| 83 |
+
launcher.validate_files(plan)
|
| 84 |
+
|
| 85 |
+
def test_actual_token_normalization_across_world_and_micro_batch_sizes(self):
|
| 86 |
+
prepare = method_without_ml_imports('kdflow/trainer/on_policy_kd_trainer.py', '_prepare_global_batches')
|
| 87 |
+
backward = method_without_ml_imports('kdflow/backend/fsdp/fsdp_strategy.py', 'backward')
|
| 88 |
+
# Unequal sequence lengths make an incorrect mean-of-means observable.
|
| 89 |
+
lengths = [1 + (index * 13 % 97) for index in range(512)]
|
| 90 |
+
gradients = [((index % 7) - 3) * length for index, length in enumerate(lengths)]
|
| 91 |
+
reference = sum(gradients) / sum(lengths)
|
| 92 |
+
|
| 93 |
+
class Mask:
|
| 94 |
+
def __init__(self, count): self.count = count
|
| 95 |
+
def sum(self): return self.count
|
| 96 |
+
|
| 97 |
+
class Loss:
|
| 98 |
+
def __init__(self, value, sink): self.value, self.sink = value, sink
|
| 99 |
+
def __truediv__(self, divisor): return Loss(self.value / divisor, self.sink)
|
| 100 |
+
def backward(self): self.sink.append(self.value)
|
| 101 |
+
|
| 102 |
+
for world, micro in [(8, 4), (1, 4), (1, 1)]:
|
| 103 |
+
batches = [{'stu_loss_mask': Mask(sum(lengths[i:i + micro])),
|
| 104 |
+
'gradient': sum(gradients[i:i + micro])} for i in range(0, 512, micro)]
|
| 105 |
+
trainer = types.SimpleNamespace(args=types.SimpleNamespace(train=types.SimpleNamespace(use_dynamic_bsz=False)))
|
| 106 |
+
batches = prepare(trainer, batches, 512 // micro)[0]
|
| 107 |
+
rank_gradients = []
|
| 108 |
+
for rank in range(world):
|
| 109 |
+
strategy = types.SimpleNamespace(step=0, accumulated_gradient=512 // micro // world)
|
| 110 |
+
sink = []
|
| 111 |
+
for batch in batches[rank::world]:
|
| 112 |
+
loss = Loss(batch['gradient'] / batch['avg_micro_batch_token_num'], sink)
|
| 113 |
+
backward(strategy, loss, None, None)
|
| 114 |
+
self.assertEqual(strategy.step, 0)
|
| 115 |
+
rank_gradients.append(sum(sink))
|
| 116 |
+
self.assertAlmostEqual(sum(rank_gradients) / world, reference, places=12)
|
| 117 |
+
|
| 118 |
+
|
| 119 |
+
if __name__ == '__main__':
|
| 120 |
+
unittest.main()
|
tools/launch_single_b200.py
ADDED
|
@@ -0,0 +1,169 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""Launch the real Qwen -> Llama training configuration on one allocated B200.
|
| 3 |
+
|
| 4 |
+
Standard library only. --dry-run prints a plan without importing the runtime,
|
| 5 |
+
checking GPUs, writing files or starting training. There is no reduced workload.
|
| 6 |
+
"""
|
| 7 |
+
import argparse
|
| 8 |
+
import datetime
|
| 9 |
+
import hashlib
|
| 10 |
+
import json
|
| 11 |
+
import os
|
| 12 |
+
import pathlib
|
| 13 |
+
import re
|
| 14 |
+
import shlex
|
| 15 |
+
import socket
|
| 16 |
+
import subprocess
|
| 17 |
+
|
| 18 |
+
SOURCE = pathlib.Path(__file__).resolve().parents[1]
|
| 19 |
+
ARCHIVE_SHA = '084c321c0b7af1cd3a36d12a595c623567d4a37525aa0eecbc3933aa7f9b15c3'
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
def parser():
|
| 23 |
+
p = argparse.ArgumentParser(description=__doc__)
|
| 24 |
+
p.add_argument('--root', type=pathlib.Path,
|
| 25 |
+
default=pathlib.Path('/workspace/storage-shared/nlp/tungks/ctopd_all'))
|
| 26 |
+
p.add_argument('--runtime', type=pathlib.Path, help='Installed prefix containing python-b200.sh')
|
| 27 |
+
p.add_argument('--algorithm', choices=['strict-top16', 'strict-full'], default='strict-top16')
|
| 28 |
+
p.add_argument('--seed', type=int, default=42, help='Original launcher default, not a verified paper seed')
|
| 29 |
+
p.add_argument('--micro-batch', type=int, choices=[1, 2, 4], default=4,
|
| 30 |
+
help='Original launcher uses 4; 1/2 retain global batch through accumulation')
|
| 31 |
+
p.add_argument('--concurrency', type=int, default=8, help='Transport concurrency; samples/batch stay unchanged')
|
| 32 |
+
p.add_argument('--eval-timeout', type=int, default=0, help='Seconds per evaluation request; 0 disables timeout')
|
| 33 |
+
p.add_argument('--run-id', help='New output directory name; existing runs are never resumed')
|
| 34 |
+
p.add_argument('--dry-run', action='store_true')
|
| 35 |
+
return p
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
def build_plan(args):
|
| 39 |
+
if args.concurrency < 1 or args.eval_timeout < 0:
|
| 40 |
+
raise ValueError('Concurrency must be positive and eval timeout nonnegative')
|
| 41 |
+
run_id = args.run_id or (datetime.datetime.now(datetime.timezone.utc).strftime('%Y%m%dT%H%M%S%fZ')
|
| 42 |
+
+ '-' + args.algorithm + '-s' + str(args.seed))
|
| 43 |
+
if not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9_.-]{0,95}', run_id):
|
| 44 |
+
raise ValueError('run-id must be a single directory name starting with a letter or digit')
|
| 45 |
+
root = args.root.expanduser().resolve()
|
| 46 |
+
runtime = (args.runtime or root / 'runtime').expanduser().resolve()
|
| 47 |
+
output = root / 'runs' / run_id
|
| 48 |
+
flags = {
|
| 49 |
+
'num_nodes': 1, 'num_gpus_per_node': 1, 'backend': 'fsdp2',
|
| 50 |
+
'train_batch_size': 512, 'micro_train_batch_size': args.micro_batch,
|
| 51 |
+
'learning_rate': '1e-6', 'lr_scheduler': 'constant', 'lr_warmup_ratio': 0.05,
|
| 52 |
+
'adam_betas': '(0.9, 0.98)', 'weight_decay': 0, 'max_norm': 1.0,
|
| 53 |
+
'num_epochs': 10, 'max_steps': 100, 'seed': args.seed,
|
| 54 |
+
'save_path': str(output / 'checkpoints'), 'ckpt_path': str(output / 'unused-resume'),
|
| 55 |
+
'bf16': True, 'gradient_checkpointing': True, 'enable_sleep': True,
|
| 56 |
+
'use_dynamic_bsz': False,
|
| 57 |
+
'student_name_or_path': str(root / 'models/Llama-3.2-3B-Instruct'),
|
| 58 |
+
'teacher_name_or_path': str(root / 'models/Qwen2.5-7B-Instruct'),
|
| 59 |
+
'enable_thinking': False,
|
| 60 |
+
'rollout_batch_size': 512, 'rollout_num_engines': 1, 'rollout_tp_size': 1,
|
| 61 |
+
'rollout_engine_concurrency': args.concurrency, 'rollout_mem_fraction_static': 0.6,
|
| 62 |
+
'n_samples_per_prompt': 1, 'generate_max_len': 8192, 'temperature': 1.0, 'top_p': 0.95,
|
| 63 |
+
'train_dataset_path': str(root / 'data/Train/math_code_20k/train.parquet'),
|
| 64 |
+
'max_len': 10240, 'prompt_max_len': 2048, 'input_key': 'prompt',
|
| 65 |
+
'label_key': 'reward_model.ground_truth', 'apply_chat_template': True,
|
| 66 |
+
'preprocess_num_workers': 8, 'packing_samples': True,
|
| 67 |
+
'eval_math_dataset_path': str(root / 'data/math500/test.jsonl'),
|
| 68 |
+
'eval_math_max_tokens': 8192, 'eval_math_max_concurrent': args.concurrency,
|
| 69 |
+
'eval_math_timeout': args.eval_timeout, 'eval_math_n_samples': 32,
|
| 70 |
+
'eval_steps': 10, 'save_steps': 10, 'logging_steps': 1,
|
| 71 |
+
'kd_ratio': 1.0, 'kd_loss_fn': 'rkl', 'kd_temperature': 1.0,
|
| 72 |
+
'kd_algorithm': 'topk_byte_align_ctkd' if args.algorithm == 'strict-top16' else 'byte_align_ctkd',
|
| 73 |
+
'ctkd_topk': 16, 'teacher_dp_size': 1, 'teacher_tp_size': 1, 'teacher_pp_size': 1,
|
| 74 |
+
'teacher_mem_fraction_static': 0.6, 'teacher_offload_tags': 'all',
|
| 75 |
+
'chunked_loss_size': 2048, 'use_wandb': False,
|
| 76 |
+
}
|
| 77 |
+
command = ['bash', str(runtime / 'python-b200.sh'), '-u', '-m', 'kdflow.cli.train_kd_on_policy']
|
| 78 |
+
for key, value in flags.items():
|
| 79 |
+
command.extend(['--' + key, str(value)])
|
| 80 |
+
return {'algorithm': args.algorithm, 'output': str(output), 'runtime': str(runtime),
|
| 81 |
+
'source': str(SOURCE), 'flags': flags, 'command': command,
|
| 82 |
+
'gradient_accumulation_microsteps': 512 // args.micro_batch,
|
| 83 |
+
'expected_runtime_archive_sha256': ARCHIVE_SHA,
|
| 84 |
+
'qualification': 'unvalidated single-B200 full workload; no metric-match claim',
|
| 85 |
+
'evaluation_scope': 'MATH500 accuracy over 32 responses/problem; not the full paper benchmark suite'}
|
| 86 |
+
|
| 87 |
+
|
| 88 |
+
def run_environment():
|
| 89 |
+
env = os.environ.copy()
|
| 90 |
+
env.update(RAY_ADDRESS='local', HF_HUB_OFFLINE='1', TRANSFORMERS_OFFLINE='1',
|
| 91 |
+
PYTHONUNBUFFERED='1', PYTHONPATH=str(SOURCE))
|
| 92 |
+
# Company HTTP proxies must not intercept local Ray/SGLang connections.
|
| 93 |
+
bypass = [env.get('NO_PROXY', ''), env.get('no_proxy', ''), 'localhost', '127.0.0.1', '::1', socket.gethostname()]
|
| 94 |
+
try:
|
| 95 |
+
bypass.extend(socket.gethostbyname_ex(socket.gethostname())[2])
|
| 96 |
+
except OSError:
|
| 97 |
+
pass
|
| 98 |
+
env['NO_PROXY'] = env['no_proxy'] = ','.join(item for item in bypass if item)
|
| 99 |
+
return env
|
| 100 |
+
|
| 101 |
+
|
| 102 |
+
def validate_files(plan):
|
| 103 |
+
flags = plan['flags']
|
| 104 |
+
required = [pathlib.Path(plan['runtime']) / 'python-b200.sh']
|
| 105 |
+
required += [pathlib.Path(flags[key]) for key in ['train_dataset_path', 'eval_math_dataset_path']]
|
| 106 |
+
for key in ['student_name_or_path', 'teacher_name_or_path']:
|
| 107 |
+
model = pathlib.Path(flags[key])
|
| 108 |
+
required += [model / 'config.json', model / 'tokenizer_config.json', model / 'tokenizer.json']
|
| 109 |
+
index = model / 'model.safetensors.index.json'
|
| 110 |
+
if index.is_file():
|
| 111 |
+
shards = set(json.loads(index.read_text())['weight_map'].values())
|
| 112 |
+
for shard in shards:
|
| 113 |
+
if pathlib.PurePosixPath(shard).name != shard:
|
| 114 |
+
raise ValueError('Unexpected checkpoint shard path: ' + shard)
|
| 115 |
+
required.append(model / shard)
|
| 116 |
+
else:
|
| 117 |
+
required.append(model / 'model.safetensors')
|
| 118 |
+
for path in required:
|
| 119 |
+
if not path.is_file():
|
| 120 |
+
raise FileNotFoundError(path)
|
| 121 |
+
if pathlib.Path(plan['output']).exists():
|
| 122 |
+
raise FileExistsError('Choose a new run-id; refusing existing output: ' + plan['output'])
|
| 123 |
+
|
| 124 |
+
|
| 125 |
+
def main(argv=None):
|
| 126 |
+
args = parser().parse_args(argv)
|
| 127 |
+
plan = build_plan(args)
|
| 128 |
+
if args.dry_run:
|
| 129 |
+
print(json.dumps(plan, indent=2))
|
| 130 |
+
return
|
| 131 |
+
validate_files(plan)
|
| 132 |
+
env = run_environment()
|
| 133 |
+
commit = subprocess.check_output(['git', '-C', str(SOURCE), 'rev-parse', 'HEAD'], text=True).strip()
|
| 134 |
+
dirty = subprocess.check_output(['git', '-C', str(SOURCE), 'status', '--porcelain'], text=True).strip()
|
| 135 |
+
if dirty:
|
| 136 |
+
raise RuntimeError('Commit source changes before starting a recorded reproduction run')
|
| 137 |
+
# This validates real model/data availability and GPU visibility, not a toy training workload.
|
| 138 |
+
probe = '''import json, pathlib, torch, pyarrow.parquet as pq
|
| 139 |
+
assert torch.cuda.device_count() == 1, 'Expose exactly the one GPU allocated to this run'
|
| 140 |
+
name = torch.cuda.get_device_name(0)
|
| 141 |
+
assert 'B200' in name, 'This launcher targets a B200'
|
| 142 |
+
train, evaluation = __import__('sys').argv[1:]
|
| 143 |
+
metadata = pq.read_metadata(train)
|
| 144 |
+
assert metadata.num_rows == 20000, 'Expected authors source pool of 20,000 prompts before tokenizer filtering'
|
| 145 |
+
assert {'prompt', 'reward_model'} <= set(pq.read_schema(train).names)
|
| 146 |
+
rows = [json.loads(line) for line in pathlib.Path(evaluation).read_text().splitlines() if line.strip()]
|
| 147 |
+
assert len(rows) == 500 and all({'problem', 'answer'} <= row.keys() for row in rows)
|
| 148 |
+
print(json.dumps({'gpu': name, 'torch': torch.__version__, 'cuda': torch.version.cuda, 'source_rows': metadata.num_rows, 'math500_rows': len(rows)}))
|
| 149 |
+
'''
|
| 150 |
+
result = subprocess.check_output(['bash', str(pathlib.Path(plan['runtime']) / 'python-b200.sh'),
|
| 151 |
+
'-c', probe, plan['flags']['train_dataset_path'], plan['flags']['eval_math_dataset_path']],
|
| 152 |
+
env=env, text=True)
|
| 153 |
+
plan['source_commit'] = commit
|
| 154 |
+
plan['preflight'] = json.loads(result.strip().splitlines()[-1])
|
| 155 |
+
plan['input_sha256'] = {}
|
| 156 |
+
for key in ['train_dataset_path', 'eval_math_dataset_path']:
|
| 157 |
+
path = pathlib.Path(plan['flags'][key])
|
| 158 |
+
plan['input_sha256'][key] = hashlib.sha256(path.read_bytes()).hexdigest()
|
| 159 |
+
output = pathlib.Path(plan['output'])
|
| 160 |
+
output.mkdir(parents=True, exist_ok=False)
|
| 161 |
+
(output / 'launch.json').write_text(json.dumps(plan, indent=2) + '\n')
|
| 162 |
+
(output / 'command.sh').write_text(shlex.join(plan['command']) + '\n')
|
| 163 |
+
print('RUN_DIRECTORY=' + str(output), flush=True)
|
| 164 |
+
os.chdir(SOURCE)
|
| 165 |
+
os.execvpe(plan['command'][0], plan['command'], env)
|
| 166 |
+
|
| 167 |
+
|
| 168 |
+
if __name__ == '__main__':
|
| 169 |
+
main()
|