CPU validation record
Date: 2026-09-20 (Asia/Bangkok).
No model weights or benchmark datasets were loaded. No GPU experiment was run. Round 2 was implemented and its disabled-launcher guard was tested; the round itself was not executed.
From verl-0.7.0/examples/nash_exp, the full test command was:
/tmp/nash-diagnostic-b-venv/bin/python3 -m pytest -q tests/diagnostic_b
Actual result after the author settings and boxed-answer revision: 172 passed, 10 warnings in 12.73 seconds. The ten CVXPY warnings report an inaccurate solver status on numerical fixtures. They are preserved; accepted solutions must separately satisfy the declared feasibility, certificate, and objective-gap checks. No test was skipped or weakened.
The suite covers population-standard-deviation advantages, exact original-token segmentation, output-head/autograd identity, temperature differentiation, held-out denominator and token-sum normalization, collision-power degeneracy and tie-breaking, matched feasible controls, a strict Nash/Linear witness, support identities, held-out isolation, paired prompt-cluster uncertainty, schema and prompt validation, symbolic equivalence and parse failures, one shared generation engine, request-level resume, cache corruption/mismatch checks, resource preflight, configuration gates, and single-file failure logging.
The DelTA revision additionally checks the Appendix-F proxy against each sampled output row's autograd derivative, sharing hidden coordinates across different token IDs, weighted response/segment aggregation, production proxy Grams and cross products, original-token prediction alignment, held-out token sums and all-draw normalization, feature-cache geometry and checksum rejection, geometry-specific preflight memory estimates, report labels, rejection of mixed geometries, and reuse of generated responses when switching feature backends. The original full-head reference and production-backend tests still pass.
The boxed-answer revision checks that the shipped prompt adds the same boxed instruction to DAPO and held-out questions, preserves earlier DAPO messages and the original dataset rows, and excludes reference answers. Real Math-Verify CPU tests check nested fractions, symbolic equivalence, rejection of unboxed numbers, selection of the final box, malformed final boxes without fallback, and escaped LaTeX braces. Missing-setting tests now build explicitly unresolved fixtures; they still check that configuration validation runs before external imports.
These additional checks passed:
ruff check diagnostic_b tests/diagnostic_b
ruff format --check diagnostic_b tests/diagnostic_b
python3 -m mypy --follow-imports=skip diagnostic_b
python3 -m compileall -q diagnostic_b tests/diagnostic_b
bash -n scripts/run_diagnostic_b_round1_gpu.sh scripts/run_diagnostic_b_round2_gpu.sh
Ruff used the parent veRL configuration. Mypy reported no issues in 14 source files under the repository's existing configuration; that configuration ignores errors by default outside its selected veRL modules, so this is not a claim of strict static type coverage. Shell syntax and Python compilation checks passed.
An actual standalone CPU smoke run also completed:
/tmp/nash-diagnostic-b-venv/bin/python3 -m diagnostic_b \
--cpu-smoke --output-dir /tmp/nash-diagnostic-b-delta-smoke --resume
It produced compressed shared caches, direction records, CSV/JSON/Markdown/LaTeX
tables, PDF/PNG figures, and a report labeled round_cpu_smoke. The three-panel
scatter was visually inspected and both axes name the DelTA token-gradient proxy.
These are synthetic fixture outputs, not Qwen
results. The CLI tests additionally run smoke/resume in fresh subprocesses while
blocking imports of all model/dataset loading libraries and verify unchanged
rollout hashes and modification times on resume.
The isolated test environment used Python 3.10.12, CPU PyTorch 2.14.0+cpu,
NumPy 2.2.6, SciPy 1.15.3, CVXPY 1.7.5, Clarabel 0.11.1, pandas 2.3.3,
PyArrow 25.0.1, matplotlib 3.10.9, pytest 9.1.1, Ruff 0.12.2, mypy 1.17.0,
and Math-Verify 0.8.0. This environment was created under /tmp; the project
does not depend on that path for server execution.
The author selected seed 42, sungyub/dapo-math-17k-verl, the boxed prompt,
Math-Verify boxed-answer scoring, radius coefficient 0.4, and a requested response
cap of 32768. Dataset metadata resolved the training revision to
3cf5c112137795c1f1d5fdae7c747871b26840a4; no full dataset was downloaded.
A real CLI dry run with the updated Round-1 YAML stopped before external loading
with only the response/context conflict: a 32768-token response cap leaves no
space for a prompt inside the fixed 32768-token context. The requested value is
preserved pending the author's choice of a smaller fixed response cap or an
explicit remaining-context policy. Its log is
/tmp/nash-author-settings-dry-run/diagnostic_b_round1.log.
Real vLLM/Transformers execution, benchmark schema snapshots, memory peaks, throughput, and measured correlations still require the author's server run. Preflight records and checks the actual server configuration before costly generation begins.
Server setup validation
The simplified scripts/setup_server.sh has two offline Bash tests in
tests/diagnostic_b/test_server_setup.py: 2 passed in 1.32 seconds.
They cover the Python 3.10 environment request, ordered editable/requirements/
vLLM/FlashAttention installs, the exact requested wheel URL, paths with spaces,
activation, reuse of an existing environment, and retry after a failed wheel
download. Package installation and wget are mocked; the Bash script runs normally.
Ruff and shell syntax checks passed. No GPU packages were installed locally.
The earlier metadata-only dependency resolution covered editable veRL, root and experiment requirements and vLLM 0.11.0 with the existing server constraints. It selected PyTorch 2.8.0+cu128, torchvision 0.23.0+cu128, torchaudio 2.8.0+cu128, Transformers 4.57.6 and NumPy 1.26.4 for Python 3.10. This did not test the newly requested FlashAttention wheel's binary imports or GPU runtime compatibility.