|
Download ops/DPO_CHAT.md from zerdovzad/WarpStateAI: direct link, hf CLI and curl.
- Browser
- Download file 2.72 kB
-
https://huggingface.co/zerdovzad/WarpStateAI/resolve/main/ops/DPO_CHAT.md
- Command line
-
hf download hf://zerdovzad/WarpStateAI/ops/DPO_CHAT.md
-
curl -L -o DPO_CHAT.md https://huggingface.co/zerdovzad/WarpStateAI/resolve/main/ops/DPO_CHAT.md
2.72 kB
| # Final DPO archive and chat | |
| DPO finished on 2026-09-15 at 01:05:06.882 UTC, with the guardian's final | |
| verification complete at 01:05:42 UTC. It completed 2,334 updates over 74,661 | |
| preference pairs. Training remains paused. | |
| The historical archive is `work/archives/step-00076294-dpo-final/` relative to | |
| `rtx6000_40b`. It preserves the full FP32 model/Adam/RNG checkpoint and a | |
| tensor-exact inference export, plus the tokenizer, evaluation, recipes, | |
| reference scores, and provenance. Files are sealed read-only and checksummed. | |
| Existing raw and SFT archives are preserved. | |
| From `/workspace/WarpState`, create or verify the archive and prepare the | |
| independent chat copy: | |
| ```bash | |
| source rtx6000_40b/ops/training-env.sh | |
| rtx6000_40b/.server-venv/bin/python -B rtx6000_40b/ops/archive_dpo.py | |
| rtx6000_40b/.server-venv/bin/python -B rtx6000_40b/ops/prepare_dpo_chat.py | |
| ``` | |
| The chat copy is `rtx6000_40b/work/chat/dpo-final/`: separate physical files, | |
| with hashes equal to the archive. The chat launcher reads this copy by default. | |
| Neither chat nor preparation modifies archived weights. | |
| ```bash | |
| rtx6000_40b/ops/chat-dpo.sh | |
| # Or attach to the already-open interface: | |
| tmux attach -t warpstate-chat | |
| ``` | |
| Type a message at `You:`. Ctrl-C discards unfinished input or cancels the current | |
| reply and returns to `You:`, preserving completed conversation turns. | |
| `/reset` clears conversation history; `/quit` exits. | |
| Detach tmux with Ctrl-B followed by D. The interface uses GPU 0, BF16 compute, | |
| original FP32 master weights, a 2,048-token context, and greedy decoding. | |
| Responses print when complete. Alternate CUDA devices can be selected with | |
| `--device cuda:1`; `--temperature 0.7` enables sampling. | |
| Prompt serialization matches the segmented role headers, BOS and assistant EOS | |
| used during posttraining. Cached inference follows the model's parallel chunk | |
| memory; shape-dependent BF16 rounding can differ from full-sequence execution. | |
| Run reproducible sample prompts, choosing a new output filename each time: | |
| ```bash | |
| rtx6000_40b/ops/chat-dpo.sh --prompts-json rtx6000_40b/configs/posttrain_smoke_prompts.json --max-new-tokens 128 --output-json rtx6000_40b/work/validation/dpo-smoke.json | |
| ``` | |
| These few prompts are sanity checks, not a capability benchmark. DPO's final | |
| held-out preference loss was 0.5790; 245 of 339 pairs had positive relative | |
| reward margins. This 72.27% is not a generation win rate or answer accuracy. | |
| Exponentiating DPO loss does not produce language-model perplexity. | |
| For a matched full SFT holdout evaluation, `ops/evaluate_sft_snapshot.py` | |
| accepts `--snapshot`, `--device`, and a new `--output` path. Reports and generated | |
| responses stay under ignored `work/`; source scripts and prompts are tracked. | |