File size: 1,389 Bytes
9a4a1e3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | # Code archive
Snapshot of /workspace on the Vast 4090 training box (instance 36391051), with git history.
## Snapshot 20260521T230246Z
- `agillm_workspace_git_20260521T230246Z.tar.gz` - full .git directory (extract with `tar -xzf ...; mv .git /target/`)
- `agillm_code_snapshot_20260521T230246Z.tar.gz` - flat snapshot of all tracked files at HEAD (no .git)
## Git history at time of snapshot
3 commits on `main`:
1. `66604cc` Baseline: vast workspace state as of 2026-05-21T23:01Z - pre-gate-fix
2. `9da9902` Fix SAT gate collapse: per-position forward + correctness-based training target
3. `e18cc73` Add AR-only ablation script (5k steps from 220k, no SAT loss)
## SAT gate fix summary (commit 9da9902)
Root-cause: training loss hardcoded gate target to torch.ones(...), causing the
SAT gate to collapse to a deterministic one-hot. Result: sat-var and sat-fixed
inference produced byte-identical output (gate always picked stride=2).
Three sub-fixes in nB300.py:
- SATHead.forward: gate applied per-position (was h_last[:, 0] only)
- Training loss: gate target = (logits_sat.argmax(-1) == tgt_sat).long()
- Inference: gate is now [B, SAT_BLOCK, 2]; stride uses gate[:, 0]
Backward-compatible with existing checkpoints. Existing 220k checkpoint's gate
weights are still collapsed from prior training (needs fresh SFT for full
benefit), but the inference math is now correct.
|