Stage2 Fine Phase — Loss Recalibration
Date: 2026-08-16 22:53 UTC
Why
At CE ~2.0 the cumulative average log was no longer a good control signal. Generation quality is still noisy (expected ~200M tokens) — keep digesting. Recalibrate metrics + LR, not weights.
Changes
| Item | Before | Fine phase |
|---|---|---|
| LR | 1e-3 | 5e-4 |
| Logged CE | cumulative mean from resume | batch CE + EMA (beta=0.98) |
| tok/s | absolute position / session time (inflated) | session tokens / elapsed |
| Objective | dense CE + 0.02*lb | unchanged |
| Weights | — | kept, resume exact token offset |
Resume offsets (at switch)
See .