# 32k -> 64k YaRN extension on b5 -- 2026-09-10

Parent: b4 full-epoch step 2227 (best 32k model: 70.0/58.0/43.0 at 8k/16k/32k).
Data:   b5_65536_131072 -- min doc 65,536 == the window, so every sample is
        ONE contiguous document.
YaRN:   SCALE=2 ORIG_MAXPOS=32768 -> 65536 exactly.
Train:  695 steps x 524288 = .364 B tokens
        = 42% of b5's 0.866 B train split (no wrap).
        WSD warmup 100 / stable 593 / decay tail 2 -- final ckpt DECAYED.
        LR peak 1.8e-4, floor 6e-5 (CP_MAX_LR is a no-op; see B4 results doc).
        4x H200, cp=4, B=1 x grad_accum 8.
Eval:   niah_single, N=100, greedy, shared haystack b2_8192_16384 val.

## Results

| context | niah_single |
|---|---|
| 8192 | 67.0
 |
| 16360 | 65.0
 |
| 32744 | 52.0
 |
| 65512 | 29.0
 |

For comparison, the 32k parent at the same lengths: 70.0 / 58.0 / 43.0 / n/a
(it cannot run at 64k -- its RoPE cache is sized 32,768).

NOTE: this is a PARTIAL b5 epoch, sized to a wall-clock deadline. It is a
first 64k data point, not a converged extension.