|
Download logs/cont64k/RESULTS.md from unparallelled/BluScriptCP: direct link, hf CLI and curl.
- Browser
- Download file 771 Bytes
-
https://huggingface.co/unparallelled/BluScriptCP/resolve/main/logs/cont64k/RESULTS.md
- Command line
-
hf download hf://unparallelled/BluScriptCP/logs/cont64k/RESULTS.md
-
curl -L -o RESULTS.md https://huggingface.co/unparallelled/BluScriptCP/resolve/main/logs/cont64k/RESULTS.md
771 Bytes
| # 64k b5 continuation to a full epoch -- 2026-09-11 | |
| Parent: ext64k run3 step 695 (67.0/65.0/52.0/29.0 at 8k/16k/32k/64k), | |
| loaded WEIGHTS-ONLY: ZeRO sidecars cannot cross world_size 4 -> 8. | |
| Data: b5_65536_131072, 9 shards = .865 B tokens. | |
| Train: 1651 steps x 524288 = .865 B tokens = a full b5 epoch. | |
| The loader restarts at token 0, so b5's first 42% is seen a | |
| second time and the rest for the first time. | |
| WSD warmup 100 / stable 1451 / decay tail 100. | |
| 8x H100, cp=8, B=1 x grad_accum 8. | |
| Eval: niah_single, N=100, greedy, shared haystack b2_8192_16384 val. | |
| ## Results | |
| | context | niah_single | | |
| |---|---| | |
| | 8192 | 45.0 | | |
| | 16360 | 17.0 | | |
| | 32744 | 2.0 | | |
| | 65512 | 1.0 | | |
| Parent for comparison: 67.0 / 65.0 / 52.0 / 29.0. | |