jayasriii/390M_batchsize

390M batch-size sweep

One folder per batch size (tokens per optimizer step). Each holds ckpts/ (all checkpoints of that batch size's run, merged across resume segments) and csv/ with one merged training log. Peak LR is 1e-3 unless noted.

Folder	Tokens/step	Old name	Merged log	Steps logged	Ckpt steps
bs_0.26M	262,144	"200k"	390M_bs_0.26M_merged.csv	0 – 53405	6611 – 53406
bs_0.52M	524,288	0.52M	390M_1e3_heldout.csv (single run)	0 – 26702	in jayasriii/390M_10B/390M_1e3_heldout
bs_0.79M	786,432	"700k"	390M_bs_0.79M_merged.csv	0 – 12152	1000 – 12153
bs_1M	1,048,576	1M	390M_bs_1M_merged.csv	0 – 8699	1000 – 8700
bs_2M	2,097,152	2M	390M_bs_2M_merged.csv	0 – 4607 (gaps, see below)	1000 – 4608
bs_2M/sqrtlr_2e-3	2,097,152	2M sqrt-LR	390M_2M_sqrtlr.csv (single run, peak LR 2e-3)	0 – 1257	1000, 1258
bs_4M	4,194,304	4M	390M_bs_4M_merged.csv	0 – 2022	1 – 1277

How the logs were merged

Segments are concatenated in run order. A resume restarts from a checkpoint at step S, so any earlier-logged row at step >= S belongs to an abandoned attempt and is replaced by the resumed run's row (this also covers the in-file restart at 13304 in 390M_200k_39k.csv).

• bs_2M: step 3004 was never logged (resume boundary), and steps 3294–3531 are lost — that part of 390M_2Mbs_main.csv was overwritten with null bytes.
• bs_4M: the merged log is 390M_4M_2k.csv, the one continuous log to step 2022. 390M_4M_1275.csv re-ran 1214–1276 from the step-1214 ckpt (|Δ train loss| ≤ 1e-3) to produce ckpt 1277.

extras/

• segment_logs/<bs>/ — the original per-segment logs the merged logs were built from, plus the old hand-stitched *_full_stitched.csv files (superseded: the 0.26M one used the abandoned pass for 13304–19339, and the 1M one is missing steps 5616–5617).
• 390M_2M_flux/ — single 2M ckpt (step 3465) with no log; not part of the bs_2M chain.
• 200kbs.csv — older partial stitch of the 0.26M log.
• loss.csv — a separate 1M run (9448 steps, different val set), no ckpts here.
• *.py — training / check scripts.