Hugging Face's logo

rubentium
/
sparse-checkpoints

Model card Files
xet
 Community
rubentium commited on
Commit
fe19ae1
·
verified ·
1 Parent(s): ed534a8

Update seed-70 checkpoint inventory

Browse files
Files changed (2) hide show
  1. inventory/PAPER_CHECKPOINT_INVENTORY.md +2 -2
  2. inventory/PAPER_CHECKPOINT_INVENTORY.tsv +34 -32
inventory/PAPER_CHECKPOINT_INVENTORY.md CHANGED
@@ -7,13 +7,13 @@ Canonical final-checkpoint evacuation set. Only complete, paper-relevant endpoin
7
  - control_dropout_365m: five ordinary attention-dropout controls (p=.28), seeds 42-46.
8
  - control_rope_365m: five scalar RoPE-distance controls (a=1.0), seeds 42-46.
9
  - scale_1p2b_factorial: older Dense/Mixed/Fixed/Resampled 1.2B factorial endpoints; descriptive/appendix evidence.
10
- - scale_1p2b_current: current 1.2B seed-70 Mixed and seed-71 Dense endpoints.
11
  - scale_3b_boundary: 3B Dense/Mixed endpoints used for scale/boundary analysis.
12
 
13
  ## Evaluation coverage
14
  Primary 365M endpoints: matched PPL ladder across FineWeb, ArXiv, PG19 at 8k/12k/16k/24k. Dropout and RoPE: matched control grids. Older 1.2B factorial: FINAL_ENDPOINT_REFRESH, FINAL_FIXED, and L1_2B_REMAINING batteries (PPL, selector, passkey/RULER/LongBench where present). 3B: 8k/16k direction and 30k reversal/boundary analysis. See PAPER_POSITIONING_AUDIT_20260821.md for caveats and exact result manifests.
15
 
16
  ## Status
17
- Current seed-70 Mixed was restarted from checkpoint-48750; inventory points to existing complete checkpoint-48962 until the restarted run reaches its terminal checkpoint. Current seed-70 Dense is unfinished and excluded.
18
 
19
  See PAPER_CHECKPOINT_INVENTORY.tsv for machine-readable paths.
 
7
  - control_dropout_365m: five ordinary attention-dropout controls (p=.28), seeds 42-46.
8
  - control_rope_365m: five scalar RoPE-distance controls (a=1.0), seeds 42-46.
9
  - scale_1p2b_factorial: older Dense/Mixed/Fixed/Resampled 1.2B factorial endpoints; descriptive/appendix evidence.
10
+ - scale_1p2b_current: current 1.2B seed-70 Dense and repaired Mixed endpoints at checkpoint-50462; seed-71/72 endpoints are also retained. The unrepaired seed-70 Mixed checkpoint-48962 is a separate earlier endpoint.
11
  - scale_3b_boundary: 3B Dense/Mixed endpoints used for scale/boundary analysis.
12
 
13
  ## Evaluation coverage
14
  Primary 365M endpoints: matched PPL ladder across FineWeb, ArXiv, PG19 at 8k/12k/16k/24k. Dropout and RoPE: matched control grids. Older 1.2B factorial: FINAL_ENDPOINT_REFRESH, FINAL_FIXED, and L1_2B_REMAINING batteries (PPL, selector, passkey/RULER/LongBench where present). 3B: 8k/16k direction and 30k reversal/boundary analysis. See PAPER_POSITIONING_AUDIT_20260821.md for caveats and exact result manifests.
15
 
16
  ## Status
17
+ The canonical seed-70 pair is Dense checkpoint-50462 and repaired Mixed checkpoint-50462. The older unrepaired Mixed checkpoint-48962 remains separately available and is not the repaired pair endpoint. Both canonical checkpoint-50462 directories include model weights, config, and training state.
18
 
19
  See PAPER_CHECKPOINT_INVENTORY.tsv for machine-readable paths.
inventory/PAPER_CHECKPOINT_INVENTORY.tsv CHANGED
@@ -1,32 +1,34 @@
1
- group family checkpoint absolute_path
2
- primary_365m l8k365m_mixed_seed42 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_mixed_seed42/run_20260719_234149/checkpoint-44000
3
- primary_365m l8k365m_mixed_seed43 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_mixed_seed43/run_20260719_234149/checkpoint-44000
4
- primary_365m l8k365m_mixed_seed44 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_mixed_seed44/run_20260722_221755/checkpoint-44000
5
- primary_365m l8k365m_mixed_seed45 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_mixed_seed45/run_20260722_223953/checkpoint-44000
6
- primary_365m l8k365m_mixed_seed46 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_mixed_seed46/run_20260722_223953/checkpoint-44000
7
- primary_365m l8k365m_mixed_seed47 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_mixed_seed47/run_20260820_031812/checkpoint-44000
8
- primary_365m l8k365m_dense checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense/run_20260707_034553/checkpoint-44000
9
- primary_365m l8k365m_dense_seed43 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_seed43/run_20260722_221755/checkpoint-44000
10
- primary_365m l8k365m_dense_seed44 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_seed44/run_20260722_223953/checkpoint-44000
11
- primary_365m l8k365m_dense_seed45 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_seed45/run_20260722_221755/checkpoint-44000
12
- primary_365m l8k365m_dense_seed46 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_seed46/run_20260722_223953/checkpoint-44000
13
- primary_365m l8k365m_dense_seed47 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_seed47/run_20260820_031247/checkpoint-44000
14
- control_dropout_365m l8k365m_dense_dropout_p28_seed42 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_dropout_p28_seed42/run_20260819_050018/checkpoint-44000
15
- control_dropout_365m l8k365m_dense_dropout_p28_seed43 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_dropout_p28_seed43/run_20260819_050018/checkpoint-44000
16
- control_dropout_365m l8k365m_dense_dropout_p28_seed44 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_dropout_p28_seed44/run_20260819_050018/checkpoint-44000
17
- control_dropout_365m l8k365m_dense_dropout_p28_seed45 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_dropout_p28_seed45/run_20260819_050018/checkpoint-44000
18
- control_dropout_365m l8k365m_dense_dropout_p28_seed46 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_dropout_p28_seed46/run_20260819_050018/checkpoint-44000
19
- control_rope_365m l8k365m_dense_ropectl_a1p0_seed42 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_ropectl_a1p0_seed42/run_20260819_153735/checkpoint-44000
20
- control_rope_365m l8k365m_dense_ropectl_a1p0_seed43 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_ropectl_a1p0_seed43/run_20260819_153735/checkpoint-44000
21
- control_rope_365m l8k365m_dense_ropectl_a1p0_seed44 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_ropectl_a1p0_seed44/run_20260819_153858/checkpoint-44000
22
- control_rope_365m l8k365m_dense_ropectl_a1p0_seed45 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_ropectl_a1p0_seed45/run_20260819_153735/checkpoint-44000
23
- control_rope_365m l8k365m_dense_ropectl_a1p0_seed46 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_ropectl_a1p0_seed46/run_20260819_153735/checkpoint-44000
24
- scale_1p2b_current l8k1p2b_mixed_seed70 checkpoint-48962 /mloscratch/homes/navasard/sparse/checkpoints/l8k1p2b_mixed_seed70/run_20260822_210107/checkpoint-48962
25
- scale_1p2b_current l8k1p2b_dense_seed71 checkpoint-50462 /mloscratch/homes/navasard/sparse/checkpoints/l8k1p2b_dense_seed71/run_20260823_140253/checkpoint-50462
26
- scale_3b_boundary l8k3b_dense_60b checkpoint-100000 /mloscratch/homes/navasard/sparse/checkpoints/l8k3b_dense_60b/run_20260720_074219/checkpoint-100000
27
- scale_3b_boundary l8k3b_mixed_60b checkpoint-100000 /mloscratch/homes/navasard/sparse/checkpoints/l8k3b_mixed_60b/run_20260722_084223/checkpoint-100000
28
- scale_1p2b_factorial l8k1_2b_mixed checkpoint-50462 /mloscratch/homes/navasard/sparse/checkpoints/l8k1_2b_mixed/run_20260715_234103/checkpoint-50462
29
- scale_1p2b_factorial l30k1_2b_dense checkpoint-37465 /mloscratch/homes/navasard/sparse/checkpoints/l30k1_2b_dense/run_20260708_065119/checkpoint-37465
30
- scale_1p2b_factorial l30k1_2b_mixed checkpoint-37465 /mloscratch/homes/navasard/sparse/checkpoints/l30k1_2b_mixed/run_20260721_055708/checkpoint-37465
31
- scale_1p2b_factorial l30k1_2b_fixed checkpoint-37465 /mloscratch/homes/navasard/sparse/checkpoints/l30k1_2b_fixed/run_20260709_000503/checkpoint-37465
32
- scale_1p2b_factorial l30k1_2b_resampled checkpoint-37465 /mloscratch/homes/navasard/sparse/checkpoints/l30k1_2b_resampled/run_20260708_112209/checkpoint-37465
 
 
 
1
+ group family checkpoint absolute_path
2
+ primary_365m l8k365m_mixed_seed42 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_mixed_seed42/run_20260719_234149/checkpoint-44000
3
+ primary_365m l8k365m_mixed_seed43 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_mixed_seed43/run_20260719_234149/checkpoint-44000
4
+ primary_365m l8k365m_mixed_seed44 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_mixed_seed44/run_20260722_221755/checkpoint-44000
5
+ primary_365m l8k365m_mixed_seed45 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_mixed_seed45/run_20260722_223953/checkpoint-44000
6
+ primary_365m l8k365m_mixed_seed46 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_mixed_seed46/run_20260722_223953/checkpoint-44000
7
+ primary_365m l8k365m_mixed_seed47 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_mixed_seed47/run_20260820_031812/checkpoint-44000
8
+ primary_365m l8k365m_dense checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense/run_20260707_034553/checkpoint-44000
9
+ primary_365m l8k365m_dense_seed43 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_seed43/run_20260722_221755/checkpoint-44000
10
+ primary_365m l8k365m_dense_seed44 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_seed44/run_20260722_223953/checkpoint-44000
11
+ primary_365m l8k365m_dense_seed45 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_seed45/run_20260722_221755/checkpoint-44000
12
+ primary_365m l8k365m_dense_seed46 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_seed46/run_20260722_223953/checkpoint-44000
13
+ primary_365m l8k365m_dense_seed47 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_seed47/run_20260820_031247/checkpoint-44000
14
+ control_dropout_365m l8k365m_dense_dropout_p28_seed42 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_dropout_p28_seed42/run_20260819_050018/checkpoint-44000
15
+ control_dropout_365m l8k365m_dense_dropout_p28_seed43 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_dropout_p28_seed43/run_20260819_050018/checkpoint-44000
16
+ control_dropout_365m l8k365m_dense_dropout_p28_seed44 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_dropout_p28_seed44/run_20260819_050018/checkpoint-44000
17
+ control_dropout_365m l8k365m_dense_dropout_p28_seed45 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_dropout_p28_seed45/run_20260819_050018/checkpoint-44000
18
+ control_dropout_365m l8k365m_dense_dropout_p28_seed46 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_dropout_p28_seed46/run_20260819_050018/checkpoint-44000
19
+ control_rope_365m l8k365m_dense_ropectl_a1p0_seed42 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_ropectl_a1p0_seed42/run_20260819_153735/checkpoint-44000
20
+ control_rope_365m l8k365m_dense_ropectl_a1p0_seed43 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_ropectl_a1p0_seed43/run_20260819_153735/checkpoint-44000
21
+ control_rope_365m l8k365m_dense_ropectl_a1p0_seed44 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_ropectl_a1p0_seed44/run_20260819_153858/checkpoint-44000
22
+ control_rope_365m l8k365m_dense_ropectl_a1p0_seed45 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_ropectl_a1p0_seed45/run_20260819_153735/checkpoint-44000
23
+ control_rope_365m l8k365m_dense_ropectl_a1p0_seed46 checkpoint-44000 /mloscratch/homes/navasard/sparse/checkpoints/l8k365m_dense_ropectl_a1p0_seed46/run_20260819_153735/checkpoint-44000
24
+ scale_1p2b_current l8k1p2b_mixed_seed70 checkpoint-48962 /mloscratch/homes/navasard/sparse/checkpoints/l8k1p2b_mixed_seed70/run_20260822_210107/checkpoint-48962
25
+ scale_1p2b_current l8k1p2b_mixed_seed70_repaired checkpoint-50462 /mloscratch/homes/navasard/sparse/checkpoints/l8k1p2b_mixed_seed70_repaired/run_20260909_054557/checkpoint-50462
26
+ scale_1p2b_current l8k1p2b_dense_seed70 checkpoint-50462 /mloscratch/homes/navasard/sparse/checkpoints/l8k1p2b_dense_seed70/run_20260823_043810/checkpoint-50462
27
+ scale_1p2b_current l8k1p2b_dense_seed71 checkpoint-50462 /mloscratch/homes/navasard/sparse/checkpoints/l8k1p2b_dense_seed71/run_20260823_140253/checkpoint-50462
28
+ scale_3b_boundary l8k3b_dense_60b checkpoint-100000 /mloscratch/homes/navasard/sparse/checkpoints/l8k3b_dense_60b/run_20260720_074219/checkpoint-100000
29
+ scale_3b_boundary l8k3b_mixed_60b checkpoint-100000 /mloscratch/homes/navasard/sparse/checkpoints/l8k3b_mixed_60b/run_20260722_084223/checkpoint-100000
30
+ scale_1p2b_factorial l8k1_2b_mixed checkpoint-50462 /mloscratch/homes/navasard/sparse/checkpoints/l8k1_2b_mixed/run_20260715_234103/checkpoint-50462
31
+ scale_1p2b_factorial l30k1_2b_dense checkpoint-37465 /mloscratch/homes/navasard/sparse/checkpoints/l30k1_2b_dense/run_20260708_065119/checkpoint-37465
32
+ scale_1p2b_factorial l30k1_2b_mixed checkpoint-37465 /mloscratch/homes/navasard/sparse/checkpoints/l30k1_2b_mixed/run_20260721_055708/checkpoint-37465
33
+ scale_1p2b_factorial l30k1_2b_fixed checkpoint-37465 /mloscratch/homes/navasard/sparse/checkpoints/l30k1_2b_fixed/run_20260709_000503/checkpoint-37465
34
+ scale_1p2b_factorial l30k1_2b_resampled checkpoint-37465 /mloscratch/homes/navasard/sparse/checkpoints/l30k1_2b_resampled/run_20260708_112209/checkpoint-37465