Upload folder using huggingface_hub

logs/cont64k/RESULTS.md

@@ -0,0 +1,22 @@
+# 64k b5 continuation to a full epoch -- 2026-09-11
+
+Parent: ext64k run3 step 695 (67.0/65.0/52.0/29.0 at 8k/16k/32k/64k),
+        loaded WEIGHTS-ONLY: ZeRO sidecars cannot cross world_size 4 -> 8.
+Data:   b5_65536_131072, 9 shards = .865 B tokens.
+Train:  1651 steps x 524288 = .865 B tokens = a full b5 epoch.
+        The loader restarts at token 0, so b5's first 42% is seen a
+        second time and the rest for the first time.
+        WSD warmup 100 / stable 1451 / decay tail 100.
+        8x H100, cp=8, B=1 x grad_accum 8.
+Eval:   niah_single, N=100, greedy, shared haystack b2_8192_16384 val.
+
+## Results
+
+| context | niah_single |
+|---|---|
+| 8192 | 45.0
 |
+| 16360 | 17.0
 |
+| 32744 | 2.0
 |
+| 65512 | 1.0
 |
+
+Parent for comparison: 67.0 / 65.0 / 52.0 / 29.0.

logs/cont64k/cont.ckpt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6d2efc22b5d48e47b8409e4ea35a9f890c73fb354f4545554d18ea133f719ab3
+size 62

logs/cont64k/eval_16360.log

@@ -0,0 +1,41 @@
+[18:22:46] longeval: T=65536 prompt_len=16360 max_new=24 n=100 greedy=1 gpus=1
+[18:22:46] haystack: /root/Adhi/BluTrain/dist/Context_Parallelism/Data_Loader/Data/b2_8192_16384/val_shard_00000.bin
+[18:22:46] prompt signature changed -- regenerating prompts and clearing state
+[18:22:46] generating prompts: niah_single
+[18:22:50] fan-out: 1 GPU(s), NP=1 -> 1 concurrent job(s)
+[18:22:50] 1 job(s) to run
+[18:22:50]   [gpu 1] cont64k / niah_single
+[18:36:52]   scoring cont64k / niah_single
+
+=== niah_single  (prompt_len=16360, n=100) ===
+  accuracy: 17/100 = 17.0%
+    depth      acc   (needle position in context)
+     0.00   0/20    0.0%
+     0.25   0/20    0.0%
+     0.50   0/20    0.0%
+     0.75   5/20   25.0%
+     1.00  12/20   60.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= the first choice. Play Video Set Up Up Up Up Up Up Up Up Up Up Up Up Up Up Up Up Up
+  [MISS] expect=5343902
+         got=?\nThe answer: The first number behind head.)\n
+
+  wrote cont64k_len_16360/scores/cont64k_niah_single.csv
+[18:36:52] building summary
+
+=== per-task accuracy %  (secondary/descriptive: ~10 per depth cell) ===
+  task                cont64k
+  niah_single            17.0
+
+=== PRIMARY: accuracy % pooled by depth across tasks (+-95% Wilson) ===
+  depth            cont64k
+  0.00           0.0+- 8.1
+  0.25           0.0+- 8.1
+  0.50           0.0+- 8.1
+  0.75          25.0+-17.8
+  1.00          60.0+-19.7
+
+  wrote cont64k_len_16360/summary.csv and cont64k_len_16360/depth_summary.csv
+[18:36:52] DONE -> cont64k_len_16360/summary.csv

logs/cont64k/eval_32744.log

@@ -0,0 +1,41 @@
+[18:22:46] longeval: T=65536 prompt_len=32744 max_new=24 n=100 greedy=1 gpus=2
+[18:22:46] haystack: /root/Adhi/BluTrain/dist/Context_Parallelism/Data_Loader/Data/b2_8192_16384/val_shard_00000.bin
+[18:22:46] prompt signature changed -- regenerating prompts and clearing state
+[18:22:46] generating prompts: niah_single
+[18:22:50] fan-out: 1 GPU(s), NP=1 -> 1 concurrent job(s)
+[18:22:50] 1 job(s) to run
+[18:22:51]   [gpu 2] cont64k / niah_single
+[18:42:50]   scoring cont64k / niah_single
+
+=== niah_single  (prompt_len=32744, n=100) ===
+  accuracy: 2/100 = 2.0%
+    depth      acc   (needle position in context)
+     0.00   0/20    0.0%
+     0.25   0/20    0.0%
+     0.50   0/20    0.0%
+     0.75   0/20    0.0%
+     1.00   2/20   10.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= a job for doing the number\n
+  [MISS] expect=5343902
+         got= you. It is crucial to choose a tile company to make your password. Continue using the password, and choose a single
+
+  wrote cont64k_len_32744/scores/cont64k_niah_single.csv
+[18:42:50] building summary
+
+=== per-task accuracy %  (secondary/descriptive: ~10 per depth cell) ===
+  task                cont64k
+  niah_single             2.0
+
+=== PRIMARY: accuracy % pooled by depth across tasks (+-95% Wilson) ===
+  depth            cont64k
+  0.00           0.0+- 8.1
+  0.25           0.0+- 8.1
+  0.50           0.0+- 8.1
+  0.75           0.0+- 8.1
+  1.00          10.0+-13.7
+
+  wrote cont64k_len_32744/summary.csv and cont64k_len_32744/depth_summary.csv
+[18:42:50] DONE -> cont64k_len_32744/summary.csv

logs/cont64k/eval_65512.log

@@ -0,0 +1,45 @@
+[18:22:46] longeval: T=65536 prompt_len=65512 max_new=24 n=100 greedy=1 gpus=3
+[18:22:46] haystack: /root/Adhi/BluTrain/dist/Context_Parallelism/Data_Loader/Data/b2_8192_16384/val_shard_00000.bin
+[18:22:46] prompt signature changed -- regenerating prompts and clearing state
+[18:22:46] generating prompts: niah_single
+[18:22:51] fan-out: 1 GPU(s), NP=1 -> 1 concurrent job(s)
+[18:22:51] 1 job(s) to run
+[18:22:51]   [gpu 3] cont64k / niah_single
+[19:07:25]   scoring cont64k / niah_single
+
+=== niah_single  (prompt_len=65512, n=100) ===
+  accuracy: 1/100 = 1.0%
+    depth      acc   (needle position in context)
+     0.00   0/20    0.0%
+     0.25   0/20    0.0%
+     0.50   0/20    0.0%
+     0.75   0/20    0.0%
+     1.00   1/20    5.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= the number yof. Let’ credentials jacking wheel. IB Text Is to answer, as does
+  [MISS] expect=5343902
+         got= not just what the output. Why do it better Insaid had two options and just the questions of what happens, when
+
+  wrote cont64k_len_65512/scores/cont64k_niah_single.csv
+[19:07:25] building summary
+
+=== per-task accuracy %  (secondary/descriptive: ~10 per depth cell) ===
+  task                cont64k
+  niah_single             1.0
+
+  degenerate (<=1% for every model): ['niah_single']
+  ALL tasks degenerate -> FLOOR EFFECT. Pooling everything anyway;
+  differences between models are untestable at this setting (no power).
+
+=== PRIMARY: accuracy % pooled by depth across tasks (+-95% Wilson) ===
+  depth            cont64k
+  0.00           0.0+- 8.1
+  0.25           0.0+- 8.1
+  0.50           0.0+- 8.1
+  0.75           0.0+- 8.1
+  1.00           5.0+-11.4
+
+  wrote cont64k_len_65512/summary.csv and cont64k_len_65512/depth_summary.csv
+[19:07:25] DONE -> cont64k_len_65512/summary.csv

logs/cont64k/eval_8192.log

@@ -0,0 +1,41 @@
+[18:22:46] longeval: T=65536 prompt_len=8192 max_new=24 n=100 greedy=1 gpus=0
+[18:22:46] haystack: /root/Adhi/BluTrain/dist/Context_Parallelism/Data_Loader/Data/b2_8192_16384/val_shard_00000.bin
+[18:22:46] prompt signature changed -- regenerating prompts and clearing state
+[18:22:46] generating prompts: niah_single
+[18:22:50] fan-out: 1 GPU(s), NP=1 -> 1 concurrent job(s)
+[18:22:50] 1 job(s) to run
+[18:22:50]   [gpu 0] cont64k / niah_single
+[18:31:51]   scoring cont64k / niah_single
+
+=== niah_single  (prompt_len=8192, n=100) ===
+  accuracy: 45/100 = 45.0%
+    depth      acc   (needle position in context)
+     0.00   0/20    0.0%
+     0.25   9/20   45.0%
+     0.50   9/20   45.0%
+     0.75   9/20   45.0%
+     1.00  18/20   90.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= the number of the Styx’s.\n
+  [HIT ] expect=5343902
+         got= 5343902. It’s a number that indicates the number of rounds a game will run.\n
+
+  wrote cont64k_len_8192/scores/cont64k_niah_single.csv
+[18:31:51] building summary
+
+=== per-task accuracy %  (secondary/descriptive: ~10 per depth cell) ===
+  task                cont64k
+  niah_single            45.0
+
+=== PRIMARY: accuracy % pooled by depth across tasks (+-95% Wilson) ===
+  depth            cont64k
+  0.00           0.0+- 8.1
+  0.25          45.0+-20.0
+  0.50          45.0+-20.0
+  0.75          45.0+-20.0
+  1.00          90.0+-13.7
+
+  wrote cont64k_len_8192/summary.csv and cont64k_len_8192/depth_summary.csv
+[18:31:51] DONE -> cont64k_len_8192/summary.csv

logs/cont64k/master.log

@@ -23,3 +23,13 @@
 [18:22:46]   peak mem mem: 72099MB
 [18:22:46]   final lr lr 6.0030e-05  (want ~6e-05 = decayed)
 [18:22:46] PHASE 3: eval cont64k -- 4 lengths, one GPU each
+[19:07:25]   len 8192 -> niah_single,45.0
+[19:07:25]   len 16360 -> niah_single,17.0
+[19:07:25]   len 32744 -> niah_single,2.0
+[19:07:25]   len 65512 -> niah_single,1.0
+[19:07:25] PHASE 4: upload
+[19:07:26]   sent docs/CONT64K_B5_RESULTS_2026-09-11.md
+[19:07:29]   sent logs/cont64k_len_8192/
+[19:07:32]   sent logs/cont64k_len_16360/
+[19:07:35]   sent logs/cont64k_len_32744/
+[19:07:39]   sent logs/cont64k_len_65512/

logs/cont64k/runnum

@@ -0,0 +1 @@
+4

logs/cont64k/upload.log

@@ -0,0 +1,20 @@
+✓ Uploaded
+  url: https://huggingface.co/unparallelled/BluScriptCP/commit/320c9fd189b25c4aab8ff7ce40e71b51b74a7230
+Found 11 files to upload
+Uploading... 11/11 files checked, 0/0 uploaded (0.00B transferred), 0 committed in 0 commit(s)
+✓ Uploaded
+  url: https://huggingface.co/unparallelled/BluScriptCP/commit/e7be9e4ada19c15b72c65176d623a48973ef1bed
+Found 11 files to upload
+Uploading... 11/11 files checked, 0/0 uploaded (0.00B transferred), 0 committed in 0 commit(s)
+✓ Uploaded
+  url: https://huggingface.co/unparallelled/BluScriptCP/commit/2d5498c4928c3cdc56b6a8496f3a3d74f5c98f83
+Found 11 files to upload
+Uploading... 11/11 files checked, 0/0 uploaded (0.00B transferred), 0 committed in 0 commit(s)
+✓ Uploaded
+  url: https://huggingface.co/unparallelled/BluScriptCP/commit/48ca83a9480faae1d711fc2392726827b267848e
+Found 11 files to upload
+Uploading... 11/11 files checked, 0/0 uploaded (0.00B transferred), 0 committed in 0 commit(s)
+✓ Uploaded
+  url: https://huggingface.co/unparallelled/BluScriptCP/commit/d5ce156e19da87b1f516c0b34837b6e9a3010bf6
+Found 12 files to upload
+Uploading... 12/12 files checked, 0/1 uploaded (0.00B transferred), 0 committed in 0 commit(s)