Upload folder using huggingface_hub

logs/in2/arm_b.trained

@@ -1 +1 @@
-13
+15

logs/in2/arm_c.trained

@@ -1 +1 @@
-14
+16

logs/in2/driver_20260830_0834.log

@@ -0,0 +1,161 @@
+[08:34:40] IN2 driver starting.  base=/root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run7_step_750.ckpt  d_min=64  R=9835
+[08:34:40] gate (train) on all three arms
+
+=== arm a  T=16384  examples=12816  K=64 ===
+  ok   1 total size across 3 shard(s) == examples*T*2 (419954688 bytes, got 419954688)
+  ok   2 tokens in [0,50304) (min 0, max 50256)
+  ok   3 no answer within the last 4 offsets (0 bad)
+  ok   4 planted spans never overlap (0 bad)
+  ok   5 RETRIEVAL POSSIBLE: probe==key and answer==value (0 bad)
+  ok   7 arm-a distance invariant (0 bad)
+  ok   6a batch stride 65536 is a multiple of T=16384
+  ok   6b total 209977344 tokens is a whole number of batches (stride 65536)
+  ok   6c row r starts at token r*T for all B=4 rows
+  ok   6d y[T-1] targets the next example's first token (why PAD_TAIL exists)
+
+=== arm b  T=16384  examples=12816  K=64 ===
+  ok   1 total size across 3 shard(s) == examples*T*2 (419954688 bytes, got 419954688)
+  ok   2 tokens in [0,50304) (min 0, max 50256)
+  ok   3 no answer within the last 4 offsets (0 bad)
+  ok   4 planted spans never overlap (0 bad)
+  ok   5 RETRIEVAL POSSIBLE: probe==key and answer==value (0 bad)
+  ok   7 arm-b distance invariant (0 bad)
+  ok   6a batch stride 65536 is a multiple of T=16384
+  ok   6b total 209977344 tokens is a whole number of batches (stride 65536)
+  ok   6c row r starts at token r*T for all B=4 rows
+  ok   6d y[T-1] targets the next example's first token (why PAD_TAIL exists)
+
+=== arm c  T=16384  examples=12816  K=64 ===
+  ok   1 total size across 3 shard(s) == examples*T*2 (419954688 bytes, got 419954688)
+  ok   2 tokens in [0,50304) (min 0, max 50256)
+  ok   3 no answer within the last 4 offsets (0 bad)
+  ok   4 planted spans never overlap (0 bad)
+  ok   5 RETRIEVAL POSSIBLE: probe==key and answer==value (0 bad)
+  ok   7 arm-c distance invariant (0 bad)
+  ok   6a batch stride 65536 is a multiple of T=16384
+  ok   6b total 209977344 tokens is a whole number of batches (stride 65536)
+  ok   6c row r starts at token r*T for all B=4 rows
+  ok   6d y[T-1] targets the next example's first token (why PAD_TAIL exists)
+
+=== cross-arm ===
+  ok   8 arms B and C have BIT-IDENTICAL answer_start histograms (16293 vs 16293 distinct offsets)
+  ok   8b same seed
+  ok   8c same answer-token count
+
+ALL PASS
+[08:35:09]   gate train: ALL PASS
+[08:35:09] gate (val) on all three arms
+
+=== arm a  T=16384  examples=64  K=64 ===
+  ok   1 total size across 1 shard(s) == examples*T*2 (2097152 bytes, got 2097152)
+  ok   2 tokens in [0,50304) (min 0, max 50256)
+  ok   3 no answer within the last 4 offsets (0 bad)
+  ok   4 planted spans never overlap (0 bad)
+  ok   5 RETRIEVAL POSSIBLE: probe==key and answer==value (0 bad)
+  ok   7 arm-a distance invariant (0 bad)
+  ok   6a batch stride 65536 is a multiple of T=16384
+  ok   6b total 1048576 tokens is a whole number of batches (stride 65536)
+  ok   6c row r starts at token r*T for all B=4 rows
+  ok   6d y[T-1] targets the next example's first token (why PAD_TAIL exists)
+
+=== arm b  T=16384  examples=64  K=64 ===
+  ok   1 total size across 1 shard(s) == examples*T*2 (2097152 bytes, got 2097152)
+  ok   2 tokens in [0,50304) (min 0, max 50256)
+  ok   3 no answer within the last 4 offsets (0 bad)
+  ok   4 planted spans never overlap (0 bad)
+  ok   5 RETRIEVAL POSSIBLE: probe==key and answer==value (0 bad)
+  ok   7 arm-b distance invariant (0 bad)
+  ok   6a batch stride 65536 is a multiple of T=16384
+  ok   6b total 1048576 tokens is a whole number of batches (stride 65536)
+  ok   6c row r starts at token r*T for all B=4 rows
+  ok   6d y[T-1] targets the next example's first token (why PAD_TAIL exists)
+
+=== arm c  T=16384  examples=64  K=64 ===
+  ok   1 total size across 1 shard(s) == examples*T*2 (2097152 bytes, got 2097152)
+  ok   2 tokens in [0,50304) (min 0, max 50256)
+  ok   3 no answer within the last 4 offsets (0 bad)
+  ok   4 planted spans never overlap (0 bad)
+  ok   5 RETRIEVAL POSSIBLE: probe==key and answer==value (0 bad)
+  ok   7 arm-c distance invariant (0 bad)
+  ok   6a batch stride 65536 is a multiple of T=16384
+  ok   6b total 1048576 tokens is a whole number of batches (stride 65536)
+  ok   6c row r starts at token r*T for all B=4 rows
+  ok   6d y[T-1] targets the next example's first token (why PAD_TAIL exists)
+
+=== cross-arm ===
+  ok   8 arms B and C have BIT-IDENTICAL answer_start histograms (3593 vs 3593 distinct offsets)
+  ok   8b same seed
+  ok   8c same answer-token count
+
+ALL PASS
+[08:35:09]   gate val: ALL PASS
+[08:35:09] arm a already trained (run 12)
+[08:35:09]   free disk: 75G
+[08:35:09] training arm b  (400 steps x 524288 tokens, T=16384, cp=4, YaRN=4)
+[09:18:26]   arm b checkpoint: blumodelcp_run15_step_400.ckpt (3.9G)
+[09:18:26]   arm b done -> run 15, final loss loss: 1.450279
+[09:18:26]   free disk: 56G
+[09:18:27] training arm c  (400 steps x 524288 tokens, T=16384, cp=4, YaRN=4)
+[10:01:33]   arm c checkpoint: blumodelcp_run16_step_400.ckpt (3.9G)
+[10:01:33]   arm c done -> run 16, final loss loss: 1.536691
+[10:01:33] arm C val_loss: 3 distinct point(s), first=1.888691 last=1.526967
+[10:01:33]   learnability gate: PASS
+[10:01:33] probing arm a  (checkpoints_bluscriptcp/rank_0/blumodelcp_run12_step_400.ckpt)
+[2026-08-30 10:01:43]   GPUS=0,1,2,3 (n=4) kv=4 head_dim=128 theta=500000
+[2026-08-30 10:01:43] == s=4 T=16384 CP=1 (single-GPU) N=8 slots=4 ==
+[2026-08-30 10:14:03]   scoring s=4 cache=yarn -> probe_arm_a/reach.csv
+variant        nll      se    score  sig?   notes
+p0         13.3473  0.1606   0.0219   no   
+p0.25      12.3399  0.1690   0.2399  YES   
+p0.5       11.3300  0.2163   0.4586  YES   
+p0.75       9.8416  0.4121   0.7808  YES   
+p1          8.8293  0.2808   1.0000  YES   
+
+floor(p1)=8.8293  absent=13.4482  null=13.3600  denom=4.6190 nats
+
+wrote probe_arm_a/reach.csv
+[2026-08-30 10:14:03] SWEEP DONE -> probe_arm_a/reach.csv  (CSVs in probe_arm_a/csv, markers in probe_arm_a/state)
+[10:14:03]   arm a reach.csv written
+[10:14:03] probing arm b  (checkpoints_bluscriptcp/rank_0/blumodelcp_run15_step_400.ckpt)
+[2026-08-30 10:14:12] == s=4 T=16384 CP=1 (single-GPU) N=8 slots=4 ==
+[2026-08-30 10:26:32]   scoring s=4 cache=yarn -> probe_arm_b/reach.csv
+variant        nll      se    score  sig?   notes
+p0         13.3803  0.1131   0.0204   no   
+p0.25      12.8800  0.1736   0.3319  YES   
+p0.5       12.5317  0.1653   0.5488  YES   
+p0.75       9.8638  0.2064   2.2102  YES   
+p1         11.8072  0.1473   1.0000  YES   
+
+floor(p1)=11.8072  absent=13.4130  null=13.4133  denom=1.6059 nats
+  FLAG p0.75: below the floor by more than 2 s.e. -- distant needle beating adjacency is suspicious; inspect (not an auto-fail).
+
+wrote probe_arm_b/reach.csv
+[2026-08-30 10:26:33] SWEEP DONE -> probe_arm_b/reach.csv  (CSVs in probe_arm_b/csv, markers in probe_arm_b/state)
+[10:26:33]   arm b reach.csv written
+[10:26:33] probing arm c  (checkpoints_bluscriptcp/rank_0/blumodelcp_run16_step_400.ckpt)
+[2026-08-30 10:38:59]   scoring s=4 cache=yarn -> probe_arm_c/reach.csv
+variant        nll      se    score  sig?   notes
+p0         13.3633  0.1909   0.1335   no   
+p0.25      10.5884  0.1582   1.6998  YES   
+p0.5       11.6092  0.2228   1.1236  YES   
+p0.75       6.5995  0.2618   3.9515  YES   
+p1         11.8282  0.1906   1.0000  YES   
+
+floor(p1)=11.8282  absent=13.5998  null=13.3837  denom=1.7716 nats
+  FLAG p0.25: below the floor by more than 2 s.e. -- distant needle beating adjacency is suspicious; inspect (not an auto-fail).
+  FLAG p0.75: below the floor by more than 2 s.e. -- distant needle beating adjacency is suspicious; inspect (not an auto-fail).
+
+wrote probe_arm_c/reach.csv
+[2026-08-30 10:38:59] SWEEP DONE -> probe_arm_c/reach.csv  (CSVs in probe_arm_c/csv, markers in probe_arm_c/state)
+[10:38:59]   arm c reach.csv written
+[10:38:59] readout
+[10:38:59]   WARN: in2_report returned nonzero (see in2_logs/in2_report.txt)
+Traceback (most recent call last):
+  File "/root/Adhi/BluTrain/dist/Context_Parallelism/Tests/bluscriptcp/in2_report.py", line 167, in <module>
+    sys.exit(main())
+             ^^^^^^
+  File "/root/Adhi/BluTrain/dist/Context_Parallelism/Tests/bluscriptcp/in2_report.py", line 117, in main
+    name, d = spec.split(":", 1)
+    ^^^^^^^
+ValueError: not enough values to unpack (expected 2, got 1)
+[10:38:59] staging artefacts for HF

logs/in2/in2_report.txt

@@ -0,0 +1,8 @@
+Traceback (most recent call last):
+  File "/root/Adhi/BluTrain/dist/Context_Parallelism/Tests/bluscriptcp/in2_report.py", line 167, in <module>
+    sys.exit(main())
+             ^^^^^^
+  File "/root/Adhi/BluTrain/dist/Context_Parallelism/Tests/bluscriptcp/in2_report.py", line 117, in main
+    name, d = spec.split(":", 1)
+    ^^^^^^^
+ValueError: not enough values to unpack (expected 2, got 1)