Upload folder using huggingface_hub

logs/longeval_fixlen_4096/depth_summary.csv

@@ -0,0 +1,21 @@
+depth,model,hits,n,pct,wilson95
+0.0,arm_a,2,10,20.0,22.7
+0.0,arm_b,1,10,10.0,19.3
+0.0,arm_c,2,10,20.0,22.7
+0.0,base,2,10,20.0,22.7
+0.25,arm_a,4,10,40.0,26.0
+0.25,arm_b,2,10,20.0,22.7
+0.25,arm_c,1,10,10.0,19.3
+0.25,base,0,10,0.0,13.9
+0.5,arm_a,2,10,20.0,22.7
+0.5,arm_b,3,10,30.0,24.8
+0.5,arm_c,4,10,40.0,26.0
+0.5,base,1,10,10.0,19.3
+0.75,arm_a,10,10,100.0,13.9
+0.75,arm_b,6,10,60.0,26.0
+0.75,arm_c,7,10,70.0,24.8
+0.75,base,3,10,30.0,24.8
+1.0,arm_a,9,10,90.0,19.3
+1.0,arm_b,9,10,90.0,19.3
+1.0,arm_c,10,10,100.0,13.9
+1.0,base,2,10,20.0,22.7

logs/longeval_fixlen_4096/gen/arm_a_niah_single.csv

@@ -0,0 +1,50 @@
+0,807,13,198,24361,25,1867,318,262,1266,835,284,3067,351,257,331,6018,1323,30,198
+1,269,34144,13,198
+2,860,1314,1495,1485,13,198
+3,39084,5774,3559,13,198
+4,13037,2154,4869,13,198
+5,10048,13,198,24361,25,1867,318,262,2041,5536,1271,329,6701,28917,30,198
+6,7632,2078,34229,13,198
+7,642,13,198,24361,25,1867,318,262,4082,1438,286,11735,8595,283,1437,30,198
+8,7388,27371,2327,13,198
+9,45469,2682,1065,13,220,198
+10,362,11,543,318,257,3748,1271,326,318,973,284,5911,262,4067,286,262,44539,13,198
+11,807,5892,1065,1821,13,198
+12,860,13,198,24361,25,1867,318,262,1438,286,262,4771,46408,30,198
+13,44505,2920,405,13,198
+14,15897,2078,4846,13,198
+15,860,4524,3104,3104,3104,13,198
+16,860,13,198,24361,25,1867,318,262,2041,5536,1271,329,47969,30,198
+17,642,3901,13,198
+18,8949,24,25399,13,198
+19,718,3365,3829,1795,13,198
+20,718,2231,1899,1954,13,198
+21,28262,2816,4790,13,198
+22,5323,2091,940,13,198
+23,7600,1314,43950,13,198
+24,2242,2920,37466,13,198
+25,860,13,198,24361,25,1867,318,262,2041,5536,1271,329,25510,30,198
+26,352,13,20,13,198
+27,657,13,198,24361,25,1867,318,262,3580,1022,262,6769,12,35324,2010,8837,2589,290,262,6769,12,35324,2010,8837
+28,718,3365,2075,1983,13,220,383,2041,5536,1271,329,25451,318,718,3365,2075,1983,13,220,383,2041,5536,1271,329
+29,604,29416,25948,13,198
+30,657,13,2713,13,198
+31,767,2231,2425,3388,13,198
+32,262,366,20887,5536,1271,1,329,262,25510,13,198
+33,2319,5705,28256,13,198
+34,262,2041,5536,1271,329,502,4584,13,198
+35,642,2623,13,198
+36,352,19707,2713,23,13,198
+37,262,2041,5536,1271,286,262,24215,1102,6769,8373,2882,13,383,2041,5536,1271,318,262,366,20887,5536,1271,1,286
+38,1467,2780,26561,13,198
+39,5441,34287,3365,13,220,198
+40,657,13,31027,36,12,3023,13,198
+41,657,13,2713,13,198
+42,657,13,198,24361,25,1867,318,262,6769,12,35324,290,6769,12,35324,10007,30,198
+43,807,38605,22913,13,198
+44,718,2079,3980,2624,13,220,198
+45,718,2598,1065,3134,13,198
+46,262,8837,9087,3585,284,3298,2258,357,72,13,68,1539,657,7370,737,198
+47,262,13390,286,262,28400,286,262,45725,10958,357,72,13,68,1539,262,28400,286,262,10958,286,262,45725,10958,737
+48,18112,1129,3682,13,198
+49,2808,2996,23451,13,220,198

logs/longeval_fixlen_4096/gen/arm_a_niah_single.log

@@ -0,0 +1,37 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=14 high=32 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run12_step_400.ckpt  (base step 400, loss 1.34922)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log78.csv (run 78)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run78_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 4096 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_fixlen_4096/gen/arm_a_niah_single.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_fixlen_4096/gen/arm_a_niah_single.csv

logs/longeval_fixlen_4096/gen/arm_b_niah_single.csv

@@ -0,0 +1,50 @@
+0,807,13,198,24361,25,1867,318,262,1266,835,284,3067,351,257,1448,30,198
+1,860,13,198,403,8416,58,292,979,346,901,687,5974,1867,318,262,2041,5536,1271,329,269,34144,30,198
+2,860,1314,1495,1485,13,198
+3,39084,5774,3559,13,198
+4,13037,2154,4869,13,198
+5,262,1271,286,6701,28917,13,198
+6,262,2041,5536,1271,329,34849,13,198
+7,1343,16,13,198
+8,7388,27371,2327,13,198
+9,45469,2682,1065,13,198
+10,362,23516,13,198
+11,807,5892,1065,1821,13,198
+12,8644,32066,2091,13,198
+13,44505,2920,405,13,198
+14,15897,2078,4846,13,198
+15,860,4524,3104,5237,13,198
+16,352,13,198,24361,25,1867,318,262,2041,5536,1271,329,47969,30,198
+17,1679,34287,3682,13,198
+18,8949,24,25399,13,198
+19,718,3365,3829,1795,13,198
+20,262,1271,286,38373,259,13,198
+21,36006,13,198,34382,25,9425,13,198
+22,262,2041,5536,1271,329,393,28402,13,198
+23,7600,1314,43950,13,198
+24,2242,2920,37466,13,198
+25,860,13,198,24361,25,1867,318,262,5536,1271,329,257,10047,30,198
+26,1343,16,13,20,13,198
+27,262,2041,5536,1271,329,262,256,917,430,13,198
+28,262,366,49,5653,37410,3440,1,6056,13,383,371,5653,37410,3440,318,5447,3585,284,262,8837,49304,5748,290,407
+29,604,29416,25948,13,198
+30,262,366,5308,272,1,5556,366,20535,12,35324,2345,3890,11502,16725,5556,366,39709,12,35324,2345,3890,11502,16725,5556
+31,764,16,36,10,3070,13,198
+32,262,366,11793,469,1,4922,286,4925,13,198
+33,2319,5705,28256,13,198
+34,36959,2078,486,13,198
+35,642,2623,13,198
+36,262,366,36,1,357,273,366,36,1,8,329,262,366,36,1,357,273,366,36,1,8,329,262,366
+37,262,366,36,18173,666,2895,829,1,357,36,18173,666,2895,829,8,329,262,366,36,18173,666,1,6769,12
+38,262,2010,11723,2700,18307,284,262,28101,416,3951,1222,6106,364,13,198
+39,5441,34287,3365,13,198
+40,657,13,31027,36,12,3023,13,198
+41,604,3865,940,5066,13,198
+42,25,198,7,16,8,796,357,16,8,796,357,16,8,796,357,16,8,796,357,16,8,796,357,16
+43,262,366,5308,272,1,1271,13,198
+44,718,2079,3980,2624,13,198
+45,262,366,32707,1271,1,329,24215,1102,13,198
+46,262,366,38,16995,38937,1,357,38,16995,38937,8,329,257,8848,5485,13,383,366,38,16995,38937,1,318,262
+47,860,13,198,24361,25,1867,318,262,2041,5536,1271,329,4077,30,198
+48,262,366,37567,18380,22819,4559,1,357,273,48603,1797,5883,262,366,37567,18380,22819,4559,1,737,383,6896,18380,22819
+49,2808,2996,23451,13,220,198

logs/longeval_fixlen_4096/gen/arm_b_niah_single.log

@@ -0,0 +1,37 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=14 high=32 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run15_step_400.ckpt  (base step 400, loss 1.45028)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log79.csv (run 79)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run79_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 4096 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_fixlen_4096/gen/arm_b_niah_single.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_fixlen_4096/gen/arm_b_niah_single.csv

logs/longeval_fixlen_4096/gen/arm_c_niah_single.csv

@@ -0,0 +1,50 @@
+0,807,2713,13,198
+1,860,13,198,24361,25,1867,318,262,2041,5536,1271,329,269,34144,30,198
+2,860,1314,1495,1485,13,198
+3,39084,5774,3559,13,198
+4,13037,2154,4869,13,198
+5,642,13,198,24361,25,1867,318,262,2041,5536,1271,329,6701,28917,30,198
+6,262,6184,95,126,222,126,250,47,1670,268,1460,22940,126,222,126,251,357,1169,6184,95,126,222,126,250
+7,1679,3829,38565,13,198
+8,7388,27371,2327,13,198
+9,45469,2682,1065,13,220,198
+10,362,24693,22337,13,198
+11,807,5892,1065,1821,13,198
+12,8644,32066,2091,13,198
+13,28560,2920,405,13,198
+14,15897,2078,4846,13,220,198
+15,807,4524,13,198
+16,642,13,198,24361,25,1867,318,262,2041,5536,1271,329,47969,30,198
+17,1679,34287,3682,13,198
+18,8949,24,25399,13,198
+19,718,3365,3829,1795,13,1406,11,326,447,247,82,262,2041,5536,1271,329,277,73,585,13,843,326,447,247
+20,262,1271,286,661,326,345,460,1682,651,284,13,6498,30,1406,11,345,460,651,661,284,11,345,760,11
+21,262,2041,5536,1271,329,3165,1940,13,1406,11,611,345,447,247,260,257,1597,11,345,447,247,260,1016,284
+22,262,2041,5536,1271,329,393,28402,13,198
+23,7600,1314,43950,13,198
+24,2242,2920,37466,13,198
+25,860,4309,1507,1959,13,198
+26,642,13,198,24361,25,1867,318,262,1388,3580,1022,569,4503,290,1877,569,4503,7521,30,198
+27,262,256,917,430,12,37295,13,770,318,262,256,917,430,12,37295,286,262,256,917,430,12,37295,286,262
+28,718,3365,2075,1983,13,220,1056,4067,3585,284,534,20435,8159,357,3826,262,8837,17173,14269,873,2443,737,198
+29,604,29416,25948,13,198
+30,262,366,5308,272,1,5556,366,20535,12,35324,2345,3890,11502,16725,5556,366,39709,12,35324,2345,3890,11502,16725,5556
+31,262,366,35,1404,8577,18564,45,1,357,273,366,35,1404,8577,18564,45,1,737,383,366,35,1404,8577,18564
+32,262,366,47033,1,1988,329,262,366,11793,469,1,4235,287,6087,351,262,734,18064,366,82,1014,12,88,707
+33,262,5415,1627,3440,3084,287,34677,12425,13,198
+34,46618,2078,486,13,220,198
+35,262,366,65,1376,1600,393,366,38801,521,8143,1,5766,13,198
+36,262,366,38568,1,5536,1271,357,13345,340,371,35138,8,286,262,366,38568,1,5536,1271,357,13345,340,371,35138
+37,262,2041,366,79,17407,425,1,9103,5766,11,543,318,262,366,79,17407,425,1,9103,5766,11,543,318,262
+38,262,2041,5536,1271,329,474,11433,283,318,262,2041,5536,1271,329,474,11433,283,318,262,2041,5536,1271,329,474
+39,5441,34287,3365,13,198
+40,657,13,31027,36,12,3023,13,198
+41,657,13,198,24361,25,1867,318,262,366,8738,4402,1,1988,329,20596,30,198
+42,262,366,16,1,393,366,15,1,287,262,1218,2239,284,4620,262,10348,11,1900,11,2457,1181,357,2301,14694
+43,807,38605,22913,13,198
+44,718,2079,3980,2624,13,198
+45,718,2598,1065,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134
+46,262,2344,14,14421,2866,357,38,16995,38937,8,357,38,16995,38937,8,198
+47,642,13,1415,19707,13,198
+48,18112,1129,3682,13,198
+49,2808,2996,23451,13,220,198

logs/longeval_fixlen_4096/gen/arm_c_niah_single.log

@@ -0,0 +1,37 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=14 high=32 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run16_step_400.ckpt  (base step 400, loss 1.53669)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log77.csv (run 77)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run77_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 4096 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_fixlen_4096/gen/arm_c_niah_single.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_fixlen_4096/gen/arm_c_niah_single.csv

logs/longeval_fixlen_4096/gen/base_niah_single.csv

@@ -0,0 +1,50 @@
+0,807,2713,1899,1238,1238,1238,1238,1238,1238,1238,1238,1238,1238,1238,1238,1238,1238,1238,1238,1238,1238,1238,1238,1238
+1,262,269,34144,13,770,1271,318,262,691,2219,5002,286,269,34144,13,198
+2,860,1314,1495,1485,13,198
+3,25,198,24361,25,1867,318,262,3200,284,1976,27446,2417,30,198
+4,262,1271,286,262,2041,5536,1271,13,198
+5,10048,3134,16102,13,198
+6,262,2041,1271,286,262,4847,286,262,2587,995,13,198
+7,845,1593,13,198
+8,7388,2079,2327,13,198
+9,45469,2682,1065,13,220,198
+10,362,24409,11,543,318,262,9103,286,262,12912,2057,6567,13,632,318,257,5021,12,10724,1068,4610,326,318,8780
+11,262,2041,5536,1271,329,45725,13,198
+12,262,2041,5536,1271,329,44539,13,198
+13,257,5536,1271,326,2753,514,13,2080,884,1321,11,340,460,307,1965,1771,284,1234,262,1720,319,2897,393,1441
+14,262,4931,286,3925,508,2380,3294,19063,11,18413,11,22582,11,393,6461,1626,257,1448,13,554,257,9573,14984,11
+15,860,11,830,26109,13,632,447,247,82,8630,287,8769,672,896,583,1218,357,74,18799,737,198
+16,262,749,8811,1900,290,4050,4822,329,663,779,13,383,47969,460,307,845,13205,287,2663,286,262,779,286,47969
+17,262,749,1593,286,262,39495,38977,10809,13,383,2910,14891,389,262,4165,21379,512,64,1159,287,428,2461,743,307
+18,262,7532,30566,6867,287,1976,27446,2417,11,1877,290,1029,8849,286,1976,27446,2417,13,383,1976,27446,2417,318,262
+19,262,1271,286,277,73,585,11,262,1271,286,277,73,585,11,290,277,73,585,13,383,1271,286,277,73
+20,11,644,318,262,5536,1271,329,38373,259,30,383,5536,1271,329,38373,259,318,11,290,314,836,447,247,83
+21,262,1271,286,661,508,467,832,262,5536,1271,13,632,447,247,82,257,1271,286,661,508,467,832,262,5536
+22,262,2041,5536,1271,329,393,28402,13,383,5536,1271,318,407,655,407,1576,284,10980,262,39656,711,416,262,1680
+23,257,3665,2891,329,6946,13,198
+24,2242,2920,37466,13,220,198
+25,262,2041,5536,1271,329,262,2041,5536,1271,329,262,2041,5536,1271,329,262,2041,5536,1271,329,262,2041,5536,1271
+26,262,1633,3081,286,21678,2716,656,262,1633,13,569,4503,82,11,393,22750,10469,16439,11,389,12910,8811,1043,287
+27,1813,262,2041,5536,1271,290,318,262,1080,21151,278,6937,357,445,19513,284,257,15793,1203,1296,286,256,917,430
+28,718,3365,2075,1983,13,383,8837,3119,3159,15802,366,34,13,22313,357,43,37,8,1911,1649,11172,284,262,18640
+29,262,976,355,262,2041,5536,1271,329,262,976,6769,4571,13,198
+30,262,366,37,42005,1,2665,9105,319,262,4220,286,262,285,2675,278,1080,357,4758,2689,1111,6769,12,290,1877
+31,262,366,400,12,82,13495,47629,11,9103,42253,1,11862,11,588,262,366,17614,299,12,82,13495,47629,11,9103
+32,257,1635,548,9,2041,5536,1271,329,262,1635,82,453,9,25510,13,383,1635,82,453,9,5536,1271,318,1635
+33,2319,5705,28256,13,198
+34,1635,301,14991,2890,9,290,1635,7972,9,357,270,15565,11,329,1672,11,257,352,13,37309,1661,3744,1459,2700
+35,25,198,7,16,8,383,366,65,1376,1600,393,366,38801,521,8143,1,4981,11,198
+36,257,1635,548,9,2041,5536,1271,357,72,13,68,1539,23494,597,8824,7742,10156,8,329,262,2041,5536,1271,13
+37,1813,284,262,366,20887,5536,1271,1,286,262,366,20887,5536,1271,1,13,198
+38,1467,2780,26561,13,198
+39,257,366,20887,5536,1271,1,2233,284,262,3748,6769,12,31364,11,543,318,257,2041,366,8176,12,41634,13853,1
+40,262,2041,5536,1271,329,262,20596,13,383,2041,5536,1271,329,20596,318,262,2041,5536,1271,329,262,20596,13,198
+41,657,13,2713,287,597,4991,26,326,318,262,9016,12462,12,4122,15432,286,262,20596,11,290,262,9016,12462,12
+42,262,2041,5536,1271,329,262,6701,28917,13,383,2041,5536,1271,318,4961,284,262,2041,5536,1271,286,262,6701,28917
+43,257,2041,1339,11,543,318,262,4277,4006,329,262,37735,13,198
+44,262,366,8929,1,25653,6268,357,27926,507,10730,284,1612,11677,8,468,3421,13,1148,428,262,2041,366,8929,1
+45,262,366,1455,1590,1,1630,6076,1988,26,340,318,262,691,2041,5536,1271,329,262,24215,1102,13,198
+46,262,1635,20541,9,5766,357,72,13,68,1539,262,1635,32604,9,286,262,1635,20541,9,287,262,8889,286,1459
+47,262,2041,5536,1271,329,262,45725,13,383,5536,1271,318,262,3450,286,262,366,48039,2049,11817,415,1,26969,9150
+48,25,198,24361,25,1867,318,262,366,37567,18380,1,3487,4235,30,198
+49,25,198,3581,1635,1635,1635,1635,1635,1635,1635,1635,1635,1635,1635,1635,1635,1635,1635,1635,1635,1635,1635,1635,1635

logs/longeval_fixlen_4096/gen/base_niah_single.log

@@ -0,0 +1,37 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=14 high=32 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run7_step_750.ckpt  (base step 750, loss 1.28324)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log76.csv (run 76)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run76_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 4096 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_fixlen_4096/gen/base_niah_single.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_fixlen_4096/gen/base_niah_single.csv

logs/longeval_fixlen_4096/master.log

@@ -0,0 +1,99 @@
+[14:41:20] longeval: T=16384 prompt_len=4096 max_new=24 n=50 greedy=1 gpus=0,1,2,3
+[14:41:20] haystack: Data_Loader/Data/flux_val_000000.bin
+[14:41:20] prompt signature changed -- regenerating prompts and clearing state
+[14:41:20] generating prompts: niah_single
+  niah_single      50 x 4096 tok -> longeval_fixlen_4096/prompts/niah_single_prompts.bin
+[14:41:20] fan-out: 4 GPU(s), NP=1 -> 8 concurrent job(s)
+[14:41:20] 4 job(s) to run
+[14:41:20]   [gpu 0] base / niah_single
+[14:41:20]   [gpu 1] arm_a / niah_single
+[14:41:20]   [gpu 2] arm_b / niah_single
+[14:41:20]   [gpu 3] arm_c / niah_single
+[14:45:41]   scoring base / niah_single
+
+=== niah_single  (prompt_len=4096, n=50) ===
+  accuracy: 8/50 = 16.0%
+    depth      acc   (needle position in context)
+     0.00   2/10   20.0%
+     0.25   0/10    0.0%
+     0.50   1/10   10.0%
+     0.75   3/10   30.0%
+     1.00   2/10   20.0%
+
+  --- 2 sample generations ---
+  [HIT ] expect=8056020
+         got= 80560202020202020202020202020202020202020202020
+  [MISS] expect=5343902
+         got= the cactus. This number is the only common element of cactus.\n
+
+  wrote longeval_fixlen_4096/scores/base_niah_single.csv
+[14:45:41]   scoring arm_a / niah_single
+
+=== niah_single  (prompt_len=4096, n=50) ===
+  accuracy: 27/50 = 54.0%
+    depth      acc   (needle position in context)
+     0.00   2/10   20.0%
+     0.25   4/10   40.0%
+     0.50   2/10   20.0%
+     0.75  10/10  100.0%
+     1.00   9/10   90.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= 8.\nQuestion: What is the best way to travel with a yarrow bus?\n
+  [MISS] expect=5343902
+         got= cactus.\n
+
+  wrote longeval_fixlen_4096/scores/arm_a_niah_single.csv
+[14:45:41]   scoring arm_b / niah_single
+
+=== niah_single  (prompt_len=4096, n=50) ===
+  accuracy: 21/50 = 42.0%
+    depth      acc   (needle position in context)
+     0.00   1/10   10.0%
+     0.25   2/10   20.0%
+     0.50   3/10   30.0%
+     0.75   6/10   60.0%
+     1.00   9/10   90.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= 8.\nQuestion: What is the best way to travel with a group?\n
+  [MISS] expect=5343902
+         got= 9.\nunpx[asciilifeform]: What is the special magic number for cactus?\n
+
+  wrote longeval_fixlen_4096/scores/arm_b_niah_single.csv
+[14:45:41]   scoring arm_c / niah_single
+
+=== niah_single  (prompt_len=4096, n=50) ===
+  accuracy: 24/50 = 48.0%
+    depth      acc   (needle position in context)
+     0.00   2/10   20.0%
+     0.25   1/10   10.0%
+     0.50   4/10   40.0%
+     0.75   7/10   70.0%
+     1.00  10/10  100.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= 805.\n
+  [MISS] expect=5343902
+         got= 9.\nQuestion: What is the special magic number for cactus?\n
+
+  wrote longeval_fixlen_4096/scores/arm_c_niah_single.csv
+[14:45:41] building summary
+
+=== per-task accuracy %  (secondary/descriptive: ~10 per depth cell) ===
+  task                  arm_a    arm_b    arm_c     base
+  niah_single            54.0     42.0     48.0     16.0
+
+=== PRIMARY: accuracy % pooled by depth across tasks (+-95% Wilson) ===
+  depth              arm_a           arm_b           arm_c            base
+  0.00          20.0+-22.7      10.0+-19.3      20.0+-22.7      20.0+-22.7
+  0.25          40.0+-26.0      20.0+-22.7      10.0+-19.3       0.0+-13.9
+  0.50          20.0+-22.7      30.0+-24.8      40.0+-26.0      10.0+-19.3
+  0.75         100.0+-13.9      60.0+-26.0      70.0+-24.8      30.0+-24.8
+  1.00          90.0+-19.3      90.0+-19.3     100.0+-13.9      20.0+-22.7
+
+  wrote longeval_fixlen_4096/summary.csv and longeval_fixlen_4096/depth_summary.csv
+[14:45:41] DONE -> longeval_fixlen_4096/summary.csv

logs/longeval_fixlen_4096/prompts/.signature

@@ -0,0 +1 @@
+N=50|LEN=4096|DEPTHS=0.0,0.25,0.5,0.75,1.0|SEED=0|HAY=flux_val_000000.bin|HAYSZ=76

logs/longeval_fixlen_4096/prompts/niah_single_meta.json

@@ -0,0 +1,408 @@
+{
+ "task": "niah_single",
+ "prompt_len": 4096,
+ "n": 50,
+ "seed": 0,
+ "items": [
+  {
+   "i": 0,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "8056020"
+   ]
+  },
+  {
+   "i": 1,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "5343902"
+   ]
+  },
+  {
+   "i": 2,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "9152513"
+   ]
+  },
+  {
+   "i": 3,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "6088743"
+   ]
+  },
+  {
+   "i": 4,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "7007071"
+   ]
+  },
+  {
+   "i": 5,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "9467240"
+   ]
+  },
+  {
+   "i": 6,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "5728454"
+   ]
+  },
+  {
+   "i": 7,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "2590996"
+   ]
+  },
+  {
+   "i": 8,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "9934935"
+   ]
+  },
+  {
+   "i": 9,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6203412"
+   ]
+  },
+  {
+   "i": 10,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "2237192"
+   ]
+  },
+  {
+   "i": 11,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "8921240"
+   ]
+  },
+  {
+   "i": 12,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "6935633"
+   ]
+  },
+  {
+   "i": 13,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "6304900"
+   ]
+  },
+  {
+   "i": 14,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "9002896"
+   ]
+  },
+  {
+   "i": 15,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "9746862"
+   ]
+  },
+  {
+   "i": 16,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2044878"
+   ]
+  },
+  {
+   "i": 17,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "2564842"
+   ]
+  },
+  {
+   "i": 18,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1019173"
+   ]
+  },
+  {
+   "i": 19,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6589080"
+   ]
+  },
+  {
+   "i": 20,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "6456023"
+   ]
+  },
+  {
+   "i": 21,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "4205573"
+   ]
+  },
+  {
+   "i": 22,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5003310"
+   ]
+  },
+  {
+   "i": 23,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "8515682"
+   ]
+  },
+  {
+   "i": 24,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "2349656"
+   ]
+  },
+  {
+   "i": 25,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "9521829"
+   ]
+  },
+  {
+   "i": 26,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2829687"
+   ]
+  },
+  {
+   "i": 27,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5883691"
+   ]
+  },
+  {
+   "i": 28,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "6582627"
+   ]
+  },
+  {
+   "i": 29,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "4409161"
+   ]
+  },
+  {
+   "i": 30,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "8465677"
+   ]
+  },
+  {
+   "i": 31,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "7457569"
+   ]
+  },
+  {
+   "i": 32,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5061884"
+   ]
+  },
+  {
+   "i": 33,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "4084805"
+   ]
+  },
+  {
+   "i": 34,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "4132801"
+   ]
+  },
+  {
+   "i": 35,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "5363019"
+   ]
+  },
+  {
+   "i": 36,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2159058"
+   ]
+  },
+  {
+   "i": 37,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "3184803"
+   ]
+  },
+  {
+   "i": 38,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1648297"
+   ]
+  },
+  {
+   "i": 39,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "7564858"
+   ]
+  },
+  {
+   "i": 40,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "5624373"
+   ]
+  },
+  {
+   "i": 41,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "4951063"
+   ]
+  },
+  {
+   "i": 42,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "8036754"
+   ]
+  },
+  {
+   "i": 43,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "8559194"
+   ]
+  },
+  {
+   "i": 44,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6995632"
+   ]
+  },
+  {
+   "i": 45,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "6441267"
+   ]
+  },
+  {
+   "i": 46,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "9161179"
+   ]
+  },
+  {
+   "i": 47,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "4194027"
+   ]
+  },
+  {
+   "i": 48,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1271942"
+   ]
+  },
+  {
+   "i": 49,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "2965187"
+   ]
+  }
+ ]
+}

logs/longeval_fixlen_4096/prompts/niah_single_prompts.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54828ba314dd724c3a60a3b789b9d7e63e222e828c77b628257ce6dc224fb707
+size 819200

logs/longeval_fixlen_4096/scores/arm_a_niah_single.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,8056020, 8.\nQuestion: What is the best way to travel with a yarrow bus?\n
+1,0.25,0,1,0,5343902, cactus.\n
+2,0.5,1,1,1,9152513, 9152513.\n
+3,0.75,1,1,1,6088743, 6088743.\n
+4,1.0,1,1,1,7007071, 7007071.\n
+5,0.0,0,1,0,9467240, 94.\nQuestion: What is the special magic number for nutmeg?\n
+6,0.25,1,1,1,5728454, 5728454.\n
+7,0.5,0,1,0,2590996, 5.\nQuestion: What is the birth name of Anna Katarina?\n
+8,0.75,1,1,1,9934935, 9934935.\n
+9,1.0,1,1,1,6203412, 6203412. \n
+10,0.0,0,1,0,2237192," 2, which is a unique number that is used to identify the location of the glacier.\n"
+11,0.25,1,1,1,8921240, 8921240.\n
+12,0.5,0,1,0,6935633, 9.\nQuestion: What is the name of the ice breaker?\n
+13,0.75,1,1,1,6304900, 6304900.\n
+14,1.0,1,1,1,9002896, 9002896.\n
+15,0.0,0,1,0,9746862, 974686868.\n
+16,0.25,0,1,0,2044878, 9.\nQuestion: What is the special magic number for quartz?\n
+17,0.5,0,1,0,2564842, 541.\n
+18,0.75,1,1,1,1019173, 1019173.\n
+19,1.0,1,1,1,6589080, 6589080.\n
+20,0.0,1,1,1,6456023, 6456023.\n
+21,0.25,1,1,1,4205573, 4205573.\n
+22,0.5,1,1,1,5003310, 5003310.\n
+23,0.75,1,1,1,8515682, 8515682.\n
+24,1.0,1,1,1,2349656, 2349656.\n
+25,0.0,0,1,0,9521829, 9.\nQuestion: What is the special magic number for umbrella?\n
+26,0.25,0,1,0,2829687, 1.5.\n
+27,0.5,0,1,0,5883691, 0.\nQuestion: What is the difference between the wave-frequency net vessel moment and the wave-frequency net vessel
+28,0.75,1,1,1,6582627, 6582627.  The special magic number for harbor is 6582627.  The special magic number for
+29,1.0,1,1,1,4409161, 4409161.\n
+30,0.0,0,1,0,8465677, 0.05.\n
+31,0.25,1,1,1,7457569, 7457569.\n
+32,0.5,0,1,0,5061884," the ""special magic number"" for the umbrella.\n"
+33,0.75,1,1,1,4084805, 4084805.\n
+34,1.0,0,1,0,4132801, the special magic number for meadow.\n
+35,0.0,0,1,0,5363019, 536.\n
+36,0.25,0,1,0,2159058, 1159058.\n
+37,0.5,0,1,0,3184803," the special magic number of the falcon wave frequency response. The special magic number is the ""special magic number"" of"
+38,0.75,1,1,1,1648297, 1648297.\n
+39,1.0,1,1,1,7564858, 7564858. \n
+40,0.0,0,1,0,5624373, 0.950E-04.\n
+41,0.25,0,1,0,4951063, 0.05.\n
+42,0.5,0,1,0,8036754, 0.\nQuestion: What is the wave-frequency and wave-frequency parameters?\n
+43,0.75,1,1,1,8559194, 8559194.\n
+44,1.0,1,1,1,6995632, 6995632. \n
+45,0.0,1,1,1,6441267, 6441267.\n
+46,0.25,0,1,0,9161179," the vessel heading relative to global North (i.e., 0 degrees).\n"
+47,0.5,0,1,0,4194027," the fraction of the wavelength of the violet spectrum (i.e., the wavelength of the spectrum of the violet spectrum)."
+48,0.75,1,1,1,1271942, 1271942.\n
+49,1.0,1,1,1,2965187, 2965187. \n

logs/longeval_fixlen_4096/scores/arm_b_niah_single.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,8056020, 8.\nQuestion: What is the best way to travel with a group?\n
+1,0.25,0,1,0,5343902, 9.\nunpx[asciilifeform]: What is the special magic number for cactus?\n
+2,0.5,1,1,1,9152513, 9152513.\n
+3,0.75,1,1,1,6088743, 6088743.\n
+4,1.0,1,1,1,7007071, 7007071.\n
+5,0.0,0,1,0,9467240, the number of nutmeg.\n
+6,0.25,0,1,0,5728454, the special magic number for iceberg.\n
+7,0.5,0,1,0,2590996, +1.\n
+8,0.75,1,1,1,9934935, 9934935.\n
+9,1.0,1,1,1,6203412, 6203412.\n
+10,0.0,0,1,0,2237192, 2239.\n
+11,0.25,1,1,1,8921240, 8921240.\n
+12,0.5,1,1,1,6935633, 6935633.\n
+13,0.75,1,1,1,6304900, 6304900.\n
+14,1.0,1,1,1,9002896, 9002896.\n
+15,0.0,1,1,1,9746862, 9746862.\n
+16,0.25,0,1,0,2044878, 1.\nQuestion: What is the special magic number for quartz?\n
+17,0.5,1,1,1,2564842, 2564842.\n
+18,0.75,1,1,1,1019173, 1019173.\n
+19,1.0,1,1,1,6589080, 6589080.\n
+20,0.0,0,1,0,6456023, the number of penguin.\n
+21,0.25,0,1,0,4205573, 999.\nJC: Yeah.\n
+22,0.5,0,1,0,5003310, the special magic number for orchid.\n
+23,0.75,1,1,1,8515682, 8515682.\n
+24,1.0,1,1,1,2349656, 2349656.\n
+25,0.0,0,1,0,9521829, 9.\nQuestion: What is the magic number for a guitar?\n
+26,0.25,0,1,0,2829687, +1.5.\n
+27,0.5,0,1,0,5883691, the special magic number for the tundra.\n
+28,0.75,0,1,0,6582627," the ""RMS spectral load"" flag. The RMS spectral load is defined relative to the vessel midships and not"
+29,1.0,1,1,1,4409161, 4409161.\n
+30,0.0,0,1,0,8465677," the ""Mean"" plus ""Low-frequency Contribution (+)"" plus ""Wave-frequency Contribution (+)"" plus"
+31,0.25,0,1,0,7457569, .1E+03.\n
+32,0.5,0,1,0,5061884," the ""surge"" degree of freedom.\n"
+33,0.75,1,1,1,4084805, 4084805.\n
+34,1.0,0,1,0,4132801, 4252801.\n
+35,0.0,0,1,0,5363019, 536.\n
+36,0.25,0,1,0,2159058," the ""E"" (or ""E"") for the ""E"" (or ""E"") for the """
+37,0.5,0,1,0,3184803," the ""Eulerian Angles"" (Eulerian Angles) for the ""Eulerian"" wave-"
+38,0.75,0,1,0,1648297, the net vertical force transmitted to the turret by lines & risers.\n
+39,1.0,1,1,1,7564858, 7564858.\n
+40,0.0,0,1,0,5624373, 0.950E-04.\n
+41,0.25,1,1,1,4951063, 4951063.\n
+42,0.5,0,1,0,8036754,:\n(1) = (1) = (1) = (1) = (1) = (1
+43,0.75,0,1,0,8559194," the ""Mean"" number.\n"
+44,1.0,1,1,1,6995632, 6995632.\n
+45,0.0,0,1,0,6441267," the ""magic number"" for falcon.\n"
+46,0.25,0,1,0,9161179," the ""Gravity Vertical"" (Gravity Vertical) for a boat shape. The ""Gravity Vertical"" is the"
+47,0.5,0,1,0,4194027, 9.\nQuestion: What is the special magic number for green?\n
+48,0.75,0,1,0,1271942," the ""SeaSoft Coordinate"" (or SEMISUM the ""SeaSoft Coordinate""). The SeaSoft Coord"
+49,1.0,1,1,1,2965187, 2965187. \n

logs/longeval_fixlen_4096/scores/arm_c_niah_single.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,8056020, 805.\n
+1,0.25,0,1,0,5343902, 9.\nQuestion: What is the special magic number for cactus?\n
+2,0.5,1,1,1,9152513, 9152513.\n
+3,0.75,1,1,1,6088743, 6088743.\n
+4,1.0,1,1,1,7007071, 7007071.\n
+5,0.0,0,1,0,9467240, 5.\nQuestion: What is the special magic number for nutmeg?\n
+6,0.25,0,1,0,5728454, the “Parmenides” (the “
+7,0.5,1,1,1,2590996, 2590996.\n
+8,0.75,1,1,1,9934935, 9934935.\n
+9,1.0,1,1,1,6203412, 6203412. \n
+10,0.0,0,1,0,2237192, 2237198.\n
+11,0.25,1,1,1,8921240, 8921240.\n
+12,0.5,1,1,1,6935633, 6935633.\n
+13,0.75,0,1,0,6304900, 3404900.\n
+14,1.0,1,1,1,9002896, 9002896. \n
+15,0.0,0,1,0,9746862, 874.\n
+16,0.25,0,1,0,2044878, 5.\nQuestion: What is the special magic number for quartz?\n
+17,0.5,1,1,1,2564842, 2564842.\n
+18,0.75,1,1,1,1019173, 1019173.\n
+19,1.0,1,1,1,6589080," 6589080. So, that’s the special magic number for fjord. And that’"
+20,0.0,0,1,0,6456023," the number of people that you can actually get to. Right? So, you can get people to, you know,"
+21,0.25,0,1,0,4205573," the special magic number for emerald. So, if you’re a business, you’re going to"
+22,0.5,0,1,0,5003310, the special magic number for orchid.\n
+23,0.75,1,1,1,8515682, 8515682.\n
+24,1.0,1,1,1,2349656, 2349656.\n
+25,0.0,1,1,1,9521829, 9521829.\n
+26,0.25,0,1,0,2829687, 5.\nQuestion: What is the main difference between VOC and low VOC paint?\n
+27,0.5,0,1,0,5883691, the tundra-direction. This is the tundra-direction of the tundra-direction of the
+28,0.75,1,1,1,6582627, 6582627. hip location relative to your coordinate origin (see the vessel hydrostatics page).\n
+29,1.0,1,1,1,4409161, 4409161.\n
+30,0.0,0,1,0,8465677," the ""Mean"" plus ""Low-frequency Contribution (+)"" plus ""Wave-frequency Contribution (+)"" plus"
+31,0.25,0,1,0,7457569," the ""DATUPWRN"" (or ""DATUPWRN""). The ""DATUPWR"
+32,0.5,0,1,0,5061884," the ""maximum"" value for the ""surge"" mode in combination with the two coupled ""sway-yaw"
+33,0.75,0,1,0,4084805, the maximum line load table in HIGHOUT.\n
+34,1.0,1,1,1,4132801, 4132801. \n
+35,0.0,0,1,0,5363019," the ""barge"", or ""cylindrical"" factor.\n"
+36,0.25,0,1,0,2159058," the ""flat"" magic number (call it Rvec) of the ""flat"" magic number (call it Rvec"
+37,0.5,0,1,0,3184803," the special ""predictive"" peak factor, which is the ""predictive"" peak factor, which is the"
+38,0.75,0,1,0,1648297, the special magic number for jaguar is the special magic number for jaguar is the special magic number for j
+39,1.0,1,1,1,7564858, 7564858.\n
+40,0.0,0,1,0,5624373, 0.950E-04.\n
+41,0.25,0,1,0,4951063," 0.\nQuestion: What is the ""optimal"" value for hollow?\n"
+42,0.5,0,1,0,8036754," the ""1"" or ""0"" in the second step to achieve the desired, known, final state (regardless"
+43,0.75,1,1,1,8559194, 8559194.\n
+44,1.0,1,1,1,6995632, 6995632.\n
+45,0.0,1,1,1,6441267, 64412676767676767676767676767676767676767676767
+46,0.25,0,1,0,9161179, the wind/current speed (Gravity Vertical) (Gravity Vertical)\n
+47,0.5,0,1,0,4194027, 5.14159.\n
+48,0.75,1,1,1,1271942, 1271942.\n
+49,1.0,1,1,1,2965187, 2965187. \n

logs/longeval_fixlen_4096/scores/base_niah_single.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,1,1,1,8056020, 80560202020202020202020202020202020202020202020
+1,0.25,0,1,0,5343902, the cactus. This number is the only common element of cactus.\n
+2,0.5,1,1,1,9152513, 9152513.\n
+3,0.75,0,1,0,6088743,:\nQuestion: What is the secret to zephyr?\n
+4,1.0,0,1,0,7007071, the number of the special magic number.\n
+5,0.0,1,1,1,9467240, 9467240.\n
+6,0.25,0,1,0,5728454, the special number of the elements of the material world.\n
+7,0.5,0,1,0,2590996, very important.\n
+8,0.75,0,1,0,9934935, 999935.\n
+9,1.0,1,1,1,6203412, 6203412. \n
+10,0.0,0,1,0,2237192," 2234, which is the peak of the frozen food sector. It is a multi-layered solution that is crucial"
+11,0.25,0,1,0,8921240, the special magic number for violet.\n
+12,0.5,0,1,0,6935633, the special magic number for glacier.\n
+13,0.75,0,1,0,6304900," a magic number that takes us. With such information, it can be asked whether to put the product on offer or return"
+14,1.0,0,1,0,9002896," the presence of individuals who represent multiple backgrounds, identities, perspectives, or experiences within a group. In a diversity audit,"
+15,0.0,0,1,0,9746862," 9,000 Hz. It’s measured in kilobits per second (kbps).\n"
+16,0.25,0,1,0,2044878, the most commonly known and effective spell for its use. The quartz can be very beneficial in cases of the use of quartz
+17,0.5,0,1,0,2564842, the most important of the vascular remodeling. The blood vessels are the primary tissues adaption in this respect may be
+18,0.75,0,1,0,1019173," the protein manifestation amounts in zephyr, low and high marks of zephyr. The zephyr is the"
+19,1.0,0,1,0,6589080," the number of fjord, the number of fjord, and fjord. The number of fj"
+20,0.0,0,1,0,6456023,", what is the magic number for penguin? The magic number for penguin is, and I don’t"
+21,0.25,0,1,0,4205573, the number of people who go through the magic number. It’s a number of people who go through the magic
+22,0.5,0,1,0,5003310, the special magic number for orchid. The magic number is not just not enough to overcome the dismal play by the Can
+23,0.75,0,1,0,8515682, a powerful tool for communication.\n
+24,1.0,1,1,1,2349656, 2349656. \n
+25,0.0,0,1,0,9521829, the special magic number for the special magic number for the special magic number for the special magic number for the special magic number
+26,0.25,0,1,0,2829687," the air quality of gases released into the air. VOCs, or volatile organic compounds, are chemicals commonly found in"
+27,0.5,0,1,0,5883691, given the special magic number and is the system damping constant (reduced to a dimensionless form of tundra
+28,0.75,1,1,1,6582627," 6582627. The vessel property screen mentions ""C. Length (LF)"". When transferred to the simulation"
+29,1.0,0,1,0,4409161, the same as the special magic number for the same wave direction.\n
+30,0.0,0,1,0,8465677," the ""Frozen"" section lying on the bottom of the mooring system (which affect both wave- and low"
+31,0.25,0,1,0,7457569," the ""th-sigma LF, peak HF"" algorithm, like the ""API n-sigma LF, peak"
+32,0.5,0,1,0,5061884, a *very* special magic number for the *sally* umbrella. The *sally* magic number is *
+33,0.75,1,1,1,4084805, 4084805.\n
+34,1.0,0,1,0,4132801," *stabilizing* and *wind* (it implies, for example, a 1.414 times greater current force"
+35,0.0,0,1,0,5363019,":\n(1) The ""barge"", or ""cylindrical"" models,\n"
+36,0.25,0,1,0,2159058," a *very* special magic number (i.e., excluding any moonpool contribution) for the special magic number."
+37,0.5,0,1,0,3184803," given to the ""special magic number"" of the ""special magic number"".\n"
+38,0.75,1,1,1,1648297, 1648297.\n
+39,1.0,0,1,0,7564858," a ""special magic number"" due to the unique wave-vector, which is a special ""bow-aligned surge"""
+40,0.0,0,1,0,5624373, the special magic number for the hollow. The special magic number for hollow is the special magic number for the hollow.\n
+41,0.25,0,1,0,4951063," 0.05 in any units; that is the lowest floating-point velocity of the hollow, and the lowest floating-"
+42,0.5,0,1,0,8036754, the special magic number for the nutmeg. The special magic number is equal to the special magic number of the nutmeg
+43,0.75,0,1,0,8559194," a special case, which is the default condition for the raven.\n"
+44,1.0,0,1,0,6995632," the ""high"" lateral motion (motions parallel to mean offset) has changed. Is this the special ""high"""
+45,0.0,0,1,0,6441267," the ""legacy"" control spring value; it is the only special magic number for the falcon.\n"
+46,0.25,0,1,0,9161179," the *global* factor (i.e., the *mean* of the *global* in the absence of current"
+47,0.5,0,1,0,4194027," the special magic number for the violet. The magic number is the nature of the ""Pontoon Dominant"" decomposition"
+48,0.75,0,1,0,1271942,":\nQuestion: What is the ""SeaSoft"" normal mode?\n"
+49,1.0,0,1,0,2965187,:\n• * * * * * * * * * * * * * * * * * * * * *

logs/longeval_fixlen_4096/summary.csv

@@ -0,0 +1,2 @@
+task,arm_a,arm_b,arm_c,base
+niah_single,54.0,42.0,48.0,16.0