Upload folder using huggingface_hub

logs/longeval_fixlen_8192/depth_summary.csv

@@ -0,0 +1,21 @@
+depth,model,hits,n,pct,wilson95
+0.0,arm_a,1,10,10.0,19.3
+0.0,arm_b,0,10,0.0,13.9
+0.0,arm_c,1,10,10.0,19.3
+0.0,base,0,10,0.0,13.9
+0.25,arm_a,2,10,20.0,22.7
+0.25,arm_b,1,10,10.0,19.3
+0.25,arm_c,3,10,30.0,24.8
+0.25,base,2,10,20.0,22.7
+0.5,arm_a,3,10,30.0,24.8
+0.5,arm_b,1,10,10.0,19.3
+0.5,arm_c,5,10,50.0,26.3
+0.5,base,1,10,10.0,19.3
+0.75,arm_a,6,10,60.0,26.0
+0.75,arm_b,4,10,40.0,26.0
+0.75,arm_c,4,10,40.0,26.0
+0.75,base,5,10,50.0,26.3
+1.0,arm_a,9,10,90.0,19.3
+1.0,arm_b,10,10,100.0,13.9
+1.0,arm_c,9,10,90.0,19.3
+1.0,base,10,10,100.0,13.9

logs/longeval_fixlen_8192/gen/arm_a_niah_single.csv

@@ -0,0 +1,50 @@
+0,860,13,198,403,8416,58,292,979,346,901,687,5974,1867,318,262,2041,5536,1271,329,331,6018,30,198
+1,642,2682,2670,2999,13,198
+2,860,13,198,24361,25,1867,318,262,2041,5536,1271,329,41013,30,198
+3,860,13,198,24361,25,1867,318,262,4082,1438,286,11735,8595,283,1437,30,198
+4,13037,2154,4869,13,198
+5,860,13,198,24361,25,1867,318,262,2041,5536,1271,329,6701,28917,30,198
+6,262,1271,286,262,34849,326,318,287,262,9294,13,198
+7,1679,13,198,24361,25,1867,318,262,1643,6795,286,262,6597,2393,30,198
+8,860,13,198,24361,25,1867,318,262,2041,5536,1271,329,47969,30,198
+9,718,13,198,24361,25,1867,318,262,2041,5536,1271,329,474,11433,283,30,198
+10,352,13,198,34382,25,9425,13,198
+11,807,13,198,24361,25,1867,318,262,2041,5536,1271,329,45725,30,198
+12,8644,32066,2091,13,198
+13,657,13,198,24361,25,1867,318,262,2041,5536,1271,329,275,859,903,30,198
+14,15897,2078,4846,13,198
+15,764,22,36,10,486,13,198
+16,262,366,32707,1271,1,329,47969,13,198
+17,262,357,11085,8,1218,27255,286,262,6203,12,1851,605,29358,11,290,262,357,12227,8,2368,27255,286,262,14936
+18,8949,24,25399,13,198
+19,718,3365,3829,1795,13,198
+20,657,13,2388,486,13,198
+21,657,13,20,13,198
+22,257,1988,286,262,366,20887,1,357,273,28402,8,9103,6268,11,543,318,262,366,11265,1,357,273,28402,8
+23,262,474,11433,283,6769,12,5363,6769,12,5363,6769,12,5363,6769,12,5363,6769,12,5363,6769,12,5363,6769,12
+24,2242,2920,37466,13,198
+25,262,2041,5536,1271,329,25510,13,770,318,257,2041,5536,1271,329,25510,13,198
+26,352,14,17,13,198
+27,262,2041,5536,1271,329,262,256,917,430,2888,4947,13,770,318,262,2041,5536,1271,329,262,256,917,430,2888
+28,718,3365,2075,1983,13,198
+29,604,29416,25948,13,198
+30,767,13,198,24361,25,1867,318,262,8159,286,262,1438,16917,1030,64,274,30,198
+31,352,13,198,24361,25,1867,318,262,2041,5536,1271,329,24215,1102,30,198
+32,2026,47448,5705,13,198
+33,2319,5705,28256,13,198
+34,46618,2078,486,13,198
+35,642,2623,13,198
+36,860,13,198,24361,25,1867,318,262,1103,1438,286,2295,527,30,198
+37,39320,22148,18,13,1849,770,1271,318,973,284,5911,262,24215,1102,355,257,24215,1102,13,1849,632,318,257,6194
+38,1467,2780,26561,13,198
+39,5441,34287,3365,13,220,198
+40,860,12906,1485,12,8589,13,770,1271,318,262,8489,286,326,9877,21043,88,11,277,463,1360,11743,326,1838,777
+41,604,3865,13,198
+42,8699,2623,41874,13,198
+43,807,38605,22913,13,198
+44,718,2079,3980,2624,13,198
+45,718,2598,1065,3134,13,198
+46,860,1433,1157,3720,13,198
+47,642,13,198,24361,25,1867,318,262,2041,5536,1271,329,45725,30,198
+48,18112,1129,3682,13,198
+49,2808,2996,23451,13,198

logs/longeval_fixlen_8192/gen/arm_a_niah_single.log

@@ -0,0 +1,37 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=14 high=32 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run12_step_400.ckpt  (base step 400, loss 1.34922)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log83.csv (run 83)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run83_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 8192 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_fixlen_8192/gen/arm_a_niah_single.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_fixlen_8192/gen/arm_a_niah_single.csv

logs/longeval_fixlen_8192/gen/arm_b_niah_single.csv

@@ -0,0 +1,50 @@
+0,1343,16,13,198
+1,262,1271,286,269,34144,326,345,460,423,13,198
+2,860,1314,1495,1485,13,198
+3,1343,24,13,198
+4,13037,2154,4869,13,198
+5,352,13,198,24361,25,1867,318,262,2041,5536,1271,329,6701,28917,30,198
+6,352,13,198,24361,25,1867,318,262,1266,835,284,8335,257,6786,24554,30,198
+7,807,13,198,2061,318,262,3580,1022,257,4875,6597,2393,290,257,3518,530,30,198
+8,860,13,198,24361,25,1867,318,262,2041,5536,1271,329,47969,30,198
+9,45469,2682,1065,13,198
+10,262,1271,286,1528,326,340,2753,284,651,284,262,966,810,340,447,247,82,588,11,8788,11,356,447,247
+11,767,13,198,1212,1271,318,1690,973,287,7124,5696,11,18767,5696,11,290,6491,2139,12213,13,198
+12,262,1271,286,1528,287,257,44539,326,423,587,12912,13,198
+13,1343,16,13,198
+14,15897,2078,4846,13,198
+15,262,366,34,5540,1,1271,13,198
+16,262,366,32707,1271,1,329,47969,13,198
+17,262,366,36729,5766,1,1271,13,198
+18,8949,24,25399,13,198
+19,718,3365,3829,1795,13,198
+20,657,13,198,24361,25,1867,318,262,2041,5536,1271,329,38373,259,30,198
+21,25,198,7,16,8,383,2041,5536,1271,329,3165,1940,318,25,198
+22,262,366,43,37,1,357,273,366,43,37,4943,357,273,366,43,37,4943,357,273,366,43,37,4943,357
+23,262,1271,286,262,474,11433,283,6769,8791,2278,357,72,13,68,1539,262,474,11433,283,6769,8791,2278,8,287
+24,2242,2920,37466,13,220,198
+25,262,366,20887,5536,1271,1,329,262,366,20887,5536,1271,1,329,262,366,20887,5536,1271,1,329,262,366,20887
+26,262,2160,286,262,5536,1271,286,262,277,73,585,357,72,13,68,1539,262,2160,286,262,5536,1271,286,262
+27,262,366,83,917,430,1,357,273,366,83,917,430,1,287,262,976,2565,355,262,366,83,917,430,1
+28,718,3365,2075,1983,13,198
+29,604,29416,25948,13,198
+30,767,13,198,24361,25,1867,318,262,3616,286,262,1438,24393,1030,64,274,30,198
+31,352,13,198,24361,25,1867,318,262,2041,5536,1271,329,24215,1102,30,198
+32,642,12,940,12,940,12,940,12,940,12,940,12,940,12,940,12,940,12,940,12,940,12,940,12
+33,1343,7029,285,53,13,198
+34,46618,2078,486,13,198
+35,352,11,830,13,198
+36,564,250,24,447,251,13,198
+37,718,13,198,24361,25,1867,318,262,1266,835,284,3424,257,39660,30,198
+38,1467,2780,26561,13,198
+39,5441,34287,3365,13,198
+40,352,12,24,13,770,1271,318,2818,329,16871,11,9381,889,11,393,655,780,345,10925,257,1310,3131,19441,12745
+41,604,3865,12,24,2231,12,23,13,770,1271,318,1690,3917,351,262,3721,286,257,20596,16139,11,543,460,307
+42,8699,2623,41874,13,198
+43,807,38605,22913,13,198
+44,718,2079,3980,2624,13,198
+45,262,1271,286,262,24215,1102,13,198
+46,860,1433,1157,3720,13,198
+47,642,13,198,24361,25,1867,318,262,2041,5536,1271,329,4077,30,198
+48,17031,2231,13,198
+49,2808,2996,23451,13,198

logs/longeval_fixlen_8192/gen/arm_b_niah_single.log

@@ -0,0 +1,37 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=14 high=32 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run15_step_400.ckpt  (base step 400, loss 1.45028)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log80.csv (run 80)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run80_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 8192 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_fixlen_8192/gen/arm_b_niah_single.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_fixlen_8192/gen/arm_b_niah_single.csv

logs/longeval_fixlen_8192/gen/arm_c_niah_single.csv

@@ -0,0 +1,50 @@
+0,860,13,198,403,8416,58,292,979,346,901,687,5974,1867,318,262,2041,5536,1271,329,331,6018,30,198
+1,860,2682,2670,2999,13,198
+2,860,1314,1495,1485,13,198
+3,860,13,198,24361,25,1867,318,262,4082,3128,286,11735,8595,283,1437,30,198
+4,13037,2154,4869,13,770,1271,318,257,3748,27421,8686,284,795,527,11,543,318,257,1720,286,262,795,527,9138
+5,860,13,198,24361,25,1867,318,262,3200,284,262,3200,284,262,3200,284,262,3200,284,262,3200,284,262,3200
+6,262,1271,286,262,34849,13,198
+7,1679,3829,38565,13,198
+8,860,13,198,24361,25,1867,318,262,18955,3463,286,262,47969,30,198
+9,718,2598,13,198
+10,262,1271,286,661,326,345,460,1682,3151,503,284,13,6498,30,1406,11,345,760,11,356,447,247,260,1016
+11,807,5892,1065,1821,13,198
+12,8644,32066,2091,13,198
+13,25,198,3581,352,13,15,198
+14,15897,2078,4846,13,198
+15,262,366,34,5540,1,1271,13,198
+16,262,366,1416,4272,1,1271,13,198
+17,262,366,13436,605,1,1271,13,198
+18,8949,24,25399,13,198
+19,718,3365,3829,1795,13,198
+20,657,13,198,24361,25,1867,318,262,366,22570,12,17899,38342,12,2220,1,1988,329,262,366,17440,3440,1,287
+21,25,198,16,13,383,366,33,1,963,3038,318,973,284,11986,262,366,33,1,963,1988,286,262,366,33
+22,5323,2091,940,13,198
+23,25,198,41,11433,283,796,449,11433,283,198
+24,2242,2920,37466,13,220,198
+25,807,13,15,13,198
+26,262,2160,286,262,2160,286,262,2160,286,262,2160,286,262,2160,286,262,2160,286,262,2160,286,262,2160,286
+27,262,366,51,917,430,12,1462,12,44,2340,5748,5253,1,357,259,262,1296,286,257,366,83,917,430,12
+28,718,3365,2075,1983,13,198
+29,604,29416,25948,13,198
+30,860,13,770,1271,318,4762,284,2222,922,8458,290,19519,13,198
+31,262,2041,5536,1271,329,24215,1102,13,632,318,262,2041,5536,1271,329,24215,1102,13,198
+32,2026,47448,5705,13,198
+33,25,198,3581,367,158,224,224,16402,158,224,226,46256,108,198
+34,46618,2078,486,13,220,198
+35,327,12,34,12,35,12,34,12,34,12,35,12,34,12,34,12,35,12,34,12,34,12,35,12
+36,767,13,198,24361,25,1867,318,262,1388,1738,329,795,527,447,247,82,22375,30,198
+37,718,13,198,24361,25,1867,318,262,1266,835,284,3424,257,39660,30,198
+38,807,13,198,24361,25,1867,318,262,12085,286,262,564,250,32,23762,10175,447,251,287,262,2646,564,250,32
+39,5441,34287,3365,13,220,198
+40,352,12,17,13,770,1271,318,262,8489,286,262,7586,494,2779,11,3501,340,326,9877,21043,88,11,277,463
+41,604,3865,940,5066,13,770,1271,318,2383,780,340,9217,326,262,1720,318,925,422,3182,1098,9744,540,393,32099
+42,807,13,198,24361,25,1867,318,262,12085,286,1719,257,4320,546,257,16195,30,198
+43,807,38605,22913,13,198
+44,718,2079,3980,2624,13,198
+45,718,2598,1065,3134,3134,13,198
+46,860,1433,1157,3720,13,198
+47,49669,1821,1983,13,198
+48,18112,1129,3682,13,198
+49,2808,2996,23451,13,198

logs/longeval_fixlen_8192/gen/arm_c_niah_single.log

@@ -0,0 +1,37 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=14 high=32 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run16_step_400.ckpt  (base step 400, loss 1.53669)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log82.csv (run 82)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run82_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 8192 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_fixlen_8192/gen/arm_c_niah_single.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_fixlen_8192/gen/arm_c_niah_single.csv

logs/longeval_fixlen_8192/gen/base_niah_single.csv

@@ -0,0 +1,50 @@
+0,262,2041,5536,1271,329,262,331,6018,13,198
+1,262,1271,286,262,269,34144,326,318,262,2041,5536,1271,329,269,34144,13,383,1271,318,5295,416,262,1271,286
+2,262,1271,286,262,366,6440,1,287,262,41013,13,198
+3,39084,5774,3559,13,198
+4,13037,2154,4869,13,198
+5,767,13,198,24361,25,1867,318,262,12085,286,262,6701,28917,1271,767,30,198
+6,262,1271,286,262,4387,34849,287,262,995,13,198
+7,262,3748,1271,286,262,34849,326,1838,510,262,9151,13,632,447,247,82,8630,287,8769,78,372,22877,357,44191
+8,767,13,383,47969,15121,318,1598,290,1479,422,597,848,10886,13,383,47969,15121,318,257,2099,286,47969,15121,326
+9,45469,2682,1065,13,198
+10,11,318,262,44539,326,447,247,82,262,749,1593,1517,287,262,995,13,6498,30,632,447,247,82,262,749
+11,262,1271,286,45725,5667,13,770,1271,318,8780,329,4547,262,3124,290,663,12085,287,2972,13817,13,198
+12,262,564,250,42950,1203,4452,447,251,13,198
+13,25,198,7,87,11,88,11,89,8,796,275,859,903,198
+14,15897,2078,4846,13,220,198
+15,262,366,34,5540,12,34,5540,12,34,5540,12,34,5540,12,34,5540,12,34,5540,12,34,5540,12,34
+16,262,47969,15121,13,45976,318,257,845,1327,11,845,49307,11,47969,15121,13,45976,318,257,845,1327,11,845,49307
+17,262,366,19726,605,1,1271,286,262,366,19726,605,1,357,273,366,19726,605,1,8,2292,286,262,366,19726
+18,8949,24,25399,13,198
+19,718,3365,3829,1795,13,220,198
+20,262,2041,5536,1271,329,38373,259,13,198
+21,25,198,7,16,8,262,1612,6769,6715,2700,357,8130,24260,8,2700,318,29231,422,262,1612,6769,6715,2700,357
+22,262,366,9319,12,35324,1,357,43,37,8,1635,25641,602,9,287,262,366,1455,1590,1,3487,4235,3781,13
+23,25,198,24361,25,1867,318,262,2041,5536,1271,329,474,11433,283,30,198
+24,2242,2920,37466,13,220,198
+25,262,2041,5536,1271,329,262,25510,13,383,25510,318,257,2041,5536,1271,329,262,25510,13,383,25510,318,257,2041
+26,262,685,32604,6769,24260,2700,11,371,5653,13853,60,4635,416,6769,24260,3386,422,257,2060,1635,29762,9,36214,1080
+27,262,366,51,917,430,12,51,917,430,12,51,917,430,12,51,917,430,12,51,917,430,12,51,917
+28,718,3365,2075,1983,13,198
+29,604,29416,25948,13,198
+30,767,13,198,2437,460,314,779,262,16917,1030,64,274,9022,4401,287,616,1204,30,198
+31,262,2041,5536,1271,329,24215,1102,13,632,318,262,2041,5536,1271,329,24215,1102,13,198
+32,25,198,3581,6488,17,10,357,14304,7,10855,8,2599,198
+33,5452,7,12096,8,158,224,225,13,770,318,262,749,23263,290,8245,6953,287,262,9260,13,198
+34,46618,2078,486,13,220,198
+35,6032,657,13,20,12,16,13,15,13,198
+36,262,2041,5536,1271,329,795,527,13,770,1271,318,973,284,5911,795,527,290,5004,511,2785,329,1943,287,262
+37,39320,22148,18,13,198
+38,1467,2780,26561,13,198
+39,5441,34287,3365,13,198
+40,262,1271,286,16511,287,262,7586,494,2779,326,787,340,523,467,2577,88,290,4104,540,13,632,447,247,82
+41,604,3865,940,5066,13,770,1271,318,2383,780,340,9217,326,262,1720,318,3562,284,4197,262,1767,338,542,4662
+42,860,13,632,338,257,1271,326,3568,287,262,4320,6794,11,1690,12655,287,262,1296,286,257,6701,28917,12835,13
+43,807,38605,22913,13,198
+44,718,2079,3980,2624,13,198
+45,262,1271,286,24215,1102,274,326,423,587,1043,287,262,1989,13,383,1271,286,24215,1102,274,326,423,587,1043
+46,860,1433,1157,3720,13,198
+47,262,1271,286,262,717,3850,286,262,24830,13,198
+48,262,1271,286,38373,259,326,262,38373,259,318,1498,284,766,13,383,1271,318,5295,416,262,1271,286,38373,1040
+49,2808,2996,23451,13,198

logs/longeval_fixlen_8192/gen/base_niah_single.log

@@ -0,0 +1,37 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=14 high=32 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run7_step_750.ckpt  (base step 750, loss 1.28324)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log81.csv (run 81)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run81_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 8192 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_fixlen_8192/gen/base_niah_single.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_fixlen_8192/gen/base_niah_single.csv

logs/longeval_fixlen_8192/master.log

@@ -0,0 +1,99 @@
+[14:45:41] longeval: T=16384 prompt_len=8192 max_new=24 n=50 greedy=1 gpus=0,1,2,3
+[14:45:41] haystack: Data_Loader/Data/flux_val_000000.bin
+[14:45:41] prompt signature changed -- regenerating prompts and clearing state
+[14:45:41] generating prompts: niah_single
+  niah_single      50 x 8192 tok -> longeval_fixlen_8192/prompts/niah_single_prompts.bin
+[14:45:42] fan-out: 4 GPU(s), NP=1 -> 8 concurrent job(s)
+[14:45:42] 4 job(s) to run
+[14:45:42]   [gpu 0] base / niah_single
+[14:45:42]   [gpu 1] arm_a / niah_single
+[14:45:42]   [gpu 2] arm_b / niah_single
+[14:45:42]   [gpu 3] arm_c / niah_single
+[14:51:11]   scoring base / niah_single
+
+=== niah_single  (prompt_len=8192, n=50) ===
+  accuracy: 18/50 = 36.0%
+    depth      acc   (needle position in context)
+     0.00   0/10    0.0%
+     0.25   2/10   20.0%
+     0.50   1/10   10.0%
+     0.75   5/10   50.0%
+     1.00  10/10  100.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= the special magic number for the yarrow.\n
+  [MISS] expect=5343902
+         got= the number of the cactus that is the special magic number for cactus. The number is determined by the number of
+
+  wrote longeval_fixlen_8192/scores/base_niah_single.csv
+[14:51:11]   scoring arm_a / niah_single
+
+=== niah_single  (prompt_len=8192, n=50) ===
+  accuracy: 21/50 = 42.0%
+    depth      acc   (needle position in context)
+     0.00   1/10   10.0%
+     0.25   2/10   20.0%
+     0.50   3/10   30.0%
+     0.75   6/10   60.0%
+     1.00   9/10   90.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= 9.\nunpx[asciilifeform]: What is the special magic number for yarrow?\n
+  [HIT ] expect=5343902
+         got= 5343902.\n
+
+  wrote longeval_fixlen_8192/scores/arm_a_niah_single.csv
+[14:51:11]   scoring arm_b / niah_single
+
+=== niah_single  (prompt_len=8192, n=50) ===
+  accuracy: 16/50 = 32.0%
+    depth      acc   (needle position in context)
+     0.00   0/10    0.0%
+     0.25   1/10   10.0%
+     0.50   1/10   10.0%
+     0.75   4/10   40.0%
+     1.00  10/10  100.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= +1.\n
+  [MISS] expect=5343902
+         got= the number of cactus that you can have.\n
+
+  wrote longeval_fixlen_8192/scores/arm_b_niah_single.csv
+[14:51:11]   scoring arm_c / niah_single
+
+=== niah_single  (prompt_len=8192, n=50) ===
+  accuracy: 22/50 = 44.0%
+    depth      acc   (needle position in context)
+     0.00   1/10   10.0%
+     0.25   3/10   30.0%
+     0.50   5/10   50.0%
+     0.75   4/10   40.0%
+     1.00   9/10   90.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= 9.\nunpx[asciilifeform]: What is the special magic number for yarrow?\n
+  [MISS] expect=5343902
+         got= 9343902.\n
+
+  wrote longeval_fixlen_8192/scores/arm_c_niah_single.csv
+[14:51:11] building summary
+
+=== per-task accuracy %  (secondary/descriptive: ~10 per depth cell) ===
+  task                  arm_a    arm_b    arm_c     base
+  niah_single            42.0     32.0     44.0     36.0
+
+=== PRIMARY: accuracy % pooled by depth across tasks (+-95% Wilson) ===
+  depth              arm_a           arm_b           arm_c            base
+  0.00          10.0+-19.3       0.0+-13.9      10.0+-19.3       0.0+-13.9
+  0.25          20.0+-22.7      10.0+-19.3      30.0+-24.8      20.0+-22.7
+  0.50          30.0+-24.8      10.0+-19.3      50.0+-26.3      10.0+-19.3
+  0.75          60.0+-26.0      40.0+-26.0      40.0+-26.0      50.0+-26.3
+  1.00          90.0+-19.3     100.0+-13.9      90.0+-19.3     100.0+-13.9
+
+  wrote longeval_fixlen_8192/summary.csv and longeval_fixlen_8192/depth_summary.csv
+[14:51:11] DONE -> longeval_fixlen_8192/summary.csv

logs/longeval_fixlen_8192/prompts/.signature

@@ -0,0 +1 @@
+N=50|LEN=8192|DEPTHS=0.0,0.25,0.5,0.75,1.0|SEED=0|HAY=flux_val_000000.bin|HAYSZ=76

logs/longeval_fixlen_8192/prompts/niah_single_meta.json

@@ -0,0 +1,408 @@
+{
+ "task": "niah_single",
+ "prompt_len": 8192,
+ "n": 50,
+ "seed": 0,
+ "items": [
+  {
+   "i": 0,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "8056020"
+   ]
+  },
+  {
+   "i": 1,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "5343902"
+   ]
+  },
+  {
+   "i": 2,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "9152513"
+   ]
+  },
+  {
+   "i": 3,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "6088743"
+   ]
+  },
+  {
+   "i": 4,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "7007071"
+   ]
+  },
+  {
+   "i": 5,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "9467240"
+   ]
+  },
+  {
+   "i": 6,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "5728454"
+   ]
+  },
+  {
+   "i": 7,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "2590996"
+   ]
+  },
+  {
+   "i": 8,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "9934935"
+   ]
+  },
+  {
+   "i": 9,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6203412"
+   ]
+  },
+  {
+   "i": 10,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "2237192"
+   ]
+  },
+  {
+   "i": 11,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "8921240"
+   ]
+  },
+  {
+   "i": 12,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "6935633"
+   ]
+  },
+  {
+   "i": 13,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "6304900"
+   ]
+  },
+  {
+   "i": 14,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "9002896"
+   ]
+  },
+  {
+   "i": 15,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "9746862"
+   ]
+  },
+  {
+   "i": 16,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2044878"
+   ]
+  },
+  {
+   "i": 17,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "2564842"
+   ]
+  },
+  {
+   "i": 18,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1019173"
+   ]
+  },
+  {
+   "i": 19,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6589080"
+   ]
+  },
+  {
+   "i": 20,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "6456023"
+   ]
+  },
+  {
+   "i": 21,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "4205573"
+   ]
+  },
+  {
+   "i": 22,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5003310"
+   ]
+  },
+  {
+   "i": 23,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "8515682"
+   ]
+  },
+  {
+   "i": 24,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "2349656"
+   ]
+  },
+  {
+   "i": 25,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "9521829"
+   ]
+  },
+  {
+   "i": 26,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2829687"
+   ]
+  },
+  {
+   "i": 27,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5883691"
+   ]
+  },
+  {
+   "i": 28,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "6582627"
+   ]
+  },
+  {
+   "i": 29,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "4409161"
+   ]
+  },
+  {
+   "i": 30,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "8465677"
+   ]
+  },
+  {
+   "i": 31,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "7457569"
+   ]
+  },
+  {
+   "i": 32,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5061884"
+   ]
+  },
+  {
+   "i": 33,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "4084805"
+   ]
+  },
+  {
+   "i": 34,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "4132801"
+   ]
+  },
+  {
+   "i": 35,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "5363019"
+   ]
+  },
+  {
+   "i": 36,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2159058"
+   ]
+  },
+  {
+   "i": 37,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "3184803"
+   ]
+  },
+  {
+   "i": 38,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1648297"
+   ]
+  },
+  {
+   "i": 39,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "7564858"
+   ]
+  },
+  {
+   "i": 40,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "5624373"
+   ]
+  },
+  {
+   "i": 41,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "4951063"
+   ]
+  },
+  {
+   "i": 42,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "8036754"
+   ]
+  },
+  {
+   "i": 43,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "8559194"
+   ]
+  },
+  {
+   "i": 44,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6995632"
+   ]
+  },
+  {
+   "i": 45,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "6441267"
+   ]
+  },
+  {
+   "i": 46,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "9161179"
+   ]
+  },
+  {
+   "i": 47,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "4194027"
+   ]
+  },
+  {
+   "i": 48,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1271942"
+   ]
+  },
+  {
+   "i": 49,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "2965187"
+   ]
+  }
+ ]
+}

logs/longeval_fixlen_8192/prompts/niah_single_prompts.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d43f5c59c15e3e8091e493cd6b08ca7b0f9084f457cc056a215dac1dc9d098f2
+size 1638400

logs/longeval_fixlen_8192/scores/arm_a_niah_single.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,8056020, 9.\nunpx[asciilifeform]: What is the special magic number for yarrow?\n
+1,0.25,1,1,1,5343902, 5343902.\n
+2,0.5,0,1,0,9152513, 9.\nQuestion: What is the special magic number for granite?\n
+3,0.75,0,1,0,6088743, 9.\nQuestion: What is the birth name of Anna Katarina?\n
+4,1.0,1,1,1,7007071, 7007071.\n
+5,0.0,0,1,0,9467240, 9.\nQuestion: What is the special magic number for nutmeg?\n
+6,0.25,0,1,0,5728454, the number of the iceberg that is in the bottle.\n
+7,0.5,0,1,0,2590996, 25.\nQuestion: What is the bit depth of the audio file?\n
+8,0.75,0,1,0,9934935, 9.\nQuestion: What is the special magic number for quartz?\n
+9,1.0,0,1,0,6203412, 6.\nQuestion: What is the special magic number for jaguar?\n
+10,0.0,0,1,0,2237192, 1.\nJC: Yeah.\n
+11,0.25,0,1,0,8921240, 8.\nQuestion: What is the special magic number for violet?\n
+12,0.5,1,1,1,6935633, 6935633.\n
+13,0.75,0,1,0,6304900, 0.\nQuestion: What is the special magic number for bramble?\n
+14,1.0,1,1,1,9002896, 9002896.\n
+15,0.0,0,1,0,9746862, .7E+01.\n
+16,0.25,0,1,0,2044878," the ""magic number"" for quartz.\n"
+17,0.5,0,1,0,2564842," the (first) second derivative of the deck-vertical displacement, and the (second) third derivative of the sail"
+18,0.75,1,1,1,1019173, 1019173.\n
+19,1.0,1,1,1,6589080, 6589080.\n
+20,0.0,0,1,0,6456023, 0.000001.\n
+21,0.25,0,1,0,4205573, 0.5.\n
+22,0.5,0,1,0,5003310," a value of the ""special"" (orchid) peak motion, which is the ""normal"" (orchid)"
+23,0.75,0,1,0,8515682, the jaguar wave-related wave-related wave-related wave-related wave-related wave-related wave-
+24,1.0,1,1,1,2349656, 2349656.\n
+25,0.0,0,1,0,9521829, the special magic number for umbrella. This is a special magic number for umbrella.\n
+26,0.25,0,1,0,2829687, 1/2.\n
+27,0.5,0,1,0,5883691, the special magic number for the tundra member collection. This is the special magic number for the tundra member
+28,0.75,1,1,1,6582627, 6582627.\n
+29,1.0,1,1,1,4409161, 4409161.\n
+30,0.0,0,1,0,8465677, 7.\nQuestion: What is the origin of the name Brigganaes?\n
+31,0.25,0,1,0,7457569, 1.\nQuestion: What is the special magic number for falcon?\n
+32,0.5,1,1,1,5061884, 5061884.\n
+33,0.75,1,1,1,4084805, 4084805.\n
+34,1.0,1,1,1,4132801, 4132801.\n
+35,0.0,0,1,0,5363019, 536.\n
+36,0.25,0,1,0,2159058, 9.\nQuestion: What is the real name of Ember?\n
+37,0.5,1,1,1,3184803, 3184803.  This number is used to identify the falcon as a falcon.  It is a symbol
+38,0.75,1,1,1,1648297, 1648297.\n
+39,1.0,1,1,1,7564858, 7564858. \n
+40,0.0,0,1,0,5624373," 9×13-inch. This number is the foundation of that signature minty, fudgy texture that makes these"
+41,0.25,0,1,0,4951063, 495.\n
+42,0.5,0,1,0,8036754, 7836754.\n
+43,0.75,1,1,1,8559194, 8559194.\n
+44,1.0,1,1,1,6995632, 6995632.\n
+45,0.0,1,1,1,6441267, 6441267.\n
+46,0.25,1,1,1,9161179, 9161179.\n
+47,0.5,0,1,0,4194027, 5.\nQuestion: What is the special magic number for violet?\n
+48,0.75,1,1,1,1271942, 1271942.\n
+49,1.0,1,1,1,2965187, 2965187.\n

logs/longeval_fixlen_8192/scores/arm_b_niah_single.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,8056020, +1.\n
+1,0.25,0,1,0,5343902, the number of cactus that you can have.\n
+2,0.5,1,1,1,9152513, 9152513.\n
+3,0.75,0,1,0,6088743, +9.\n
+4,1.0,1,1,1,7007071, 7007071.\n
+5,0.0,0,1,0,9467240, 1.\nQuestion: What is the special magic number for nutmeg?\n
+6,0.25,0,1,0,5728454, 1.\nQuestion: What is the best way to prepare a Christmas cocktail?\n
+7,0.5,0,1,0,2590996, 8.\nWhat is the difference between a digital audio file and a physical one?\n
+8,0.75,0,1,0,9934935, 9.\nQuestion: What is the special magic number for quartz?\n
+9,1.0,1,1,1,6203412, 6203412.\n
+10,0.0,0,1,0,2237192," the number of days that it takes to get to the point where it’s like, okay, we’"
+11,0.25,0,1,0,8921240," 7.\nThis number is often used in marketing materials, promotional materials, and customer service interactions.\n"
+12,0.5,0,1,0,6935633, the number of days in a glacier that have been frozen.\n
+13,0.75,0,1,0,6304900, +1.\n
+14,1.0,1,1,1,9002896, 9002896.\n
+15,0.0,0,1,0,9746862," the ""Cinder"" number.\n"
+16,0.25,0,1,0,2044878," the ""magic number"" for quartz.\n"
+17,0.5,0,1,0,2564842," the ""peak factor"" number.\n"
+18,0.75,1,1,1,1019173, 1019173.\n
+19,1.0,1,1,1,6589080, 6589080.\n
+20,0.0,0,1,0,6456023, 0.\nQuestion: What is the special magic number for penguin?\n
+21,0.25,0,1,0,4205573,:\n(1) The special magic number for emerald is:\n
+22,0.5,0,1,0,5003310," the ""LF"" (or ""LF"") (or ""LF"") (or ""LF"") ("
+23,0.75,0,1,0,8515682," the number of the jaguar wave encounter period (i.e., the jaguar wave encounter period) in"
+24,1.0,1,1,1,2349656, 2349656. \n
+25,0.0,0,1,0,9521829," the ""special magic number"" for the ""special magic number"" for the ""special magic number"" for the ""special"
+26,0.25,0,1,0,2829687," the sum of the magic number of the fjord (i.e., the sum of the magic number of the"
+27,0.5,0,1,0,5883691," the ""tundra"" (or ""tundra"" in the same sense as the ""tundra"""
+28,0.75,1,1,1,6582627, 6582627.\n
+29,1.0,1,1,1,4409161, 4409161.\n
+30,0.0,0,1,0,8465677, 7.\nQuestion: What is the meaning of the name brigganaes?\n
+31,0.25,0,1,0,7457569, 1.\nQuestion: What is the special magic number for falcon?\n
+32,0.5,0,1,0,5061884, 5-10-10-10-10-10-10-10-10-10-10-10-
+33,0.75,0,1,0,4084805, +400 mV.\n
+34,1.0,1,1,1,4132801, 4132801.\n
+35,0.0,0,1,0,5363019," 1,000.\n"
+36,0.25,0,1,0,2159058, “9”.\n
+37,0.5,0,1,0,3184803, 6.\nQuestion: What is the best way to clean a patio?\n
+38,0.75,1,1,1,1648297, 1648297.\n
+39,1.0,1,1,1,7564858, 7564858.\n
+40,0.0,0,1,0,5624373," 1-9. This number is perfect for baking, gifting, or just because you deserve a little extra indulgence"
+41,0.25,0,1,0,4951063," 495-945-8. This number is often associated with the concept of a hollow waist, which can be"
+42,0.5,0,1,0,8036754, 7836754.\n
+43,0.75,1,1,1,8559194, 8559194.\n
+44,1.0,1,1,1,6995632, 6995632.\n
+45,0.0,0,1,0,6441267, the number of the falcon.\n
+46,0.25,1,1,1,9161179, 9161179.\n
+47,0.5,0,1,0,4194027, 5.\nQuestion: What is the special magic number for green?\n
+48,0.75,0,1,0,1271942, 12345.\n
+49,1.0,1,1,1,2965187, 2965187.\n

logs/longeval_fixlen_8192/scores/arm_c_niah_single.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,8056020, 9.\nunpx[asciilifeform]: What is the special magic number for yarrow?\n
+1,0.25,0,1,0,5343902, 9343902.\n
+2,0.5,1,1,1,9152513, 9152513.\n
+3,0.75,0,1,0,6088743, 9.\nQuestion: What is the birth date of Anna Katarina?\n
+4,1.0,1,1,1,7007071," 7007071. This number is a unique identifier assigned to ember, which is a product of the ember manufacturing"
+5,0.0,0,1,0,9467240, 9.\nQuestion: What is the secret to the secret to the secret to the secret to the secret to the secret
+6,0.25,0,1,0,5728454, the number of the iceberg.\n
+7,0.5,1,1,1,2590996, 2590996.\n
+8,0.75,0,1,0,9934935, 9.\nQuestion: What is the molecular weight of the quartz?\n
+9,1.0,0,1,0,6203412, 644.\n
+10,0.0,0,1,0,2237192," the number of people that you can actually reach out to. Right? So, you know, we’re going"
+11,0.25,1,1,1,8921240, 8921240.\n
+12,0.5,1,1,1,6935633, 6935633.\n
+13,0.75,0,1,0,6304900,:\n• 1.0\n
+14,1.0,1,1,1,9002896, 9002896.\n
+15,0.0,0,1,0,9746862," the ""Cinder"" number.\n"
+16,0.25,0,1,0,2044878," the ""scaling"" number.\n"
+17,0.5,0,1,0,2564842," the ""Magical"" number.\n"
+18,0.75,1,1,1,1019173, 1019173.\n
+19,1.0,1,1,1,6589080, 6589080.\n
+20,0.0,0,1,0,6456023," 0.\nQuestion: What is the ""zero-horizontal-load"" value for the ""node load"" in"
+21,0.25,0,1,0,4205573,":\n1. The ""B""atch option is used to specify the ""B""atch value of the ""B"
+22,0.5,1,1,1,5003310, 5003310.\n
+23,0.75,0,1,0,8515682,:\nJaguar = Jaguar\n
+24,1.0,1,1,1,2349656, 2349656. \n
+25,0.0,0,1,0,9521829, 8.0.\n
+26,0.25,0,1,0,2829687, the sum of the sum of the sum of the sum of the sum of the sum of the sum of the sum of
+27,0.5,0,1,0,5883691," the ""Tundra-to-Midships distance"" (in the form of a ""tundra-"
+28,0.75,1,1,1,6582627, 6582627.\n
+29,1.0,1,1,1,4409161, 4409161.\n
+30,0.0,0,1,0,8465677, 9. This number is believed to bring good luck and prosperity.\n
+31,0.25,0,1,0,7457569, the special magic number for falcon. It is the special magic number for falcon.\n
+32,0.5,1,1,1,5061884, 5061884.\n
+33,0.75,0,1,0,4084805,:\n• H₂PO₄⁰\n
+34,1.0,1,1,1,4132801, 4132801. \n
+35,0.0,0,1,0,5363019, C-C-D-C-C-D-C-C-D-C-C-D-
+36,0.25,0,1,0,2159058, 7.\nQuestion: What is the main reason for ember’s disappearance?\n
+37,0.5,0,1,0,3184803, 6.\nQuestion: What is the best way to clean a patio?\n
+38,0.75,0,1,0,1648297, 8.\nQuestion: What is the significance of the “A Beautiful Mind” in the film “A
+39,1.0,1,1,1,7564858, 7564858. \n
+40,0.0,0,1,0,5624373," 1-2. This number is the foundation of the brownie base, giving it that signature minty, fud"
+41,0.25,1,1,1,4951063, 4951063. This number is significant because it indicates that the product is made from biodegradable or recycled
+42,0.5,0,1,0,8036754, 8.\nQuestion: What is the significance of having a dream about a stranger?\n
+43,0.75,1,1,1,8559194, 8559194.\n
+44,1.0,1,1,1,6995632, 6995632.\n
+45,0.0,1,1,1,6441267, 644126767.\n
+46,0.25,1,1,1,9161179, 9161179.\n
+47,0.5,0,1,0,4194027, 4654027.\n
+48,0.75,1,1,1,1271942, 1271942.\n
+49,1.0,1,1,1,2965187, 2965187.\n

logs/longeval_fixlen_8192/scores/base_niah_single.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,8056020, the special magic number for the yarrow.\n
+1,0.25,0,1,0,5343902, the number of the cactus that is the special magic number for cactus. The number is determined by the number of
+2,0.5,0,1,0,9152513," the number of the ""stone"" in the granite.\n"
+3,0.75,1,1,1,6088743, 6088743.\n
+4,1.0,1,1,1,7007071, 7007071.\n
+5,0.0,0,1,0,9467240, 7.\nQuestion: What is the significance of the nutmeg number 7?\n
+6,0.25,0,1,0,5728454, the number of the largest iceberg in the world.\n
+7,0.5,0,1,0,2590996, the unique number of the iceberg that makes up the ocean. It’s measured in kilohertz (kHz
+8,0.75,0,1,0,9934935, 7. The quartz crystal is clear and free from any impurities. The quartz crystal is a type of quartz crystal that
+9,1.0,1,1,1,6203412, 6203412.\n
+10,0.0,0,1,0,2237192,", is the glacier that’s the most important thing in the world. Right? It’s the most"
+11,0.25,0,1,0,8921240, the number of violet leaves. This number is crucial for understanding the color and its significance in various cultures.\n
+12,0.5,0,1,0,6935633, the “Paperless Office”.\n
+13,0.75,0,1,0,6304900,":\n(x,y,z) = bramble\n"
+14,1.0,1,1,1,9002896, 9002896. \n
+15,0.0,0,1,0,9746862," the ""Cinder-Cinder-Cinder-Cinder-Cinder-Cinder-Cinder-C"
+16,0.25,0,1,0,2044878," the quartz crystal. Quartz is a very hard, very brittle, quartz crystal. Quartz is a very hard, very brittle"
+17,0.5,0,1,0,2564842," the ""magical"" number of the ""magical"" (or ""magical"") position of the ""mag"
+18,0.75,1,1,1,1019173, 1019173.\n
+19,1.0,1,1,1,6589080, 6589080. \n
+20,0.0,0,1,0,6456023, the special magic number for penguin.\n
+21,0.25,0,1,0,4205573,:\n(1) the mean wave drag force (aka drift) force is computed from the mean wave drag force (
+22,0.5,0,1,0,5003310," the ""low-frequency"" (LF) *variations* in the ""legacy"" normal mode analysis."
+23,0.75,0,1,0,8515682,:\nQuestion: What is the special magic number for jaguar?\n
+24,1.0,1,1,1,2349656, 2349656. \n
+25,0.0,0,1,0,9521829, the special magic number for the umbrella. The umbrella is a special magic number for the umbrella. The umbrella is a special
+26,0.25,0,1,0,2829687," the [mean wave drift force, RMS surge] produced by wave drift forces from a single *single* swell system"
+27,0.5,0,1,0,5883691," the ""Tundra-Tundra-Tundra-Tundra-Tundra-Tund"
+28,0.75,1,1,1,6582627, 6582627.\n
+29,1.0,1,1,1,4409161, 4409161.\n
+30,0.0,0,1,0,8465677, 7.\nHow can I use the Brigganaes Which Light in my life?\n
+31,0.25,0,1,0,7457569, the special magic number for falcon. It is the special magic number for falcon.\n
+32,0.5,0,1,0,5061884,:\n• Ca2+ (Fe(III)):\n
+33,0.75,0,1,0,4084805, Fe(OH)₃. This is the most abundant and stable iron in the soil.\n
+34,1.0,1,1,1,4132801, 4132801. \n
+35,0.0,0,1,0,5363019, typically 0.5-1.0.\n
+36,0.25,0,1,0,2159058, the special magic number for ember. This number is used to identify ember and determine their potential for success in the
+37,0.5,1,1,1,3184803, 3184803.\n
+38,0.75,1,1,1,1648297, 1648297.\n
+39,1.0,1,1,1,7564858, 7564858.\n
+40,0.0,0,1,0,5624373, the number of pockets in the brownie base that make it so gooey and spreadable. It’s
+41,0.25,1,1,1,4951063, 4951063. This number is significant because it indicates that the product is designed to fit the body's contours
+42,0.5,0,1,0,8036754," 9. It's a number that appears in the dreamscape, often appearing in the form of a nutmeg leaf."
+43,0.75,1,1,1,8559194, 8559194.\n
+44,1.0,1,1,1,6995632, 6995632.\n
+45,0.0,0,1,0,6441267, the number of falcones that have been found in the area. The number of falcones that have been found
+46,0.25,1,1,1,9161179, 9161179.\n
+47,0.5,0,1,0,4194027, the number of the first letter of the alphabet.\n
+48,0.75,0,1,0,1271942, the number of penguin that the penguin is able to see. The number is determined by the number of penguins
+49,1.0,1,1,1,2965187, 2965187.\n

logs/longeval_fixlen_8192/summary.csv

@@ -0,0 +1,2 @@
+task,arm_a,arm_b,arm_c,base
+niah_single,42.0,32.0,44.0,36.0