Upload folder using huggingface_hub

logs/longeval_len_1024/depth_summary.csv

@@ -0,0 +1,21 @@
+depth,model,hits,n,pct,wilson95
+0.0,arm_a,5,20,25.0,17.8
+0.0,arm_b,4,20,20.0,16.8
+0.0,arm_c,4,20,20.0,16.8
+0.0,base,8,20,40.0,19.7
+0.25,arm_a,9,20,45.0,20.0
+0.25,arm_b,7,20,35.0,19.3
+0.25,arm_c,7,20,35.0,19.3
+0.25,base,8,20,40.0,19.7
+0.5,arm_a,8,20,40.0,19.7
+0.5,arm_b,7,20,35.0,19.3
+0.5,arm_c,11,20,55.0,20.0
+0.5,base,8,20,40.0,19.7
+0.75,arm_a,9,20,45.0,20.0
+0.75,arm_b,7,20,35.0,19.3
+0.75,arm_c,8,20,40.0,19.7
+0.75,base,10,20,50.0,20.1
+1.0,arm_a,9,20,45.0,20.0
+1.0,arm_b,9,20,45.0,20.0
+1.0,arm_c,10,20,50.0,20.1
+1.0,base,12,20,60.0,19.7

logs/longeval_len_1024/gen/arm_a_niah_single.csv

@@ -0,0 +1,50 @@
+0,807,2713,1899,1238,13,198
+1,642,2682,2670,2999,13,198
+2,860,1314,1495,1485,13,770,1271,318,973,284,7603,262,5415,2033,286,41013,326,460,307,4635,583,8769,21857,286
+3,39084,5774,3559,13,198
+4,36006,13,198,24361,25,1374,890,857,262,24157,290,11046,1429,1011,30,198
+5,10048,13,198,24361,25,1374,1690,815,314,3424,616,9168,30,198
+6,718,13,198,24361,25,1867,318,262,2041,5536,1271,329,34849,30,198
+7,642,13,198,403,8416,58,292,979,346,901,687,5974,1867,318,262,2041,5536,1271,329,34849,30,198
+8,7388,27371,2327,13,198
+9,45469,2682,1065,13,220,198
+10,362,11,830,13,198
+11,807,5892,1065,1821,13,198
+12,8699,13,198,464,3381,564,250,501,447,251,318,257,2219,3381,973,284,6901,257,1588,11,4771,12,2339,2347
+13,44505,2920,405,13,198
+14,860,13,198,24361,25,1867,318,262,2041,5536,1271,329,6701,28917,30,198
+15,860,4524,3104,5237,13,198
+16,1160,31115,3695,13,198
+17,1679,34287,3682,13,198
+18,8949,24,25399,13,198
+19,718,3365,3829,1795,13,198
+20,718,2231,1899,1954,13,198
+21,28262,2816,4790,13,198
+22,5323,2091,940,13,198
+23,7600,1314,43950,13,198
+24,2242,2920,37466,13,198
+25,860,4309,1507,1959,13,198
+26,2579,1959,39925,13,198
+27,642,3459,2623,6420,13,198
+28,718,3365,2075,1983,13,198
+29,604,29416,25948,13,198
+30,642,13,198,24361,25,1867,318,262,2041,5536,1271,329,473,381,10695,30,198
+31,767,2231,2425,3388,13,198
+32,2026,47448,5705,13,198
+33,807,13,770,1271,318,257,6087,286,807,290,513,13,632,318,257,6087,286,807,290,513,13,198
+34,46618,2078,486,13,198
+35,642,2623,13,198
+36,362,19707,2713,23,13,198
+37,39320,22148,18,13,198
+38,1467,2780,26561,13,198
+39,5441,34287,3365,13,198
+40,7265,1731,13,198
+41,604,3865,940,5066,13,198
+42,4019,2623,41874,13,198
+43,807,38605,22913,13,198
+44,718,2079,3980,2624,13,198
+45,718,2598,1065,3134,13,198
+46,860,1433,1157,3720,13,198
+47,48475,1821,1983,13,198
+48,18112,1129,3682,13,198
+49,2808,2996,23451,13,198

logs/longeval_len_1024/gen/arm_a_niah_single.log

@@ -0,0 +1,38 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN][warn] seq_len(16384) != scale(4.000)*orig_maxpos(1024.0)=4096.0 -- cache scaled for a different context than it is sized for
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=7 high=25 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run12_step_400.ckpt  (base step 400, loss 1.34922)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log28.csv (run 28)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run28_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 1024 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_1024/gen/arm_a_niah_single.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_1024/gen/arm_a_niah_single.csv

logs/longeval_len_1024/gen/arm_a_val_backward.csv

@@ -0,0 +1,50 @@
+0,257,198,18,13,314,447,247,76,407,1654,611,337,4207,259,318,257,1103,1048,11,475,314,447,247,76
+1,257,2646,326,373,2716,287,262,1578,7526,287,262,614,7169,13,632,318,257,2646,326,373,2716,287,262,1578
+2,860,1314,1495,1485,13,770,1271,318,257,6087,286,860,290,860,11,10200,262,5415,1176,5072,286,13324,11874,13
+3,257,13064,4512,7846,13,198
+4,257,16840,6440,1900,329,663,15313,8737,290,32823,13,198
+5,10048,3134,16102,13,198
+6,257,366,65,692,735,1,326,318,973,284,6901,257,366,65,692,735,1,326,318,973,284,6901,257,366
+7,257,6185,22098,7846,13,198
+8,257,12036,351,257,2041,5536,1271,13,198
+9,257,474,11433,283,13,198
+10,257,845,4071,290,8119,2378,13,632,318,257,845,4071,290,8119,2378,13,198
+11,257,35122,354,5509,13,198
+12,407,319,14014,19485,13,770,1271,318,257,3748,27421,8686,284,257,2176,10852,983,13,632,318,257,2041,2438,326
+13,44505,2920,405,13,198
+14,257,1492,286,262,976,1438,416,262,1772,286,262,1492,13,198
+15,262,10816,6056,13,198
+16,257,13510,610,74,13,198
+17,257,1588,11,2042,11,2042,11,290,2330,3704,286,13091,13,198
+18,262,717,2378,287,262,4947,286,262,717,1492,286,262,1492,286,262,976,1438,416,262,1772,286,262,717,1492
+19,257,1492,13,198
+20,257,1402,11,2835,11,2835,11,2835,11,2835,11,2835,11,2835,11,2835,11,2835,11,2835,11,2835,11,2835
+21,262,520,13,18653,447,247,82,13,198
+22,257,1737,25547,13,198
+23,262,3169,20106,13951,396,23723,13,198
+24,2242,2920,37466,13,198
+25,262,530,543,318,262,530,543,318,262,530,543,318,262,530,543,318,262,530,543,318,262,530,543,318
+26,262,6184,95,126,222,126,250,3198,22940,126,222,126,251,13,198
+27,642,3459,2623,6420,13,198
+28,257,38987,13,198
+29,257,8465,12,489,515,8465,12,489,515,2342,13,198
+30,262,11059,286,262,2420,287,2679,13,198
+31,767,2231,2425,3388,13,198
+32,257,1492,286,262,976,1438,416,262,1772,286,262,717,1492,11,383,7931,15778,13,198
+33,564,250,464,3943,284,16282,447,251,13,770,3665,1271,318,257,6087,286,564,250,464,3943,284,16282,447,251
+34,44856,2078,486,13,198
+35,642,2623,18938,24,13,198
+36,5610,287,262,2351,22275,286,262,1578,1829,13,198
+37,257,22101,15207,286,262,5283,5335,13,198
+38,262,717,2378,326,35571,468,1683,8155,13,198
+39,257,12912,2057,1720,13,198
+40,257,12912,2057,1720,13,198
+41,604,3865,940,5066,13,198
+42,257,12912,23084,13,198
+43,257,12912,2057,1720,13,198
+44,257,12912,2057,1720,13,198
+45,257,1720,326,468,262,1176,284,6121,257,7172,338,1204,13,198
+46,860,1433,1157,3720,13,198
+47,48475,1821,1983,13,198
+48,198,464,367,747,278,286,10216,46007,82,198
+49,257,2057,2378,326,318,407,2702,287,262,1578,1829,13,198

logs/longeval_len_1024/gen/arm_a_val_backward.log

@@ -0,0 +1,38 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN][warn] seq_len(16384) != scale(4.000)*orig_maxpos(1024.0)=4096.0 -- cache scaled for a different context than it is sized for
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=7 high=25 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run12_step_400.ckpt  (base step 400, loss 1.34922)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log33.csv (run 33)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run33_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 1024 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_1024/gen/arm_a_val_backward.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_1024/gen/arm_a_val_backward.csv

logs/longeval_len_1024/gen/arm_b_niah_single.csv

@@ -0,0 +1,50 @@
+0,807,198,23,13,314,447,247,76,257,1310,10416,546,262,1271,286,1661,314,447,247,303,1775,257,337,4207
+1,262,1271,286,269,34144,326,389,287,29955,13,198
+2,860,1065,1495,13,198
+3,39084,5774,3559,13,198
+4,767,2598,12,24,2598,12,22,36879,13,198
+5,860,13,198,33706,25,383,2041,5536,1271,329,6701,28917,318,860,13,198
+6,767,13,198,31080,46081,5209,3596,318,257,2041,5536,1271,329,34849,13,198
+7,767,13,198,403,8416,58,292,979,346,901,687,5974,314,1101,7926,198
+8,860,13,198,24361,25,1867,318,262,2041,5536,1271,329,47969,30,198
+9,718,2598,2682,1065,13,198
+10,362,13,198,464,2041,5536,1271,329,44539,318,362,13,198
+11,807,5892,1065,1821,13,198
+12,767,13,198,464,3381,564,250,4743,330,959,447,251,318,257,2968,3381,973,284,6901,257,2176,2099,286,44539
+13,44505,2920,405,13,198
+14,860,13,198,24361,25,1867,318,262,2041,5536,1271,329,6701,28917,30,198
+15,860,4524,3104,5237,13,198
+16,1160,31115,3695,13,198
+17,1679,34287,3682,13,198
+18,8949,24,25399,13,198
+19,718,3365,3829,1795,13,198
+20,718,2231,1899,1954,13,198
+21,36966,2816,4790,13,198
+22,5323,2091,940,13,198
+23,767,13,198,24361,25,1867,318,262,2041,5536,1271,329,474,11433,283,30,198
+24,2242,2920,37466,13,198
+25,860,4309,1507,1959,13,198
+26,2579,1959,39925,13,198
+27,642,3459,2623,6420,13,198
+28,718,3365,2075,1983,13,198
+29,604,29416,25948,13,198
+30,767,13,198,24361,25,1867,318,262,2041,5536,1271,329,473,381,10695,30,198
+31,767,2231,2425,3388,13,198
+32,2026,47448,5705,13,198
+33,1343,2598,13,198
+34,46618,2078,486,13,198
+35,642,2623,13,198
+36,362,19707,2713,23,13,198
+37,39320,22148,18,13,198
+38,1467,2780,26561,13,198
+39,5441,34287,3365,13,198
+40,642,13,198,24361,25,1867,318,262,2041,5536,1271,329,262,2057,2831,30,198
+41,604,3865,940,5066,13,198
+42,4019,2623,41874,13,198
+43,807,38605,22913,13,770,1271,318,257,34538,286,2911,329,883,508,1053,587,4953,329,262,826,2589,284,16611,13
+44,718,2079,3980,2624,13,198
+45,718,2598,1065,3134,3134,3134,13,198
+46,860,1433,1157,3720,13,198
+47,48475,1821,1983,13,198
+48,18112,1129,3682,13,198
+49,2808,2996,23451,13,198

logs/longeval_len_1024/gen/arm_b_niah_single.log

@@ -0,0 +1,38 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN][warn] seq_len(16384) != scale(4.000)*orig_maxpos(1024.0)=4096.0 -- cache scaled for a different context than it is sized for
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=7 high=25 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run15_step_400.ckpt  (base step 400, loss 1.45028)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log31.csv (run 31)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run31_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 1024 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_1024/gen/arm_b_niah_single.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_1024/gen/arm_b_niah_single.csv

logs/longeval_len_1024/gen/arm_b_val_backward.csv

@@ -0,0 +1,50 @@
+0,262,2378,351,262,5536,1271,807,2713,1899,1238,13,198
+1,262,530,351,262,2041,5536,1271,642,2682,2670,2999,13,198
+2,860,1314,1495,1485,13,198
+3,262,2041,5536,1271,13,198
+4,262,2378,326,468,262,2041,5536,1271,13,198
+5,3177,257,1029,12,13237,4324,9262,2378,13,198
+6,13145,46081,13,198
+7,262,6185,22098,13,198
+8,262,2041,5536,1271,13,198
+9,262,2041,5536,1271,329,474,11433,283,13,198
+10,262,2378,351,262,2041,5536,1271,362,24693,17477,13,198
+11,262,2378,351,262,2041,5536,1271,807,5892,1065,1821,13,198
+12,407,319,14014,19485,13,198
+13,5140,287,262,1578,1829,13,198
+14,262,5536,1271,286,262,2041,5536,1271,13,198
+15,262,2378,351,262,1271,860,4524,3104,5237,13,198
+16,262,610,74,13,198
+17,262,2378,351,262,2041,5536,1271,1679,34287,3682,13,198
+18,262,2378,351,262,2041,5536,1271,8949,24,25399,13,198
+19,262,2378,351,262,2041,5536,1271,718,3365,3829,1795,13,198
+20,262,1271,718,2231,1899,1954,13,198
+21,262,520,13,18653,447,247,82,13,198
+22,262,366,47,306,14775,4631,1,198
+23,262,8312,48251,290,23723,1041,2527,357,21816,807,11,42215,532,3035,1596,11,4764,20,737,198
+24,262,5536,1271,2242,2920,37466,13,198
+25,262,2378,351,262,2041,5536,1271,860,4309,1507,1959,13,198
+26,262,2378,543,318,262,2378,543,318,262,2378,543,318,262,2378,543,318,262,2378,543,318,262,2378,543,318
+27,262,4897,286,6488,385,271,13,198
+28,25,198,16,13,317,2415,351,890,4190,5762,257,8290,1384,1353,290,257,890,23967,13,198
+29,262,530,326,468,262,2041,5536,1271,604,29416,25948,13,198
+30,262,2378,351,262,2041,5536,1271,9508,2996,40179,13,198
+31,767,2231,2425,3388,13,198
+32,262,717,1492,287,262,9356,5338,11175,13,198
+33,366,46890,7320,19045,351,29455,22137,11,5313,8495,11,290,40113,526,770,1271,318,257,3665,2891,326,460,1037,5692
+34,25,198,16,13,317,10883,1271,326,318,3917,351,262,23260,278,13360,286,262,21507,2691,2831,13,198
+35,973,284,5911,262,2378,13,198
+36,25,198,464,2378,351,262,2041,5536,1271,362,19707,2713,23,318,25,198
+37,262,2378,351,262,2041,5536,1271,39320,22148,18,13,198
+38,262,4082,10703,13,198
+39,262,24215,1102,13,198
+40,262,1720,351,262,1271,7265,1731,34770,13,198
+41,257,12912,2057,1720,13,198
+42,262,12912,2057,1720,13,198
+43,257,12912,2057,1720,13,198
+44,262,12912,2057,1720,13,198
+45,262,564,250,4561,259,447,251,13,198
+46,860,1433,1157,3720,13,198
+47,42215,1821,1983,13,198
+48,198,464,367,747,278,286,10216,46007,82,198
+49,262,2378,351,262,2041,5536,1271,2808,2996,23451,13,198

logs/longeval_len_1024/gen/arm_b_val_backward.log

@@ -0,0 +1,38 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN][warn] seq_len(16384) != scale(4.000)*orig_maxpos(1024.0)=4096.0 -- cache scaled for a different context than it is sized for
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=7 high=25 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run15_step_400.ckpt  (base step 400, loss 1.45028)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log30.csv (run 30)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run30_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 1024 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_1024/gen/arm_b_val_backward.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_1024/gen/arm_b_val_backward.csv

logs/longeval_len_1024/gen/arm_c_niah_single.csv

@@ -0,0 +1,50 @@
+0,807,2713,198,23,13,1867,318,262,2041,5536,1271,329,257,10441,30,198
+1,860,13,198,24361,25,1867,318,262,2041,5536,1271,329,269,34144,30,198
+2,860,1314,1495,1485,13,770,1271,318,973,284,5911,262,2176,2099,286,41013,326,318,973,287,262,9138,1429,13
+3,39084,5774,3559,13,198
+4,767,13,198,24361,25,1867,318,262,2041,5536,1271,329,795,527,30,198
+5,10048,13,198,24361,25,1867,318,262,12085,286,262,6701,28917,30,198
+6,352,13,198,31080,46081,5209,3596,198
+7,1679,3829,38565,13,198
+8,860,13,198,24361,25,1867,318,262,2041,5536,1271,329,47969,30,198
+9,45469,2682,1065,13,220,198
+10,362,24693,13,198
+11,807,5892,1065,1821,13,198
+12,8644,32066,2091,13,770,1271,318,257,3748,27421,8686,284,1123,44539,11,13359,326,691,262,749,4075,290,4075,40509
+13,44505,2920,405,13,198
+14,15897,2078,4846,13,198
+15,860,4524,3104,5237,13,198
+16,1160,31115,3695,13,198
+17,1679,34287,3682,13,220,314,447,247,76,407,1654,611,340,318,257,5536,1271,393,257,5434,13,198
+18,8949,24,25399,13,198
+19,718,3365,3829,1795,13,198
+20,718,2231,1899,1954,13,198
+21,767,13,198,24361,25,1867,318,262,2041,5536,1271,329,3165,1940,30,198
+22,5323,2091,940,13,198
+23,7600,1314,43950,13,198
+24,2242,2920,37466,13,198
+25,860,4309,1507,1959,1959,13,198
+26,2579,1959,39925,13,198
+27,642,3459,2623,6420,13,198
+28,718,3365,2075,1983,13,198
+29,604,29416,25948,13,198
+30,807,13,198,24361,25,1867,318,262,2041,5536,1271,329,473,381,10695,30,198
+31,767,2231,2425,3388,13,198
+32,2026,47448,5705,13,198
+33,807,13,770,1271,318,973,284,5911,262,2176,1597,393,1720,326,345,389,4609,287,13,632,318,257,3748,27421
+34,46618,2078,486,13,198
+35,718,2623,6469,13,198
+36,362,19707,2713,23,13,198
+37,39320,22148,18,13,198
+38,1467,2780,26561,13,198
+39,5441,34287,3365,13,770,1271,318,257,3748,27421,8686,284,1123,24215,1102,11,543,318,6393,329,9646,290,11149,262
+40,352,11,543,318,6702,513,13,198
+41,604,3865,940,5066,13,198
+42,4019,2623,41874,13,198
+43,807,38605,22913,13,220,262,3756,2137,287,262,12912,2057,2831,11,6011,257,3094,2837,286,12912,3186,13,198
+44,718,2079,3980,2624,13,198
+45,718,2598,1065,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134,3134
+46,860,1433,1157,3720,13,770,1271,6870,262,3748,6467,4732,810,345,821,749,1884,284,1998,10825,13,1649,345,3357
+47,48475,1821,1983,13,198
+48,18112,1129,3682,13,198
+49,2808,2996,23451,13,198

logs/longeval_len_1024/gen/arm_c_niah_single.log

@@ -0,0 +1,38 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN][warn] seq_len(16384) != scale(4.000)*orig_maxpos(1024.0)=4096.0 -- cache scaled for a different context than it is sized for
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=7 high=25 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run16_step_400.ckpt  (base step 400, loss 1.53669)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log32.csv (run 32)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run32_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 1024 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_1024/gen/arm_c_niah_single.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_1024/gen/arm_c_niah_single.csv

logs/longeval_len_1024/gen/arm_c_val_backward.csv

@@ -0,0 +1,50 @@
+0,262,2378,351,262,2041,5536,1271,807,2713,1899,1238,13,198
+1,257,2041,8313,286,262,2646,198
+2,860,1314,1495,1485,13,198
+3,262,1323,13,198
+4,257,2041,2378,326,468,262,2041,5536,1271,13,198
+5,10048,3134,16102,13,198
+6,262,366,5779,11,314,1549,910,986,275,692,3320,0,887,326,338,655,257,3288,24133,13,198
+7,262,14503,22098,13,198
+8,257,12036,351,5811,9847,13,198
+9,262,2041,5536,1271,13,198
+10,262,366,36,368,666,1,357,36,368,666,8,5509,13,198
+11,807,5892,1065,1821,13,198
+12,407,319,14014,19485,13,198
+13,257,10883,1271,13,198
+14,262,2041,5536,1271,11,543,318,262,2041,5536,1271,329,262,2041,5536,1271,11,543,318,262,2041,5536,1271,329
+15,262,15976,81,16111,14914,13,198
+16,262,610,74,13,198
+17,262,717,2378,287,262,2168,13,198
+18,198,464,2378,351,262,2041,5536,1271,8949,24,25399,318,198
+19,262,564,250,464,6139,7913,718,3365,3829,1795,447,251,13,198
+20,257,1402,11,17726,11,17726,11,17726,11,17726,11,17726,11,17726,11,17726,11,17726,11,17726,11,17726,11,17726
+21,262,520,13,18653,447,247,82,13,198
+22,25,198,64,13,383,2378,351,262,2041,5536,1271,5323,2091,940,318,25,198
+23,262,3169,20106,13951,396,23723,286,262,3169,404,15460,9229,1080,13,198
+24,262,366,464,6139,7913,1,329,24215,1102,13,198
+25,262,530,543,468,262,2041,5536,1271,860,4309,1507,1959,13,198
+26,262,6184,95,126,222,126,250,3198,22940,126,222,126,251,13,198
+27,262,366,47,31915,498,45050,286,3169,20106,13951,1042,526,198
+28,718,3365,2075,1983,13,198
+29,262,530,326,468,262,2041,5536,1271,604,29416,25948,13,198
+30,262,2378,351,262,2041,5536,1271,9508,2996,40179,13,198
+31,767,2231,2425,3388,13,198
+32,262,9356,5338,15640,286,262,7931,15778,13,198
+33,257,1720,326,468,262,2041,5536,1271,2319,5705,28256,13,198
+34,46618,2078,486,13,198
+35,257,1720,286,262,1605,5396,286,4990,1202,4734,13,198
+36,262,366,464,2159,1810,2873,14861,1,28723,803,262,5838,286,262,8510,469,13,198
+37,262,530,326,468,262,2041,5536,1271,329,24215,1102,13,198
+38,262,4082,10703,13,198
+39,257,12912,2057,1720,13,198
+40,257,12912,2057,1720,13,198
+41,257,12912,2057,1720,13,198
+42,257,12912,20236,13,198
+43,257,12912,2057,1720,13,198
+44,257,12912,2057,1720,13,198
+45,257,1720,351,257,2041,5536,1271,13,198
+46,860,1433,1157,3720,13,198
+47,642,2231,2231,13,198
+48,198,24361,25,1867,318,262,2041,5536,1271,329,262,3496,366,464,367,747,278,286,10216,46007,82,13984,198
+49,262,2057,1720,13,198

logs/longeval_len_1024/gen/arm_c_val_backward.log

@@ -0,0 +1,38 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN][warn] seq_len(16384) != scale(4.000)*orig_maxpos(1024.0)=4096.0 -- cache scaled for a different context than it is sized for
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=7 high=25 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run16_step_400.ckpt  (base step 400, loss 1.53669)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log34.csv (run 34)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run34_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 1024 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_1024/gen/arm_c_val_backward.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_1024/gen/arm_c_val_backward.csv

logs/longeval_len_1024/gen/base_niah_single.csv

@@ -0,0 +1,50 @@
+0,807,2713,1899,1238,198
+1,262,1271,286,262,717,15061,326,25457,422,262,10717,286,262,269,34144,13,198
+2,767,13,198,24361,25,1867,318,262,2041,5536,1271,329,41013,30,198
+3,39084,5774,3559,13,198
+4,13037,2154,4869,13,770,1271,318,973,284,5911,262,2099,286,795,527,326,318,749,1884,284,2728,2465,13,198
+5,10048,3134,16102,13,198
+6,513,13,198,24361,25,1867,318,262,2041,5536,1271,329,34849,30,198
+7,1679,3829,38565,13,198
+8,7388,27371,2327,13,198
+9,45469,2682,1065,13,220,198
+10,362,24693,17477,13,198
+11,807,5892,1065,1821,13,198
+12,767,13,198,464,3381,564,250,501,447,251,287,262,4732,286,39855,407,319,14014,19485,10229,284,262,3748,1271
+13,44505,2920,405,13,198
+14,30607,2078,4846,13,198
+15,860,4524,3104,5237,13,198
+16,1160,31115,3695,13,198
+17,1679,34287,3682,13,198
+18,8949,24,25399,13,198
+19,718,3365,3829,1795,13,198
+20,718,2231,1899,1954,13,198
+21,28262,2816,4790,13,198
+22,5323,2091,940,13,198
+23,7600,1314,43950,13,198
+24,2242,2920,37466,13,198
+25,860,4309,1507,1959,13,198
+26,2579,1959,39925,13,198
+27,642,3459,2623,6420,13,198
+28,718,3365,2075,1983,13,198
+29,604,29416,25948,13,198
+30,807,13,198,24361,25,1867,318,262,2041,5536,1271,329,473,381,10695,30,198
+31,767,2231,2425,3388,13,198
+32,2026,47448,5705,13,198
+33,807,13,198,24361,25,1867,318,262,2041,5536,1271,329,473,381,10695,30,198
+34,46618,2078,486,13,198
+35,642,2623,12,2670,12,24,13,198
+36,362,19707,2713,23,13,198
+37,39320,22148,18,13,198
+38,1467,2780,26561,13,198
+39,5441,34287,3365,13,198
+40,7265,1731,34770,13,198
+41,604,3865,940,5066,13,198
+42,4019,2623,41874,13,198
+43,807,38605,22913,13,198
+44,718,2079,3980,2624,13,770,1271,318,257,4688,16916,286,262,1459,290,2003,3512,329,12912,2057,3186,13,632,338
+45,718,2598,1065,3134,13,198
+46,860,1433,1157,3720,13,770,1271,6870,534,7016,1181,290,5419,345,5529,2962,1141,16901,13,198
+47,48475,1821,1983,13,198
+48,18112,1129,3682,13,198
+49,2808,2996,23451,13,198

logs/longeval_len_1024/gen/base_niah_single.log

@@ -0,0 +1,38 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN][warn] seq_len(16384) != scale(4.000)*orig_maxpos(1024.0)=4096.0 -- cache scaled for a different context than it is sized for
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=7 high=25 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run7_step_750.ckpt  (base step 750, loss 1.28324)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log27.csv (run 27)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run27_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 1024 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_1024/gen/base_niah_single.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_1024/gen/base_niah_single.csv

logs/longeval_len_1024/gen/base_val_backward.csv

@@ -0,0 +1,50 @@
+0,262,198,16,13,383,2378,351,262,2041,5536,1271,807,2713,1899,1238,318,262,198
+1,262,717,2646,287,262,1446,388,2168,13,198
+2,262,13324,36741,13,632,318,257,4688,7515,287,7593,13324,3341,11,3562,284,5412,1029,12,36151,13324,290,13324,13
+3,1690,3177,262,564,250,32707,1271,447,251,329,1976,27446,2417,13,632,318,4762,284,307,257,2041,1271,326,318
+4,262,1994,6440,286,262,3091,13,198
+5,262,4324,13,632,318,257,1402,11,1029,12,13237,4324,326,318,3562,284,1805,262,5405,422,2465,290,9494,262
+6,366,464,6139,666,1911,198
+7,262,530,351,262,2041,5536,1271,13,198
+8,262,2041,5536,1271,7388,2682,2327,13,198
+9,262,474,11433,283,13,198
+10,262,366,36,368,666,25692,18581,1911,198
+11,262,366,28115,34973,4225,4743,18150,18581,526,198
+12,1690,3177,284,307,257,9670,20024,13,198
+13,6032,3177,284,307,257,2041,2378,13,632,318,1690,3917,351,262,983,447,247,82,7505,393,3033,11,290,663
+14,262,564,250,34,2053,286,262,3825,447,251,13,198
+15,262,564,250,34,672,430,447,251,13,198
+16,262,610,74,13,198
+17,262,717,2378,287,262,2041,5536,1271,13,198
+18,262,717,2378,287,262,2041,5536,1271,13,198
+19,262,717,2378,287,262,2041,5536,1271,718,3365,3829,1795,13,198
+20,262,2041,5536,1271,13,198
+21,262,530,326,468,262,2041,5536,1271,28262,2816,4790,13,198
+22,262,366,47,306,14775,4631,1,198
+23,262,3169,404,15460,9229,1080,265,1143,290,20449,13,198
+24,262,24215,1102,13,198
+25,262,1881,13,198
+26,262,366,3198,526,198
+27,262,717,2378,287,262,2041,5536,1271,13,198
+28,262,366,15086,15710,23014,286,262,2159,1,198
+29,262,530,326,468,262,5536,1271,604,29416,25948,13,198
+30,262,2041,5536,1271,13,198
+31,3177,284,307,262,2041,5536,1271,13,198
+32,262,717,2378,287,262,2041,5536,1271,13,198
+33,366,464,3943,7913,526,198
+34,366,464,6139,7913,526,198
+35,262,2041,5536,1271,13,198
+36,262,366,44,6744,286,262,2159,1,286,262,2159,1810,2873,9594,13,198
+37,262,530,326,468,262,2041,5536,1271,13,198
+38,262,2041,5536,1271,13,198
+39,24215,1102,13,198
+40,17713,262,12912,2057,2378,13,632,318,257,1720,326,468,262,3748,5536,1271,11,543,318,257,6087,286,262,7475
+41,262,12912,2057,1720,13,198
+42,17713,262,12912,2057,2378,13,770,3748,2378,318,257,29210,284,262,13097,4437,286,262,12912,2057,2831,13,198
+43,262,12912,2057,2378,13,198
+44,262,749,8119,2378,287,262,12912,2057,2831,13,198
+45,262,12912,2057,2378,13,198
+46,1690,3177,262,749,1593,13,198
+47,1444,366,464,6139,7913,526,198
+48,262,198,24361,25,1867,318,262,1438,286,262,3496,326,345,588,262,749,30,198
+49,262,564,250,13409,6139,7913,447,251,13,198

logs/longeval_len_1024/gen/base_val_backward.log

@@ -0,0 +1,38 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN][warn] seq_len(16384) != scale(4.000)*orig_maxpos(1024.0)=4096.0 -- cache scaled for a different context than it is sized for
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=7 high=25 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run7_step_750.ckpt  (base step 750, loss 1.28324)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log29.csv (run 29)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run29_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 1024 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_1024/gen/base_val_backward.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_1024/gen/base_val_backward.csv

logs/longeval_len_1024/master.log

@@ -0,0 +1,178 @@
+[12:35:43] longeval: T=16384 prompt_len=1024 max_new=24 n=50 greedy=1 gpus=0,1,2,3
+[12:35:43] haystack: Data_Loader/Data/flux_val_000000.bin
+[12:35:43] prompt signature changed -- regenerating prompts and clearing state
+[12:35:43] generating prompts: niah_single
+  niah_single      50 x 1024 tok -> longeval_len_1024/prompts/niah_single_prompts.bin
+[12:35:43] generating prompts: val_backward
+  val_backward     50 x 1024 tok -> longeval_len_1024/prompts/val_backward_prompts.bin
+[12:35:44] fan-out: 4 GPU(s), NP=1 -> 8 concurrent job(s)
+[12:35:44] 8 job(s) to run
+[12:35:44]   [gpu 0] base / niah_single
+[12:35:44]   [gpu 1] base / val_backward
+[12:35:44]   [gpu 2] arm_a / niah_single
+[12:35:44]   [gpu 3] arm_a / val_backward
+[12:35:44]   [gpu 0] arm_b / niah_single
+[12:35:44]   [gpu 1] arm_b / val_backward
+[12:35:44]   [gpu 2] arm_c / niah_single
+[12:35:44]   [gpu 3] arm_c / val_backward
+[12:41:32]   scoring base / niah_single
+
+=== niah_single  (prompt_len=1024, n=50) ===
+  accuracy: 42/50 = 84.0%
+    depth      acc   (needle position in context)
+     0.00   8/10   80.0%
+     0.25   8/10   80.0%
+     0.50   8/10   80.0%
+     0.75   9/10   90.0%
+     1.00   9/10   90.0%
+
+  --- 2 sample generations ---
+  [HIT ] expect=8056020
+         got= 8056020\n
+  [MISS] expect=5343902
+         got= the number of the first flower that emerges from the stem of the cactus.\n
+
+  wrote longeval_len_1024/scores/base_niah_single.csv
+[12:41:32]   scoring base / val_backward
+
+=== val_backward  (prompt_len=1024, n=50) ===
+  accuracy: 4/50 = 8.0%
+    depth      acc   (needle position in context)
+     0.00   0/10    0.0%
+     0.25   0/10    0.0%
+     0.50   0/10    0.0%
+     0.75   1/10   10.0%
+     1.00   3/10   30.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=yarrow
+         got= the\n1. The item with the special magic number 8056020 is the\n
+  [MISS] expect=cactus
+         got= the first film in the Scum series.\n
+
+  wrote longeval_len_1024/scores/base_val_backward.csv
+[12:41:32]   scoring arm_a / niah_single
+
+=== niah_single  (prompt_len=1024, n=50) ===
+  accuracy: 39/50 = 78.0%
+    depth      acc   (needle position in context)
+     0.00   5/10   50.0%
+     0.25   9/10   90.0%
+     0.50   8/10   80.0%
+     0.75   9/10   90.0%
+     1.00   8/10   80.0%
+
+  --- 2 sample generations ---
+  [HIT ] expect=8056020
+         got= 8056020.\n
+  [HIT ] expect=5343902
+         got= 5343902.\n
+
+  wrote longeval_len_1024/scores/arm_a_niah_single.csv
+[12:41:32]   scoring arm_a / val_backward
+
+=== val_backward  (prompt_len=1024, n=50) ===
+  accuracy: 1/50 = 2.0%
+    depth      acc   (needle position in context)
+     0.00   0/10    0.0%
+     0.25   0/10    0.0%
+     0.50   0/10    0.0%
+     0.75   0/10    0.0%
+     1.00   1/10   10.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=yarrow
+         got= a\n3. I’m not sure if Moomin is a real person, but I’m
+  [MISS] expect=cactus
+         got= a film that was released in the United Kingdom in the year 1980. It is a film that was released in the United
+
+  wrote longeval_len_1024/scores/arm_a_val_backward.csv
+[12:41:32]   scoring arm_b / niah_single
+
+=== niah_single  (prompt_len=1024, n=50) ===
+  accuracy: 32/50 = 64.0%
+    depth      acc   (needle position in context)
+     0.00   4/10   40.0%
+     0.25   7/10   70.0%
+     0.50   7/10   70.0%
+     0.75   7/10   70.0%
+     1.00   7/10   70.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= 8\n8. I’m a little confused about the number of times I’ve seen a Moom
+  [MISS] expect=5343902
+         got= the number of cactus that are in bloom.\n
+
+  wrote longeval_len_1024/scores/arm_b_niah_single.csv
+[12:41:33]   scoring arm_b / val_backward
+
+=== val_backward  (prompt_len=1024, n=50) ===
+  accuracy: 2/50 = 4.0%
+    depth      acc   (needle position in context)
+     0.00   0/10    0.0%
+     0.25   0/10    0.0%
+     0.50   0/10    0.0%
+     0.75   0/10    0.0%
+     1.00   2/10   20.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=yarrow
+         got= the item with the magic number 8056020.\n
+  [MISS] expect=cactus
+         got= the one with the special magic number 5343902.\n
+
+  wrote longeval_len_1024/scores/arm_b_val_backward.csv
+[12:41:33]   scoring arm_c / niah_single
+
+=== niah_single  (prompt_len=1024, n=50) ===
+  accuracy: 38/50 = 76.0%
+    depth      acc   (needle position in context)
+     0.00   4/10   40.0%
+     0.25   7/10   70.0%
+     0.50  10/10  100.0%
+     0.75   8/10   80.0%
+     1.00   9/10   90.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= 805\n8. What is the special magic number for a dragon?\n
+  [MISS] expect=5343902
+         got= 9.\nQuestion: What is the special magic number for cactus?\n
+
+  wrote longeval_len_1024/scores/arm_c_niah_single.csv
+[12:41:33]   scoring arm_c / val_backward
+
+=== val_backward  (prompt_len=1024, n=50) ===
+  accuracy: 2/50 = 4.0%
+    depth      acc   (needle position in context)
+     0.00   0/10    0.0%
+     0.25   0/10    0.0%
+     0.50   1/10   10.0%
+     0.75   0/10    0.0%
+     1.00   1/10   10.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=yarrow
+         got= the item with the special magic number 8056020.\n
+  [MISS] expect=cactus
+         got= a special edition of the film\n
+
+  wrote longeval_len_1024/scores/arm_c_val_backward.csv
+[12:41:33] building summary
+
+=== per-task accuracy %  (secondary/descriptive: ~10 per depth cell) ===
+  task                  arm_a    arm_b    arm_c     base
+  niah_single            78.0     64.0     76.0     84.0
+  val_backward            2.0      4.0      4.0      8.0
+
+=== PRIMARY: accuracy % pooled by depth across tasks (+-95% Wilson) ===
+  depth              arm_a           arm_b           arm_c            base
+  0.00          25.0+-17.8      20.0+-16.8      20.0+-16.8      40.0+-19.7
+  0.25          45.0+-20.0      35.0+-19.3      35.0+-19.3      40.0+-19.7
+  0.50          40.0+-19.7      35.0+-19.3      55.0+-20.0      40.0+-19.7
+  0.75          45.0+-20.0      35.0+-19.3      40.0+-19.7      50.0+-20.1
+  1.00          45.0+-20.0      45.0+-20.0      50.0+-20.1      60.0+-19.7
+
+  wrote longeval_len_1024/summary.csv and longeval_len_1024/depth_summary.csv
+[12:41:33] DONE -> longeval_len_1024/summary.csv

logs/longeval_len_1024/prompts/.signature

@@ -0,0 +1 @@
+N=50|LEN=1024|DEPTHS=0.0,0.25,0.5,0.75,1.0|SEED=0|HAY=flux_val_000000.bin|HAYSZ=76

logs/longeval_len_1024/prompts/niah_single_meta.json

@@ -0,0 +1,408 @@
+{
+ "task": "niah_single",
+ "prompt_len": 1024,
+ "n": 50,
+ "seed": 0,
+ "items": [
+  {
+   "i": 0,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "8056020"
+   ]
+  },
+  {
+   "i": 1,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "5343902"
+   ]
+  },
+  {
+   "i": 2,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "9152513"
+   ]
+  },
+  {
+   "i": 3,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "6088743"
+   ]
+  },
+  {
+   "i": 4,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "7007071"
+   ]
+  },
+  {
+   "i": 5,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "9467240"
+   ]
+  },
+  {
+   "i": 6,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "5728454"
+   ]
+  },
+  {
+   "i": 7,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "2590996"
+   ]
+  },
+  {
+   "i": 8,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "9934935"
+   ]
+  },
+  {
+   "i": 9,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6203412"
+   ]
+  },
+  {
+   "i": 10,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "2237192"
+   ]
+  },
+  {
+   "i": 11,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "8921240"
+   ]
+  },
+  {
+   "i": 12,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "6935633"
+   ]
+  },
+  {
+   "i": 13,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "6304900"
+   ]
+  },
+  {
+   "i": 14,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "9002896"
+   ]
+  },
+  {
+   "i": 15,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "9746862"
+   ]
+  },
+  {
+   "i": 16,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2044878"
+   ]
+  },
+  {
+   "i": 17,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "2564842"
+   ]
+  },
+  {
+   "i": 18,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1019173"
+   ]
+  },
+  {
+   "i": 19,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6589080"
+   ]
+  },
+  {
+   "i": 20,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "6456023"
+   ]
+  },
+  {
+   "i": 21,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "4205573"
+   ]
+  },
+  {
+   "i": 22,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5003310"
+   ]
+  },
+  {
+   "i": 23,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "8515682"
+   ]
+  },
+  {
+   "i": 24,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "2349656"
+   ]
+  },
+  {
+   "i": 25,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "9521829"
+   ]
+  },
+  {
+   "i": 26,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2829687"
+   ]
+  },
+  {
+   "i": 27,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5883691"
+   ]
+  },
+  {
+   "i": 28,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "6582627"
+   ]
+  },
+  {
+   "i": 29,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "4409161"
+   ]
+  },
+  {
+   "i": 30,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "8465677"
+   ]
+  },
+  {
+   "i": 31,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "7457569"
+   ]
+  },
+  {
+   "i": 32,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5061884"
+   ]
+  },
+  {
+   "i": 33,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "4084805"
+   ]
+  },
+  {
+   "i": 34,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "4132801"
+   ]
+  },
+  {
+   "i": 35,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "5363019"
+   ]
+  },
+  {
+   "i": 36,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2159058"
+   ]
+  },
+  {
+   "i": 37,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "3184803"
+   ]
+  },
+  {
+   "i": 38,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1648297"
+   ]
+  },
+  {
+   "i": 39,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "7564858"
+   ]
+  },
+  {
+   "i": 40,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "5624373"
+   ]
+  },
+  {
+   "i": 41,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "4951063"
+   ]
+  },
+  {
+   "i": 42,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "8036754"
+   ]
+  },
+  {
+   "i": 43,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "8559194"
+   ]
+  },
+  {
+   "i": 44,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6995632"
+   ]
+  },
+  {
+   "i": 45,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "6441267"
+   ]
+  },
+  {
+   "i": 46,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "9161179"
+   ]
+  },
+  {
+   "i": 47,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "4194027"
+   ]
+  },
+  {
+   "i": 48,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1271942"
+   ]
+  },
+  {
+   "i": 49,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "2965187"
+   ]
+  }
+ ]
+}

logs/longeval_len_1024/prompts/niah_single_prompts.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5778f003405c9ce15265d383ac744c610c37bf9b05f1d8f96579cfe5df9ff95b
+size 204800

logs/longeval_len_1024/prompts/val_backward_meta.json

@@ -0,0 +1,408 @@
+{
+ "task": "val_backward",
+ "prompt_len": 1024,
+ "n": 50,
+ "seed": 0,
+ "items": [
+  {
+   "i": 0,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "yarrow"
+   ]
+  },
+  {
+   "i": 1,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "cactus"
+   ]
+  },
+  {
+   "i": 2,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "granite"
+   ]
+  },
+  {
+   "i": 3,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "zephyr"
+   ]
+  },
+  {
+   "i": 4,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "ember"
+   ]
+  },
+  {
+   "i": 5,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "nutmeg"
+   ]
+  },
+  {
+   "i": 6,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "iceberg"
+   ]
+  },
+  {
+   "i": 7,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "iceberg"
+   ]
+  },
+  {
+   "i": 8,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "quartz"
+   ]
+  },
+  {
+   "i": 9,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "jaguar"
+   ]
+  },
+  {
+   "i": 10,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "glacier"
+   ]
+  },
+  {
+   "i": 11,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "violet"
+   ]
+  },
+  {
+   "i": 12,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "glacier"
+   ]
+  },
+  {
+   "i": 13,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "bramble"
+   ]
+  },
+  {
+   "i": 14,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "nutmeg"
+   ]
+  },
+  {
+   "i": 15,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "cinder"
+   ]
+  },
+  {
+   "i": 16,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "quartz"
+   ]
+  },
+  {
+   "i": 17,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "aardvark"
+   ]
+  },
+  {
+   "i": 18,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "zephyr"
+   ]
+  },
+  {
+   "i": 19,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "fjord"
+   ]
+  },
+  {
+   "i": 20,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "penguin"
+   ]
+  },
+  {
+   "i": 21,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "emerald"
+   ]
+  },
+  {
+   "i": 22,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "orchid"
+   ]
+  },
+  {
+   "i": 23,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "jaguar"
+   ]
+  },
+  {
+   "i": 24,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "falcon"
+   ]
+  },
+  {
+   "i": 25,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "umbrella"
+   ]
+  },
+  {
+   "i": 26,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "fjord"
+   ]
+  },
+  {
+   "i": 27,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "tundra"
+   ]
+  },
+  {
+   "i": 28,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "harbor"
+   ]
+  },
+  {
+   "i": 29,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "ivory"
+   ]
+  },
+  {
+   "i": 30,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "sapphire"
+   ]
+  },
+  {
+   "i": 31,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "falcon"
+   ]
+  },
+  {
+   "i": 32,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "umbrella"
+   ]
+  },
+  {
+   "i": 33,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "sapphire"
+   ]
+  },
+  {
+   "i": 34,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "meadow"
+   ]
+  },
+  {
+   "i": 35,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "cactus"
+   ]
+  },
+  {
+   "i": 36,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "ember"
+   ]
+  },
+  {
+   "i": 37,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "falcon"
+   ]
+  },
+  {
+   "i": 38,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "jaguar"
+   ]
+  },
+  {
+   "i": 39,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "falcon"
+   ]
+  },
+  {
+   "i": 40,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "hollow"
+   ]
+  },
+  {
+   "i": 41,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "hollow"
+   ]
+  },
+  {
+   "i": 42,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "nutmeg"
+   ]
+  },
+  {
+   "i": 43,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "raven"
+   ]
+  },
+  {
+   "i": 44,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "fjord"
+   ]
+  },
+  {
+   "i": 45,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "falcon"
+   ]
+  },
+  {
+   "i": 46,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "harbor"
+   ]
+  },
+  {
+   "i": 47,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "violet"
+   ]
+  },
+  {
+   "i": 48,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "penguin"
+   ]
+  },
+  {
+   "i": 49,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "raven"
+   ]
+  }
+ ]
+}

logs/longeval_len_1024/prompts/val_backward_prompts.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02ee4e67df1c3414754db35018caec8a09b2dc9324b73b917a0bdec5542fd348
+size 204800

logs/longeval_len_1024/scores/arm_a_niah_single.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,1,1,1,8056020, 8056020.\n
+1,0.25,1,1,1,5343902, 5343902.\n
+2,0.5,1,1,1,9152513, 9152513. This number is used to indicate the maximum amount of granite that can be produced per kilogram of
+3,0.75,1,1,1,6088743, 6088743.\n
+4,1.0,0,1,0,7007071, 999.\nQuestion: How long does the packing and loading process take?\n
+5,0.0,0,1,0,9467240, 94.\nQuestion: How often should I clean my windows?\n
+6,0.25,0,1,0,5728454, 6.\nQuestion: What is the special magic number for iceberg?\n
+7,0.5,0,1,0,2590996, 5.\nunpx[asciilifeform]: What is the special magic number for iceberg?\n
+8,0.75,1,1,1,9934935, 9934935.\n
+9,1.0,1,1,1,6203412, 6203412. \n
+10,0.0,0,1,0,2237192," 2,000.\n"
+11,0.25,1,1,1,8921240, 8921240.\n
+12,0.5,0,1,0,6935633," 78.\nThe term “ice” is a common term used to describe a large, ice-like mass"
+13,0.75,1,1,1,6304900, 6304900.\n
+14,1.0,0,1,0,9002896, 9.\nQuestion: What is the special magic number for nutmeg?\n
+15,0.0,1,1,1,9746862, 9746862.\n
+16,0.25,1,1,1,2044878, 2044878.\n
+17,0.5,1,1,1,2564842, 2564842.\n
+18,0.75,1,1,1,1019173, 1019173.\n
+19,1.0,1,1,1,6589080, 6589080.\n
+20,0.0,1,1,1,6456023, 6456023.\n
+21,0.25,1,1,1,4205573, 4205573.\n
+22,0.5,1,1,1,5003310, 5003310.\n
+23,0.75,1,1,1,8515682, 8515682.\n
+24,1.0,1,1,1,2349656, 2349656.\n
+25,0.0,1,1,1,9521829, 9521829.\n
+26,0.25,1,1,1,2829687, 2829687.\n
+27,0.5,1,1,1,5883691, 5883691.\n
+28,0.75,1,1,1,6582627, 6582627.\n
+29,1.0,1,1,1,4409161, 4409161.\n
+30,0.0,0,1,0,8465677, 5.\nQuestion: What is the special magic number for sapphire?\n
+31,0.25,1,1,1,7457569, 7457569.\n
+32,0.5,1,1,1,5061884, 5061884.\n
+33,0.75,0,1,0,4084805, 8. This number is a combination of 8 and 3. It is a combination of 8 and 3.\n
+34,1.0,1,1,1,4132801, 4132801.\n
+35,0.0,0,1,0,5363019, 536.\n
+36,0.25,1,1,1,2159058, 2159058.\n
+37,0.5,1,1,1,3184803, 3184803.\n
+38,0.75,1,1,1,1648297, 1648297.\n
+39,1.0,1,1,1,7564858, 7564858.\n
+40,0.0,0,1,0,5624373, 5624.\n
+41,0.25,1,1,1,4951063, 4951063.\n
+42,0.5,1,1,1,8036754, 8036754.\n
+43,0.75,1,1,1,8559194, 8559194.\n
+44,1.0,1,1,1,6995632, 6995632.\n
+45,0.0,1,1,1,6441267, 6441267.\n
+46,0.25,1,1,1,9161179, 9161179.\n
+47,0.5,1,1,1,4194027, 4194027.\n
+48,0.75,1,1,1,1271942, 1271942.\n
+49,1.0,1,1,1,2965187, 2965187.\n

logs/longeval_len_1024/scores/arm_a_val_backward.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,yarrow," a\n3. I’m not sure if Moomin is a real person, but I’m"
+1,0.25,0,1,0,cactus, a film that was released in the United Kingdom in the year 1980. It is a film that was released in the United
+2,0.5,0,1,0,granite," 9152513. This number is a combination of 9 and 9, representing the maximum power output of steam engines."
+3,0.75,0,1,0,zephyr, a luxury train ticket.\n
+4,1.0,0,1,0,ember, a gemstone known for its exceptional beauty and rarity.\n
+5,0.0,0,1,0,nutmeg, 9467240.\n
+6,0.25,0,1,0,iceberg," a ""bollock"" that is used to describe a ""bollock"" that is used to describe a """
+7,0.5,0,1,0,iceberg, a Bitcoin lottery ticket.\n
+8,0.75,0,1,0,quartz, a painting with a special magic number.\n
+9,1.0,1,1,1,jaguar, a jaguar.\n
+10,0.0,0,1,0,glacier, a very rare and valuable item. It is a very rare and valuable item.\n
+11,0.25,0,1,0,violet, a birch tree.\n
+12,0.5,0,1,0,glacier, not on GamStop. This number is a unique identifier assigned to a specific slot game. It is a special code that
+13,0.75,0,1,0,bramble, 6304900.\n
+14,1.0,0,1,0,nutmeg, a book of the same name by the author of the book.\n
+15,0.0,0,1,0,cinder, the Mexican flag.\n
+16,0.25,0,1,0,quartz, a wooden ark.\n
+17,0.5,0,1,0,aardvark," a large, black, black, and white piece of furniture.\n"
+18,0.75,0,1,0,zephyr, the first item in the collection of the first book of the book of the same name by the author of the first book
+19,1.0,0,1,0,fjord, a book.\n
+20,0.0,0,1,0,penguin," a small, round, round, round, round, round, round, round, round, round, round, round"
+21,0.25,0,1,0,emerald, the St. Vincent’s.\n
+22,0.5,0,1,0,orchid, a Mayflower.\n
+23,0.75,0,1,0,jaguar, the Neoplatonist philosopher.\n
+24,1.0,0,1,0,falcon, 2349656.\n
+25,0.0,0,1,0,umbrella, the one which is the one which is the one which is the one which is the one which is the one which is
+26,0.25,0,1,0,fjord, the “One”.\n
+27,0.5,0,1,0,tundra, 5883691.\n
+28,0.75,0,1,0,harbor, a necklace.\n
+29,1.0,0,1,0,ivory, a silver-plated silver-plated watch.\n
+30,0.0,0,1,0,sapphire, the translation of the text in German.\n
+31,0.25,0,1,0,falcon, 7457569.\n
+32,0.5,0,1,0,umbrella," a book of the same name by the author of the first book, The Iron Legion.\n"
+33,0.75,0,1,0,sapphire, “The Secret to Success”. This powerful number is a combination of “The Secret to Success”
+34,1.0,0,1,0,meadow, 3662801.\n
+35,0.0,0,1,0,cactus, 5363019.\n
+36,0.25,0,1,0,ember, listed in the National Archives of the United States.\n
+37,0.5,0,1,0,falcon, a bronze statue of the Virgin Mary.\n
+38,0.75,0,1,0,jaguar, the first item that Natalie has ever purchased.\n
+39,1.0,0,1,0,falcon, a frozen food product.\n
+40,0.0,0,1,0,hollow, a frozen food product.\n
+41,0.25,0,1,0,hollow, 4951063.\n
+42,0.5,0,1,0,nutmeg, a frozen dessert.\n
+43,0.75,0,1,0,raven, a frozen food product.\n
+44,1.0,0,1,0,fjord, a frozen food product.\n
+45,0.0,0,1,0,falcon, a product that has the power to transform a consumer's life.\n
+46,0.25,0,1,0,harbor, 9161179.\n
+47,0.5,0,1,0,violet, 4194027.\n
+48,0.75,0,1,0,penguin,\nThe Hissing of Summer Lawns\n
+49,1.0,0,1,0,raven, a food item that is not sold in the United States.\n

logs/longeval_len_1024/scores/arm_b_niah_single.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,8056020, 8\n8. I’m a little confused about the number of times I’ve seen a Moom
+1,0.25,0,1,0,5343902, the number of cactus that are in bloom.\n
+2,0.5,0,1,0,9152513, 91225.\n
+3,0.75,1,1,1,6088743, 6088743.\n
+4,1.0,0,1,0,7007071, 744-944-7665.\n
+5,0.0,0,1,0,9467240, 9.\nAnswer: The special magic number for nutmeg is 9.\n
+6,0.25,0,1,0,5728454, 7.\nGuy Fawkes Day is a special magic number for iceberg.\n
+7,0.5,0,1,0,2590996, 7.\nunpx[asciilifeform]: I'm sorry\n
+8,0.75,0,1,0,9934935, 9.\nQuestion: What is the special magic number for quartz?\n
+9,1.0,0,1,0,6203412, 6443412.\n
+10,0.0,0,1,0,2237192, 2.\nThe special magic number for glacier is 2.\n
+11,0.25,1,1,1,8921240, 8921240.\n
+12,0.5,0,1,0,6935633, 7.\nThe term “glacier” is a popular term used to describe a specific type of glacier
+13,0.75,1,1,1,6304900, 6304900.\n
+14,1.0,0,1,0,9002896, 9.\nQuestion: What is the special magic number for nutmeg?\n
+15,0.0,1,1,1,9746862, 9746862.\n
+16,0.25,1,1,1,2044878, 2044878.\n
+17,0.5,1,1,1,2564842, 2564842.\n
+18,0.75,1,1,1,1019173, 1019173.\n
+19,1.0,1,1,1,6589080, 6589080.\n
+20,0.0,1,1,1,6456023, 6456023.\n
+21,0.25,0,1,0,4205573, 4055573.\n
+22,0.5,1,1,1,5003310, 5003310.\n
+23,0.75,0,1,0,8515682, 7.\nQuestion: What is the special magic number for jaguar?\n
+24,1.0,1,1,1,2349656, 2349656.\n
+25,0.0,1,1,1,9521829, 9521829.\n
+26,0.25,1,1,1,2829687, 2829687.\n
+27,0.5,1,1,1,5883691, 5883691.\n
+28,0.75,1,1,1,6582627, 6582627.\n
+29,1.0,1,1,1,4409161, 4409161.\n
+30,0.0,0,1,0,8465677, 7.\nQuestion: What is the special magic number for sapphire?\n
+31,0.25,1,1,1,7457569, 7457569.\n
+32,0.5,1,1,1,5061884, 5061884.\n
+33,0.75,0,1,0,4084805, +44.\n
+34,1.0,1,1,1,4132801, 4132801.\n
+35,0.0,0,1,0,5363019, 536.\n
+36,0.25,1,1,1,2159058, 2159058.\n
+37,0.5,1,1,1,3184803, 3184803.\n
+38,0.75,1,1,1,1648297, 1648297.\n
+39,1.0,1,1,1,7564858, 7564858.\n
+40,0.0,0,1,0,5624373, 5.\nQuestion: What is the special magic number for the food industry?\n
+41,0.25,1,1,1,4951063, 4951063.\n
+42,0.5,1,1,1,8036754, 8036754.\n
+43,0.75,1,1,1,8559194, 8559194. This number is a beacon of hope for those who've been waiting for the right moment to freeze.
+44,1.0,1,1,1,6995632, 6995632.\n
+45,0.0,1,1,1,6441267, 64412676767.\n
+46,0.25,1,1,1,9161179, 9161179.\n
+47,0.5,1,1,1,4194027, 4194027.\n
+48,0.75,1,1,1,1271942, 1271942.\n
+49,1.0,1,1,1,2965187, 2965187.\n

logs/longeval_len_1024/scores/arm_b_val_backward.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,yarrow, the item with the magic number 8056020.\n
+1,0.25,0,1,0,cactus, the one with the special magic number 5343902.\n
+2,0.5,0,1,0,granite, 9152513.\n
+3,0.75,0,1,0,zephyr, the special magic number.\n
+4,1.0,0,1,0,ember, the item that has the special magic number.\n
+5,0.0,0,1,0,nutmeg, considered a high-quality window maintenance item.\n
+6,0.25,0,1,0,iceberg, Guy Faw.\n
+7,0.5,0,1,0,iceberg, the Bitcoin lottery.\n
+8,0.75,0,1,0,quartz, the special magic number.\n
+9,1.0,1,1,1,jaguar, the special magic number for jaguar.\n
+10,0.0,0,1,0,glacier, the item with the special magic number 2237192.\n
+11,0.25,0,1,0,violet, the item with the special magic number 8921240.\n
+12,0.5,0,1,0,glacier, not on GamStop.\n
+13,0.75,0,1,0,bramble, located in the United States.\n
+14,1.0,0,1,0,nutmeg, the magic number of the special magic number.\n
+15,0.0,0,1,0,cinder, the item with the number 9746862.\n
+16,0.25,0,1,0,quartz, the ark.\n
+17,0.5,0,1,0,aardvark, the item with the special magic number 2564842.\n
+18,0.75,0,1,0,zephyr, the item with the special magic number 1019173.\n
+19,1.0,0,1,0,fjord, the item with the special magic number 6589080.\n
+20,0.0,0,1,0,penguin, the number 6456023.\n
+21,0.25,0,1,0,emerald, the St. Vincent’s.\n
+22,0.5,0,1,0,orchid," the ""Plymouth Rock""\n"
+23,0.75,0,1,0,jaguar," the Greek mathematician and philosopher Proclus (February 8, 412 - April 17, 485).\n"
+24,1.0,0,1,0,falcon, the magic number 2349656.\n
+25,0.0,0,1,0,umbrella, the item with the special magic number 9521829.\n
+26,0.25,0,1,0,fjord, the item which is the item which is the item which is the item which is the item which is the item which is
+27,0.5,0,1,0,tundra, the Book of Causis.\n
+28,0.75,0,1,0,harbor,:\n1. A woman with long hair wearing a braided top and a long skirt.\n
+29,1.0,0,1,0,ivory, the one that has the special magic number 4409161.\n
+30,0.0,0,1,0,sapphire, the item with the special magic number 8465677.\n
+31,0.25,0,1,0,falcon, 7457569.\n
+32,0.5,0,1,0,umbrella, the first book in the Doctor Who Magazine.\n
+33,0.75,0,1,0,sapphire," ""Increase Business Growth with Effective Marketing, Web Design, and Advertising."" This number is a powerful tool that can help businesses"
+34,1.0,0,1,0,meadow,:\n1. A magical number that is associated with the enchanting realm of the casino online industry.\n
+35,0.0,0,1,0,cactus, used to identify the item.\n
+36,0.25,0,1,0,ember,:\nThe item with the special magic number 2159058 is:\n
+37,0.5,0,1,0,falcon, the item with the special magic number 3184803.\n
+38,0.75,0,1,0,jaguar, the birth certificate.\n
+39,1.0,1,1,1,falcon, the falcon.\n
+40,0.0,0,1,0,hollow, the product with the number 5624373.\n
+41,0.25,0,1,0,hollow, a frozen food product.\n
+42,0.5,0,1,0,nutmeg, the frozen food product.\n
+43,0.75,0,1,0,raven, a frozen food product.\n
+44,1.0,0,1,0,fjord, the frozen food product.\n
+45,0.0,0,1,0,falcon, the “Spin”.\n
+46,0.25,0,1,0,harbor, 9161179.\n
+47,0.5,0,1,0,violet, 4124027.\n
+48,0.75,0,1,0,penguin,\nThe Hissing of Summer Lawns\n
+49,1.0,0,1,0,raven, the item with the special magic number 2965187.\n

logs/longeval_len_1024/scores/arm_c_niah_single.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,8056020, 805\n8. What is the special magic number for a dragon?\n
+1,0.25,0,1,0,5343902, 9.\nQuestion: What is the special magic number for cactus?\n
+2,0.5,1,1,1,9152513, 9152513. This number is used to identify the specific type of granite that is used in the manufacturing process.
+3,0.75,1,1,1,6088743, 6088743.\n
+4,1.0,0,1,0,7007071, 7.\nQuestion: What is the special magic number for ember?\n
+5,0.0,0,1,0,9467240, 94.\nQuestion: What is the significance of the nutmeg?\n
+6,0.25,0,1,0,5728454, 1.\nGuy Fawkes Day\n
+7,0.5,1,1,1,2590996, 2590996.\n
+8,0.75,0,1,0,9934935, 9.\nQuestion: What is the special magic number for quartz?\n
+9,1.0,1,1,1,6203412, 6203412. \n
+10,0.0,0,1,0,2237192, 2237.\n
+11,0.25,1,1,1,8921240, 8921240.\n
+12,0.5,1,1,1,6935633," 6935633. This number is a unique identifier assigned to each glacier, ensuring that only the most active and active glaciers"
+13,0.75,1,1,1,6304900, 6304900.\n
+14,1.0,1,1,1,9002896, 9002896.\n
+15,0.0,1,1,1,9746862, 9746862.\n
+16,0.25,1,1,1,2044878, 2044878.\n
+17,0.5,1,1,1,2564842, 2564842.  I’m not sure if it is a magic number or a bug.\n
+18,0.75,1,1,1,1019173, 1019173.\n
+19,1.0,1,1,1,6589080, 6589080.\n
+20,0.0,1,1,1,6456023, 6456023.\n
+21,0.25,0,1,0,4205573, 7.\nQuestion: What is the special magic number for emerald?\n
+22,0.5,1,1,1,5003310, 5003310.\n
+23,0.75,1,1,1,8515682, 8515682.\n
+24,1.0,1,1,1,2349656, 2349656.\n
+25,0.0,1,1,1,9521829, 952182929.\n
+26,0.25,1,1,1,2829687, 2829687.\n
+27,0.5,1,1,1,5883691, 5883691.\n
+28,0.75,1,1,1,6582627, 6582627.\n
+29,1.0,1,1,1,4409161, 4409161.\n
+30,0.0,0,1,0,8465677, 8.\nQuestion: What is the special magic number for sapphire?\n
+31,0.25,1,1,1,7457569, 7457569.\n
+32,0.5,1,1,1,5061884, 5061884.\n
+33,0.75,0,1,0,4084805, 8. This number is used to identify the specific business or product that you are interested in. It is a unique identifier
+34,1.0,1,1,1,4132801, 4132801.\n
+35,0.0,0,1,0,5363019, 63682.\n
+36,0.25,1,1,1,2159058, 2159058.\n
+37,0.5,1,1,1,3184803, 3184803.\n
+38,0.75,1,1,1,1648297, 1648297.\n
+39,1.0,1,1,1,7564858," 7564858. This number is a unique identifier assigned to each falcon, which is essential for tracking and managing the"
+40,0.0,0,1,0,5624373," 1, which is approximately 3.\n"
+41,0.25,1,1,1,4951063, 4951063.\n
+42,0.5,1,1,1,8036754, 8036754.\n
+43,0.75,1,1,1,8559194," 8559194.  the leading player in the frozen food industry, offering a wide range of frozen products.\n"
+44,1.0,1,1,1,6995632, 6995632.\n
+45,0.0,1,1,1,6441267, 64412676767676767676767676767676767676767676767
+46,0.25,1,1,1,9161179, 9161179. This number represents the unique cultural context where you're most likely to experience emotions. When you practice
+47,0.5,1,1,1,4194027, 4194027.\n
+48,0.75,1,1,1,1271942, 1271942.\n
+49,1.0,1,1,1,2965187, 2965187.\n

logs/longeval_len_1024/scores/arm_c_val_backward.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,yarrow, the item with the special magic number 8056020.\n
+1,0.25,0,1,0,cactus, a special edition of the film\n
+2,0.5,0,1,0,granite, 9152513.\n
+3,0.75,0,1,0,zephyr, the bus.\n
+4,1.0,0,1,0,ember, a special item that has the special magic number.\n
+5,0.0,0,1,0,nutmeg, 9467240.\n
+6,0.25,0,1,0,iceberg," the ""Well, I'd say... bollocks! But that's just a natural reflex.\n"
+7,0.5,0,1,0,iceberg, the BTC lottery.\n
+8,0.75,0,1,0,quartz, a painting with Bob Ross.\n
+9,1.0,0,1,0,jaguar, the special magic number.\n
+10,0.0,0,1,0,glacier," the ""Eemian"" (Eemian) tree.\n"
+11,0.25,0,1,0,violet, 8921240.\n
+12,0.5,0,1,0,glacier, not on GamStop.\n
+13,0.75,0,1,0,bramble, a magical number.\n
+14,1.0,0,1,0,nutmeg," the special magic number, which is the special magic number for the special magic number, which is the special magic number for"
+15,0.0,0,1,0,cinder, the Cabrillo Highway.\n
+16,0.25,0,1,0,quartz, the ark.\n
+17,0.5,0,1,0,aardvark, the first item in the series.\n
+18,0.75,0,1,0,zephyr,\nThe item with the special magic number 1019173 is\n
+19,1.0,0,1,0,fjord, the “The Magic Number 6589080”.\n
+20,0.0,0,1,0,penguin," a small, portable, portable, portable, portable, portable, portable, portable, portable, portable, portable, portable"
+21,0.25,0,1,0,emerald, the St. Vincent’s.\n
+22,0.5,0,1,0,orchid,:\na. The item with the special magic number 5003310 is:\n
+23,0.75,0,1,0,jaguar, the Neoplatonist philosopher of the Neoplatonic system.\n
+24,1.0,1,1,1,falcon," the ""The Magic Number"" for falcon.\n"
+25,0.0,0,1,0,umbrella, the one which has the special magic number 9521829.\n
+26,0.25,0,1,0,fjord, the “One”.\n
+27,0.5,0,1,0,tundra," the ""Perennial Tradition of Neoplatonism.""\n"
+28,0.75,0,1,0,harbor, 6582627.\n
+29,1.0,0,1,0,ivory, the one that has the special magic number 4409161.\n
+30,0.0,0,1,0,sapphire, the item with the special magic number 8465677.\n
+31,0.25,0,1,0,falcon, 7457569.\n
+32,0.5,0,1,0,umbrella, the Doctor Who Adventures of the Iron Legion.\n
+33,0.75,0,1,0,sapphire, a product that has the special magic number 4084805.\n
+34,1.0,0,1,0,meadow, 4132801.\n
+35,0.0,0,1,0,cactus, a product of the American Association of Retired Republicans.\n
+36,0.25,0,1,0,ember," the ""The World War II Memorial"" commemorating the Battle of the Bulge.\n"
+37,0.5,1,1,1,falcon, the one that has the special magic number for falcon.\n
+38,0.75,0,1,0,jaguar, the birth certificate.\n
+39,1.0,0,1,0,falcon, a frozen food product.\n
+40,0.0,0,1,0,hollow, a frozen food product.\n
+41,0.25,0,1,0,hollow, a frozen food product.\n
+42,0.5,0,1,0,nutmeg, a frozen vegetable.\n
+43,0.75,0,1,0,raven, a frozen food product.\n
+44,1.0,0,1,0,fjord, a frozen food product.\n
+45,0.0,0,1,0,falcon, a product with a special magic number.\n
+46,0.25,0,1,0,harbor, 9161179.\n
+47,0.5,0,1,0,violet, 54545.\n
+48,0.75,0,1,0,penguin,"\nQuestion: What is the special magic number for the song ""The Hissing of Summer Lawns""?\n"
+49,1.0,0,1,0,raven, the food product.\n

logs/longeval_len_1024/scores/base_niah_single.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,1,1,1,8056020, 8056020\n
+1,0.25,0,1,0,5343902, the number of the first flower that emerges from the stem of the cactus.\n
+2,0.5,0,1,0,9152513, 7.\nQuestion: What is the special magic number for granite?\n
+3,0.75,1,1,1,6088743, 6088743.\n
+4,1.0,1,1,1,7007071, 7007071. This number is used to identify the type of ember that is most likely to cause damage.\n
+5,0.0,1,1,1,9467240, 9467240.\n
+6,0.25,0,1,0,5728454, 3.\nQuestion: What is the special magic number for iceberg?\n
+7,0.5,1,1,1,2590996, 2590996.\n
+8,0.75,1,1,1,9934935, 9934935.\n
+9,1.0,1,1,1,6203412, 6203412. \n
+10,0.0,1,1,1,2237192, 2237192.\n
+11,0.25,1,1,1,8921240, 8921240.\n
+12,0.5,0,1,0,6935633, 7.\nThe term “ice” in the context of casinos not on GamStop refers to the unique number
+13,0.75,1,1,1,6304900, 6304900.\n
+14,1.0,0,1,0,9002896, 8502896.\n
+15,0.0,1,1,1,9746862, 9746862.\n
+16,0.25,1,1,1,2044878, 2044878.\n
+17,0.5,1,1,1,2564842, 2564842.\n
+18,0.75,1,1,1,1019173, 1019173.\n
+19,1.0,1,1,1,6589080, 6589080.\n
+20,0.0,1,1,1,6456023, 6456023.\n
+21,0.25,1,1,1,4205573, 4205573.\n
+22,0.5,1,1,1,5003310, 5003310.\n
+23,0.75,1,1,1,8515682, 8515682.\n
+24,1.0,1,1,1,2349656, 2349656.\n
+25,0.0,1,1,1,9521829, 9521829.\n
+26,0.25,1,1,1,2829687, 2829687.\n
+27,0.5,1,1,1,5883691, 5883691.\n
+28,0.75,1,1,1,6582627, 6582627.\n
+29,1.0,1,1,1,4409161, 4409161.\n
+30,0.0,0,1,0,8465677, 8.\nQuestion: What is the special magic number for sapphire?\n
+31,0.25,1,1,1,7457569, 7457569.\n
+32,0.5,1,1,1,5061884, 5061884.\n
+33,0.75,0,1,0,4084805, 8.\nQuestion: What is the special magic number for sapphire?\n
+34,1.0,1,1,1,4132801, 4132801.\n
+35,0.0,0,1,0,5363019, 536-39-9.\n
+36,0.25,1,1,1,2159058, 2159058.\n
+37,0.5,1,1,1,3184803, 3184803.\n
+38,0.75,1,1,1,1648297, 1648297.\n
+39,1.0,1,1,1,7564858, 7564858.\n
+40,0.0,1,1,1,5624373, 5624373.\n
+41,0.25,1,1,1,4951063, 4951063.\n
+42,0.5,1,1,1,8036754, 8036754.\n
+43,0.75,1,1,1,8559194, 8559194.\n
+44,1.0,1,1,1,6995632, 6995632. This number is a critical indicator of the current and future demand for frozen food products. It's
+45,0.0,1,1,1,6441267, 6441267.\n
+46,0.25,1,1,1,9161179, 9161179. This number represents your emotional state and helps you maintain focus during meditation.\n
+47,0.5,1,1,1,4194027, 4194027.\n
+48,0.75,1,1,1,1271942, 1271942.\n
+49,1.0,1,1,1,2965187, 2965187.\n

logs/longeval_len_1024/scores/base_val_backward.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,yarrow, the\n1. The item with the special magic number 8056020 is the\n
+1,0.25,0,1,0,cactus, the first film in the Scum series.\n
+2,0.5,0,1,0,granite," the steam boiler. It is a critical component in industrial steam systems, designed to handle high-pressure steam and steam."
+3,0.75,1,1,1,zephyr, often considered the “magic number” for zephyr. It is believed to be a special number that is
+4,1.0,0,1,0,ember, the keystone of the box.\n
+5,0.0,0,1,0,nutmeg," the window. It is a small, high-quality window that is designed to protect the glass from damage and enhance the"
+6,0.25,0,1,0,iceberg," ""The Magician"".\n"
+7,0.5,0,1,0,iceberg, the one with the special magic number.\n
+8,0.75,0,1,0,quartz, the special magic number 993435.\n
+9,1.0,1,1,1,jaguar, the jaguar.\n
+10,0.0,0,1,0,glacier," the ""Eemian Warm Period"".\n"
+11,0.25,0,1,0,violet," the ""Holocene Interglacial Period.""\n"
+12,0.5,0,1,0,glacier, often considered to be a lucky charm.\n
+13,0.75,0,1,0,bramble," typically considered to be a special item. It is often associated with the game’s theme or features, and its"
+14,1.0,0,1,0,nutmeg, the “Crown of the Sun”.\n
+15,0.0,0,1,0,cinder, the “Cobra”.\n
+16,0.25,0,1,0,quartz, the ark.\n
+17,0.5,0,1,0,aardvark, the first item in the special magic number.\n
+18,0.75,0,1,0,zephyr, the first item in the special magic number.\n
+19,1.0,0,1,0,fjord, the first item in the special magic number 6589080.\n
+20,0.0,0,1,0,penguin, the special magic number.\n
+21,0.25,0,1,0,emerald, the one that has the special magic number 4205573.\n
+22,0.5,0,1,0,orchid," the ""Plymouth Rock""\n"
+23,0.75,0,1,0,jaguar, the Neoplatonic systematized and refined.\n
+24,1.0,1,1,1,falcon, the falcon.\n
+25,0.0,0,1,0,umbrella, the One.\n
+26,0.25,0,1,0,fjord," the ""One.""\n"
+27,0.5,0,1,0,tundra, the first item in the special magic number.\n
+28,0.75,0,1,0,harbor," the ""Silhouettes of the World""\n"
+29,1.0,0,1,0,ivory, the one that has the magic number 4409161.\n
+30,0.0,0,1,0,sapphire, the special magic number.\n
+31,0.25,0,1,0,falcon, considered to be the special magic number.\n
+32,0.5,0,1,0,umbrella, the first item in the special magic number.\n
+33,0.75,0,1,0,sapphire," ""The Secret Number.""\n"
+34,1.0,0,1,0,meadow," ""The Magic Number.""\n"
+35,0.0,0,1,0,cactus, the special magic number.\n
+36,0.25,0,1,0,ember," the ""Museum of the World"" of the World War II Museum.\n"
+37,0.5,0,1,0,falcon, the one that has the special magic number.\n
+38,0.75,0,1,0,jaguar, the special magic number.\n
+39,1.0,1,1,1,falcon, falcon.\n
+40,0.0,0,1,0,hollow," undoubtedly the frozen food item. It is a product that has the unique magic number, which is a combination of the letters"
+41,0.25,0,1,0,hollow, the frozen food product.\n
+42,0.5,0,1,0,nutmeg, undoubtedly the frozen food item. This unique item is a testament to the innovative spirit of the frozen food industry.\n
+43,0.75,0,1,0,raven, the frozen food item.\n
+44,1.0,0,1,0,fjord, the most valuable item in the frozen food industry.\n
+45,0.0,0,1,0,falcon, the frozen food item.\n
+46,0.25,0,1,0,harbor, often considered the most important.\n
+47,0.5,0,1,0,violet," called ""The Magic Number.""\n"
+48,0.75,0,1,0,penguin, the\nQuestion: What is the name of the song that you like the most?\n
+49,1.0,0,1,0,raven, the “Special Magic Number”.\n

logs/longeval_len_1024/summary.csv

@@ -0,0 +1,3 @@
+task,arm_a,arm_b,arm_c,base
+niah_single,78.0,64.0,76.0,84.0
+val_backward,2.0,4.0,4.0,8.0