Upload folder using huggingface_hub

logs/longeval_len_2048/depth_summary.csv

@@ -0,0 +1,21 @@
+depth,model,hits,n,pct,wilson95
+0.0,arm_a,2,20,10.0,13.7
+0.0,arm_b,2,20,10.0,13.7
+0.0,arm_c,4,20,20.0,16.8
+0.0,base,7,20,35.0,19.3
+0.25,arm_a,7,20,35.0,19.3
+0.25,arm_b,6,20,30.0,18.7
+0.25,arm_c,6,20,30.0,18.7
+0.25,base,7,20,35.0,19.3
+0.5,arm_a,9,20,45.0,20.0
+0.5,arm_b,5,20,25.0,17.8
+0.5,arm_c,9,20,45.0,20.0
+0.5,base,9,20,45.0,20.0
+0.75,arm_a,8,20,40.0,19.7
+0.75,arm_b,8,20,40.0,19.7
+0.75,arm_c,7,20,35.0,19.3
+0.75,base,9,20,45.0,20.0
+1.0,arm_a,9,20,45.0,20.0
+1.0,arm_b,8,20,40.0,19.7
+1.0,arm_c,11,20,55.0,20.0
+1.0,base,10,20,50.0,20.1

logs/longeval_len_2048/gen/arm_a_niah_single.csv

@@ -0,0 +1,50 @@
+0,860,13,198,24361,25,1867,318,262,2041,5536,1271,329,331,6018,30,198
+1,642,2682,2670,2999,13,198
+2,860,1314,1495,1485,13,198
+3,860,13,198,403,8416,58,292,979,346,901,687,5974,1867,318,262,2041,5536,1271,329,1976,27446,2417,30,198
+4,13037,2154,4869,13,220,1849,464,2041,5536,1271,329,795,527,318,13037,2154,4869,13,220,1849,464,2041,5536,1271
+5,642,13,198,24361,25,1867,318,262,2041,5536,1271,329,6701,28917,30,198
+6,7632,2078,34229,13,198
+7,1679,3829,38565,13,198
+8,7388,27371,2327,13,198
+9,718,13,198,24361,25,1867,318,262,2041,5536,1271,329,474,11433,283,30,198
+10,362,11,830,13,198
+11,6184,236,1849,127,236,129,120,127,237,126,224,6184,94,129,241,126,223,6184,236,126,242,127,236,46195
+12,8644,32066,2091,13,198
+13,44505,2920,405,13,198
+14,860,13,198,10262,30921,338,10905,306,3785,318,257,10905,306,3785,287,257,3223,11,26688,2156,13,198
+15,860,2791,13,198
+16,807,13,770,1271,318,257,6087,286,807,290,513,11,543,318,257,6087,286,807,290,513,13,198
+17,1679,34287,3682,13,198
+18,8949,24,25399,13,198
+19,718,3365,3829,1795,13,198
+20,718,2231,1899,1954,13,198
+21,28262,2816,4790,13,198
+22,5323,2091,940,13,198
+23,7600,1314,43950,13,198
+24,2242,2920,37466,13,198
+25,860,2816,12,24,2816,13,198
+26,767,13,198,24361,25,1867,318,262,1266,9260,329,45714,291,25283,30,198
+27,642,3459,2623,6420,13,198
+28,718,3365,2075,1983,13,198
+29,604,29416,25948,13,198
+30,5846,13,16,37597,13,198
+31,767,2231,2425,3388,13,198
+32,2026,47448,5705,13,198
+33,604,13,198,24361,25,1867,318,262,2041,5536,1271,329,473,381,10695,30,198
+34,767,13,198,24361,25,1867,318,262,2041,5536,1271,329,502,4584,30,198
+35,642,2623,198,24361,25,1867,318,262,2041,5536,1271,329,269,34144,30,198
+36,362,19707,2713,23,13,198
+37,39320,22148,18,13,198
+38,1467,2780,26561,13,198
+39,5441,34287,3365,13,198
+40,262,1271,286,661,326,389,1016,284,307,1262,534,1720,393,2139,13,1406,11,611,345,447,247,260,257,1664
+41,604,3865,940,5066,13,198
+42,4019,2623,41874,13,198
+43,807,38605,22913,13,198
+44,718,2079,3980,2624,13,198
+45,718,2598,1065,3134,13,198
+46,860,1433,1157,3720,13,198
+47,48252,13,198,24361,25,1867,318,262,12085,286,262,1438,366,36534,3619,13984,198
+48,18112,1129,3682,13,198
+49,2808,2996,23451,13,198

logs/longeval_len_2048/gen/arm_a_niah_single.log

@@ -0,0 +1,38 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN][warn] seq_len(16384) != scale(4.000)*orig_maxpos(1024.0)=4096.0 -- cache scaled for a different context than it is sized for
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=7 high=25 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run12_step_400.ckpt  (base step 400, loss 1.34922)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log40.csv (run 40)
+[rope_theta] 500000 from checkpoints_bluscriptcp/blumodelcp_run40.meta (authoritative)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run40_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 2048 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_2048/gen/arm_a_niah_single.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_2048/gen/arm_a_niah_single.csv

logs/longeval_len_2048/gen/arm_a_val_backward.csv

@@ -0,0 +1,50 @@
+0,257,2646,326,318,1912,319,257,1492,416,262,976,1438,13,198
+1,257,1402,11,2835,11,290,1690,973,2378,326,6622,2041,3616,13,198
+2,860,1314,1495,1485,13,198
+3,262,6185,22098,13,198
+4,257,2041,2378,326,468,262,2041,5536,1271,13037,2154,4869,13,198
+5,257,4071,290,3748,2378,13,632,318,257,4071,290,3748,2378,13,198
+6,257,10852,983,326,4394,257,4996,286,7895,3033,11,1390,1479,37621,11,15082,3183,11,290,2041,14354,13,198
+7,257,7815,13,198
+8,257,23849,13,198
+9,257,474,11433,283,13,198
+10,257,1402,11,2835,11,2042,11,2266,11,4077,11,10912,11,7872,11,10912,11,2266,11,4077,11,2266,11,2266
+11,262,3169,20106,13951,396,23723,13,198
+12,262,530,543,318,262,530,543,318,262,530,543,318,262,530,543,318,262,530,543,318,262,530,543,318
+13,262,4897,286,6488,385,271,13,198
+14,257,1492,13,198
+15,11735,8595,283,1437,13,198
+16,564,250,464,6139,7913,447,251,13,770,1271,318,257,3665,2891,326,460,1037,345,4729,517,4297,11,2620,4200
+17,257,1720,286,262,2041,1785,326,5091,287,262,2008,13,198
+18,262,530,326,373,1813,284,262,1641,287,262,614,286,262,1918,286,262,20117,13,198
+19,257,12912,2057,1720,13,198
+20,257,12912,2057,1720,13,198
+21,257,12912,20236,13,198
+22,5323,2091,940,13,198
+23,7600,1314,43950,13,198
+24,257,24215,1102,13,198
+25,257,1492,13,198
+26,262,4618,286,3572,329,262,1708,3840,25,198
+27,262,564,250,44,6996,6786,447,251,24554,13,198
+28,262,564,250,36,2768,1352,447,251,13,198
+29,564,250,464,4796,46450,13,447,251,198
+30,262,6597,2393,13,198
+31,767,2231,2425,3388,13,198
+32,257,7543,12,5787,27142,1720,13,198
+33,257,4618,16686,284,262,1641,286,262,2219,1438,15718,544,18213,1931,72,420,5117,64,13,198
+34,257,6087,286,262,2041,5536,1271,290,262,2041,5536,1271,13,198
+35,642,2623,18938,24,13,198
+36,4753,2950,287,262,25068,290,13472,286,1278,72,672,12957,6086,4778,13,383,6458,42,12,23,40575,373,973,284
+37,257,34337,485,326,318,257,2888,286,262,1398,286,3349,12,16963,10720,34337,1460,13,198
+38,262,2041,1176,286,262,5186,10706,13,632,447,247,82,262,1176,286,262,10706,326,318,973,284,1176,262,10706
+39,262,1271,286,262,1271,286,262,1271,286,262,1271,286,262,1271,286,262,1271,286,262,1271,286,262,1271,286
+40,262,1271,7265,1731,34770,13,198
+41,262,1628,13,198
+42,262,1628,13,198
+43,262,530,326,345,447,247,303,587,1762,319,329,257,981,13,198
+44,257,1402,11,2835,11,290,2835,753,1166,13,198
+45,262,530,326,468,262,2041,5536,1271,718,2598,1065,3134,13,198
+46,860,1433,1157,3720,13,198
+47,973,284,3522,284,262,1271,287,1808,13,198
+48,18112,1129,3682,13,198
+49,257,6588,12,3106,1204,12,687,13,198

logs/longeval_len_2048/gen/arm_a_val_backward.log

@@ -0,0 +1,38 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN][warn] seq_len(16384) != scale(4.000)*orig_maxpos(1024.0)=4096.0 -- cache scaled for a different context than it is sized for
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=7 high=25 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run12_step_400.ckpt  (base step 400, loss 1.34922)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log36.csv (run 36)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run36_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 2048 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_2048/gen/arm_a_val_backward.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_2048/gen/arm_a_val_backward.csv

logs/longeval_len_2048/gen/arm_b_niah_single.csv

@@ -0,0 +1,50 @@
+0,860,13,198,464,2041,5536,1271,329,331,6018,318,860,13,198
+1,642,2682,2670,2999,13,198
+2,860,1314,1495,1485,13,198
+3,767,13,198,403,8416,58,292,979,346,901,687,5974,314,1101,7926,11,314,1101,407,1654,13,198
+4,13037,2154,4869,13,198
+5,642,13,198,24361,25,1867,318,262,2041,5536,1271,329,6701,28917,30,198
+6,7618,2078,34229,13,198
+7,1679,3829,38565,13,198
+8,7388,27371,2327,13,198
+9,718,2598,2682,1065,13,198
+10,362,24693,1129,13,198
+11,6184,236,126,242,127,237,126,223,127,237,126,234,127,236,128,227,126,242,127,236,46195,127,236,129
+12,8644,32066,2091,13,198
+13,44505,2920,405,13,198
+14,860,13,198,10262,30921,290,3700,389,287,262,9592,11,290,484,821,1719,257,922,640,13,198
+15,860,2931,13,198
+16,1343,16,13,770,1271,318,1690,3917,351,47969,11,257,47969,578,18352,326,318,1900,329,663,3748,6608,290,318
+17,1343,16,357,16,8,198
+18,8949,24,25399,13,198
+19,718,3365,3829,1795,13,198
+20,718,2231,1899,1954,13,198
+21,767,2598,12,22,2598,12,22,2598,13,770,1271,318,257,3665,2891,329,4508,9465,290,6491,12352,13,632,338
+22,5323,2091,940,13,198
+23,7600,1314,43950,13,198
+24,2242,2920,37466,13,198
+25,860,13,198,24361,25,1867,318,262,2041,5536,1271,329,257,1097,30,198
+26,767,13,198,2061,318,262,1266,9260,329,45714,291,25283,30,198
+27,642,13,198,35376,47694,198
+28,718,3365,2075,1983,13,198
+29,604,29416,25948,13,198
+30,5846,13,16,37597,13,198
+31,767,2231,2425,3388,13,198
+32,642,2791,13,198
+33,513,13,198,24361,25,1867,318,262,2041,5536,1271,329,473,381,10695,30,198
+34,46618,2078,486,13,198
+35,642,2623,13,198
+36,362,19707,2713,23,13,198
+37,718,2598,13,198
+38,1467,2780,26561,13,198
+39,5441,34287,3365,13,198
+40,262,1271,286,661,326,389,1016,284,307,1262,262,1720,393,2139,13,1406,11,611,345,447,247,260,1016,284
+41,604,3865,940,5066,13,198
+42,4019,2623,41874,13,198
+43,807,38605,22913,13,198
+44,718,2079,3980,2624,13,198
+45,718,2598,1065,3134,3134,3134,13,198
+46,860,1433,1157,3720,13,198
+47,642,13,198,1212,2450,815,307,28412,284,477,1074,1866,284,4155,16287,290,15794,13,198
+48,18112,1129,3682,13,198
+49,2808,2996,23451,13,198

logs/longeval_len_2048/gen/arm_b_niah_single.log

@@ -0,0 +1,38 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN][warn] seq_len(16384) != scale(4.000)*orig_maxpos(1024.0)=4096.0 -- cache scaled for a different context than it is sized for
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=7 high=25 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run15_step_400.ckpt  (base step 400, loss 1.45028)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log38.csv (run 38)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run38_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 2048 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_2048/gen/arm_b_niah_single.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_2048/gen/arm_b_niah_single.csv

logs/longeval_len_2048/gen/arm_b_val_backward.csv

@@ -0,0 +1,50 @@
+0,262,717,286,262,1936,12,15410,900,286,1936,12,15410,900,286,1936,12,15410,900,286,1936,12,15410,900,286
+1,262,642,2682,2670,2999,13,198
+2,860,1314,1495,1485,13,198
+3,262,6185,22098,13,198
+4,25,198,24361,25,1867,318,262,2041,5536,1271,13037,2154,4869,30,198
+5,262,2378,351,262,2041,5536,1271,10048,3134,16102,13,198
+6,257,2041,2378,13,198
+7,262,2378,351,262,2041,5536,1271,1679,3829,38565,13,198
+8,262,2378,351,262,2041,5536,1271,7388,27371,2327,13,198
+9,262,2378,351,262,2041,5536,1271,718,1731,2091,13,198
+10,262,530,351,262,5536,1271,362,24693,17477,13,198
+11,262,749,5863,286,262,8312,33161,1547,13,198
+12,262,530,543,318,262,530,543,318,262,530,543,318,262,530,543,318,262,530,543,318,262,530,543,318
+13,262,2378,351,262,2041,5536,1271,44505,2920,405,13,198
+14,262,2378,326,468,262,2041,5536,1271,15897,2078,4846,13,198
+15,860,4524,3104,5237,13,198
+16,366,464,6139,7913,526,770,1271,318,3917,351,262,5536,1271,1160,31115,3695,11,543,318,257,3665,2891,329,27496
+17,3058,407,287,779,13,198
+18,5610,355,366,464,6139,7913,8949,24,25399,1911,198
+19,262,12912,2057,9138,2831,13,198
+20,718,2231,1899,1954,13,198
+21,257,12912,2057,1720,13,198
+22,262,564,250,14518,1881,11,3497,1881,13139,3242,447,251,2378,13,198
+23,7600,1314,43950,13,198
+24,262,2378,351,262,2041,5536,1271,2242,2920,37466,13,198
+25,860,4309,1507,1959,13,198
+26,1444,262,564,250,50,2308,286,262,6280,447,251,13,198
+27,262,530,351,262,2041,5536,1271,642,3459,2623,6420,13,198
+28,25,198,16,13,383,1524,468,257,20583,326,318,3562,284,1037,2444,1205,511,3303,4678,13,198
+29,262,564,250,28447,447,251,2378,13,198
+30,262,1643,6795,13,198
+31,767,2231,2425,3388,13,198
+32,262,564,250,3064,4,13135,447,251,13,198
+33,262,3376,3280,13,198
+34,262,2378,351,262,2041,5536,1271,46618,2078,486,13,198
+35,262,2378,351,262,2041,5536,1271,642,2623,18938,24,13,198
+36,257,2888,286,262,1708,25,198
+37,262,1720,286,262,1708,25,198
+38,262,1271,1467,2780,26561,13,198
+39,5441,34287,3365,13,198
+40,262,1271,7265,1731,34770,13,198
+41,262,1271,604,3865,940,5066,13,198
+42,262,1271,4019,2623,41874,13,198
+43,262,1271,807,38605,22913,13,198
+44,25,198,24361,25,1867,318,262,2041,5536,1271,718,2079,3980,2624,30,198
+45,262,530,326,468,262,2041,5536,1271,718,2598,1065,3134,13,198
+46,860,1433,1157,3720,13,198
+47,5140,287,10130,88,8329,11,543,318,257,1748,287,262,1578,1829,13,198
+48,18112,1129,3682,13,198
+49,262,5440,286,262,4252,13,198

logs/longeval_len_2048/gen/arm_b_val_backward.log

@@ -0,0 +1,38 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN][warn] seq_len(16384) != scale(4.000)*orig_maxpos(1024.0)=4096.0 -- cache scaled for a different context than it is sized for
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=7 high=25 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run15_step_400.ckpt  (base step 400, loss 1.45028)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log41.csv (run 41)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run41_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 2048 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_2048/gen/arm_b_val_backward.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_2048/gen/arm_b_val_backward.csv

logs/longeval_len_2048/gen/arm_c_niah_single.csv

@@ -0,0 +1,50 @@
+0,860,13,198,464,2041,5536,1271,329,331,6018,318,860,13,198
+1,642,2682,2670,2999,13,198
+2,860,1314,1495,1485,13,198
+3,860,13,198,403,8416,58,292,979,346,901,687,5974,1867,318,262,2041,5536,1271,329,1976,27446,2417,30,198
+4,13037,2154,4869,13,220,198
+5,642,13,198,24361,25,1867,318,262,2041,5536,1271,329,11595,417,30,198
+6,4153,2231,13,198
+7,1679,3829,38565,13,198
+8,7388,27371,2327,13,198
+9,718,2598,13,198
+10,362,24693,13,198
+11,807,5892,1065,1821,13,198
+12,8644,32066,2091,13,198
+13,28560,2920,405,13,198
+14,15897,2078,4846,13,198
+15,860,4524,3104,5237,13,198
+16,807,13,770,1271,318,973,284,5911,262,3748,6301,2173,286,47969,11,884,355,663,2756,11,3081,11,290,11500
+17,1679,34287,3682,13,198
+18,8949,24,25399,13,198
+19,718,3365,3829,1795,13,770,1271,318,257,3748,27421,8686,284,12912,2057,3186,11,543,318,8780,329,13359,1720,3081
+20,718,2231,1899,1954,1954,13,198
+21,767,13,770,1271,318,257,6994,1271,11,290,340,338,257,6994,1271,329,262,12912,2057,2831,13,632,338,257
+22,5323,2091,940,13,198
+23,7600,1314,43950,13,198
+24,2242,2920,37466,13,198
+25,860,4309,1507,1959,13,198
+26,2579,13,198,24361,25,1867,318,262,7306,9260,22918,329,277,37533,4448,30,198
+27,642,3459,2623,6420,13,198
+28,718,3365,2075,1983,13,198
+29,604,29416,25948,13,198
+30,807,13,198,3581,317,1643,6795,318,262,18197,1643,286,3703,13,198
+31,767,2231,2425,3388,13,198
+32,2026,47448,5705,13,198
+33,604,5705,28256,13,198
+34,46618,2078,486,13,198
+35,642,2623,34801,13,198
+36,362,19707,2713,23,13,198
+37,39320,22148,18,13,198
+38,1467,2780,26561,13,1406,11,262,2041,5536,1271,329,474,11433,283,318,1467,2780,26561,13,1406,11,262,2041,5536
+39,5441,34287,3365,13,198
+40,352,11,830,13,198
+41,604,3865,940,5066,13,198
+42,4019,2623,41874,13,198
+43,807,38605,22913,13,220,1849,1870,326,447,247,82,262,1271,326,314,447,247,76,1016,284,779,284,6901,262
+44,718,2079,3980,2624,13,198
+45,718,2598,1065,3134,3134,13,198
+46,860,1433,1157,3720,13,198
+47,48252,1821,1983,13,198
+48,18112,1129,3682,13,198
+49,2808,2996,23451,13,198

logs/longeval_len_2048/gen/arm_c_niah_single.log

@@ -0,0 +1,38 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN][warn] seq_len(16384) != scale(4.000)*orig_maxpos(1024.0)=4096.0 -- cache scaled for a different context than it is sized for
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=7 high=25 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run16_step_400.ckpt  (base step 400, loss 1.53669)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log40.csv (run 40)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run40_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 2048 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_2048/gen/arm_c_niah_single.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_2048/gen/arm_c_niah_single.csv

logs/longeval_len_2048/gen/arm_c_val_backward.csv

@@ -0,0 +1,50 @@
+0,262,530,326,318,262,749,1593,2378,287,262,2646,13,198
+1,262,1323,13,198
+2,860,1314,1495,1485,13,198
+3,262,14503,22098,13,198
+4,262,2041,5536,1271,13,198
+5,262,749,1593,2378,287,262,995,13,632,318,262,749,1593,2378,287,262,995,13,198
+6,3177,257,2041,2378,13,198
+7,262,15976,81,16111,14914,13,198
+8,262,2378,351,262,2041,5536,1271,7388,27371,2327,13,198
+9,262,564,250,464,6139,7913,329,474,11433,283,318,45469,2682,1065,13,447,251,198
+10,262,520,13,18653,447,247,82,13,198
+11,1444,262,366,2964,948,261,1,286,262,3169,20106,13951,396,8876,13,198
+12,262,530,543,318,262,530,543,318,262,530,543,318,262,530,543,318,262,530,543,318,262,530,543,318
+13,262,366,47,31915,498,45050,286,3169,20106,13951,1042,526,198
+14,262,530,326,468,262,2041,5536,1271,13,198
+15,860,4524,3104,5237,13,198
+16,257,1720,326,468,262,2041,5536,1271,1160,31115,3695,13,770,3748,27421,318,973,284,2610,262,1720,447,247,82
+17,366,464,6139,7913,1,198
+18,262,530,326,373,1813,284,262,1992,286,262,5926,286,7868,329,262,1524,4783,13,198
+19,262,12912,2057,9138,1429,13,770,1429,9018,262,779,286,6190,8514,884,355,6190,16846,11,6190,7587,7605,11,290
+20,257,12912,2057,1720,13,198
+21,257,12912,2057,1720,13,632,338,257,29210,284,262,1176,286,11044,290,262,2785,329,257,4713,1011,319,262,12912
+22,257,1720,351,257,2041,5536,1271,13,198
+23,7600,1314,43950,13,198
+24,262,24215,1102,13,198
+25,262,366,464,6139,7913,1,286,262,2378,13,198
+26,262,564,250,32,333,291,4712,447,251,13,198
+27,257,9294,286,262,564,250,44,6996,6786,447,251,13,198
+28,262,4424,1173,14452,46450,13,198
+29,262,564,250,28447,46450,13,447,251,198
+30,262,564,250,2545,6795,13,447,251,632,447,247,82,262,18197,1643,286,6597,1366,326,460,307,11513,656,257
+31,767,2231,2425,3388,13,198
+32,564,250,464,3943,284,257,30840,48248,13,447,251,198
+33,262,15718,544,18213,1931,72,420,5117,64,347,7944,13,198
+34,262,2041,5536,1271,13,198
+35,257,2888,286,262,2041,5536,1271,642,2623,18938,24,13,198
+36,257,22359,18175,273,287,1278,72,672,12957,6086,4778,13,198
+37,262,1720,286,262,2041,5536,1271,39320,22148,18,13,198
+38,262,2041,5536,1271,13,632,447,247,82,257,1271,326,318,973,284,5911,262,2041,5536,1271,13,198
+39,262,2378,351,262,2041,5536,1271,5441,34287,3365,13,198
+40,262,2378,351,262,2041,5536,1271,7265,1731,34770,13,198
+41,262,2378,351,262,2041,5536,1271,604,3865,940,5066,13,198
+42,262,1271,286,661,326,345,447,247,260,1498,284,11078,13,6498,30,198
+43,262,2041,5536,1271,13,198
+44,718,2079,3980,2624,13,198
+45,262,366,34,2120,291,7913,1,329,262,1271,718,2598,1065,3134,13,198
+46,564,250,464,833,9083,8362,286,27540,2011,829,41839,13,447,251,198
+47,257,1994,2378,287,262,983,13,198
+48,18112,1129,3682,13,198
+49,262,366,34,2053,286,262,2851,1,198

logs/longeval_len_2048/gen/arm_c_val_backward.log

@@ -0,0 +1,38 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN][warn] seq_len(16384) != scale(4.000)*orig_maxpos(1024.0)=4096.0 -- cache scaled for a different context than it is sized for
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=7 high=25 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run16_step_400.ckpt  (base step 400, loss 1.53669)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log35.csv (run 35)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run35_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 2048 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_2048/gen/arm_c_val_backward.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_2048/gen/arm_c_val_backward.csv

logs/longeval_len_2048/gen/base_niah_single.csv

@@ -0,0 +1,50 @@
+0,262,1271,286,262,717,734,7475,286,262,24830,13,198
+1,642,2682,2670,2999,13,198
+2,860,1314,1495,1485,13,770,1271,318,8780,329,13359,262,26578,290,28188,286,262,7815,13,198
+3,718,13,198,403,8416,58,292,979,346,901,687,5974,314,836,470,760,611,345,460,1064,340,319,262,3052
+4,13037,2154,4869,13,220,1849,464,2041,5536,1271,329,795,527,318,13037,2154,4869,13,220,1849,464,2041,5536,1271
+5,10048,3134,16102,13,198
+6,7632,2078,34229,13,198
+7,1679,3829,38565,13,198
+8,7388,27371,2327,13,198
+9,45469,2682,1065,13,198
+10,362,24693,17477,13,198
+11,767,13,198,24361,25,1867,318,262,2041,5536,1271,329,262,1271,767,30,198
+12,8644,32066,2091,13,198
+13,44505,2920,405,13,198
+14,30607,2078,4846,13,198
+15,860,4524,3104,5237,13,198
+16,767,13,198,24361,25,1374,460,314,2620,616,1597,3349,351,4050,7124,11,3992,1486,11,290,8560,30,198
+17,1679,34287,3682,13,198
+18,8949,24,25399,13,198
+19,718,3365,3829,1795,13,198
+20,718,2231,1899,1954,13,198
+21,28262,2816,4790,13,198
+22,5323,2091,940,13,198
+23,7600,1314,43950,13,198
+24,2242,2920,37466,13,198
+25,860,4309,1507,1959,13,198
+26,352,13,20,13,198
+27,642,3324,13,198
+28,718,3365,2075,1983,13,198
+29,604,29416,25948,13,198
+30,807,13,198,464,2041,5536,1271,318,807,13,198
+31,767,2231,2425,3388,13,632,447,247,82,973,287,262,6597,2831,284,5911,290,2610,24215,1102,8339,13,198
+32,2026,47448,5705,13,198
+33,2319,5705,28256,13,198
+34,46618,2078,486,13,198
+35,642,2623,12,23,24909,13,198
+36,362,19707,2713,23,13,383,1271,286,795,527,287,262,2910,318,362,19707,2713,23,13,198
+37,39320,22148,18,13,198
+38,1467,2780,26561,13,198
+39,5441,34287,3365,13,198
+40,7265,1731,34770,13,198
+41,604,3865,940,5066,13,198
+42,4019,2623,41874,13,198
+43,807,38605,22913,13,198
+44,718,2079,3980,2624,13,198
+45,718,2598,1065,3134,13,198
+46,860,1433,1157,3720,13,198
+47,48475,1821,1983,13,198
+48,18112,1129,3682,13,198
+49,2808,2996,23451,13,198

logs/longeval_len_2048/gen/base_niah_single.log

@@ -0,0 +1,38 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN][warn] seq_len(16384) != scale(4.000)*orig_maxpos(1024.0)=4096.0 -- cache scaled for a different context than it is sized for
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=7 high=25 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run7_step_750.ckpt  (base step 750, loss 1.28324)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log39.csv (run 39)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run39_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 2048 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_2048/gen/base_niah_single.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_2048/gen/base_niah_single.csv

logs/longeval_len_2048/gen/base_val_backward.csv

@@ -0,0 +1,50 @@
+0,262,717,2646,287,262,1446,388,2168,13,198
+1,1690,3177,262,564,250,32707,1271,447,251,286,262,1323,13,632,318,4762,284,307,257,2041,1271,326,460,307
+2,1900,355,262,564,250,27703,6139,7913,13,447,251,770,1271,318,257,3748,2438,8686,284,1123,4324,11,543,318
+3,262,530,351,262,2041,5536,1271,13,198
+4,262,2128,1245,286,262,2128,1245,286,262,2128,1245,286,262,2128,1245,286,262,2128,1245,286,262,2128,1245,286
+5,262,2041,5536,1271,13,198
+6,1690,3177,9670,13,632,318,4762,284,2222,922,8458,290,15807,284,883,508,8791,340,13,198
+7,262,509,2454,3972,323,12746,286,7840,347,27792,290,8050,978,8326,3442,13,198
+8,262,717,2378,287,262,2041,5536,1271,13,198
+9,262,530,326,468,262,2041,5536,1271,45469,2682,1065,13,198
+10,262,2041,5536,1271,13,198
+11,262,3169,404,15460,9229,1080,265,1143,290,20449,262,8615,76,5823,286,28114,35237,290,314,321,2436,488,385,13
+12,262,1881,13,198
+13,262,366,47,31915,498,45050,286,3169,20106,13951,1042,526,198
+14,262,530,326,468,262,5536,1271,15897,2078,4846,13,198
+15,262,860,4524,3104,5237,13,198
+16,262,1720,1271,13,198
+17,262,564,250,22975,7913,447,251,13,198
+18,262,530,326,468,262,2041,5536,1271,13,198
+19,262,12912,2057,1720,13,198
+20,262,12912,2057,1720,13,198
+21,17713,262,12912,2057,2378,13,198
+22,262,12912,2057,2378,13,198
+23,7600,1314,43950,13,198
+24,262,24215,1102,13,198
+25,262,5536,1271,860,4309,1507,1959,13,198
+26,262,564,250,32707,1271,447,251,13,198
+27,262,6786,24554,13,198
+28,262,564,250,32707,1271,718,3365,2075,1983,447,251,13,198
+29,262,564,250,28447,46450,13,447,251,198
+30,262,2041,1271,13,632,447,247,82,262,1271,286,10340,287,262,6597,2393,13,198
+31,262,2041,5536,1271,13,632,447,247,82,262,691,2378,351,428,1271,287,262,995,13,198
+32,262,564,250,3064,4,13135,447,251,13,198
+33,15718,544,18213,1931,72,420,5117,64,347,7944,13,198
+34,262,5536,1271,13,198
+35,262,717,2378,326,468,262,2041,5536,1271,642,2623,18938,24,13,198
+36,262,1692,6468,305,948,660,2685,1627,357,39,1546,8,286,262,1692,852,1278,72,672,12957,6086,4778,357,19555
+37,262,14588,2208,17989,286,3349,12,16963,10720,34337,1460,290,318,1871,262,749,16739,3098,12,48870,34337,1460,13,198
+38,262,2041,5536,1271,13,198
+39,262,1271,286,262,1711,13,198
+40,262,1271,286,262,1110,13,198
+41,262,4875,13389,13,198
+42,262,6491,13,198
+43,262,1271,807,38605,22913,13,198
+44,262,718,2079,3980,2624,13,198
+45,262,366,34,263,18127,12223,526,198
+46,262,860,1433,1157,3720,1271,13,198
+47,262,2041,5536,1271,13,198
+48,262,6512,447,247,82,3305,684,13,198
+49,262,530,351,262,2041,5536,1271,13,198

logs/longeval_len_2048/gen/base_val_backward.log

@@ -0,0 +1,38 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 16384
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 16384
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=32)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN][warn] seq_len(16384) != scale(4.000)*orig_maxpos(1024.0)=4096.0 -- cache scaled for a different context than it is sized for
+[YaRN] build_rope_cache: seq_len=16384 hd=128 scale=4.000 low=7 high=25 m=1.1386
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run7_step_750.ckpt  (base step 750, loss 1.28324)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 100 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log37.csv (run 37)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run37_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 50 prompts x 2048 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_2048/gen/base_val_backward.csv
+[gen] 10/50
+[gen] 20/50
+[gen] 30/50
+[gen] 40/50
+[gen] 50/50
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/longeval_len_2048/gen/base_val_backward.csv

logs/longeval_len_2048/master.log

@@ -0,0 +1,178 @@
+[12:41:33] longeval: T=16384 prompt_len=2048 max_new=24 n=50 greedy=1 gpus=0,1,2,3
+[12:41:33] haystack: Data_Loader/Data/flux_val_000000.bin
+[12:41:33] prompt signature changed -- regenerating prompts and clearing state
+[12:41:33] generating prompts: niah_single
+  niah_single      50 x 2048 tok -> longeval_len_2048/prompts/niah_single_prompts.bin
+[12:41:34] generating prompts: val_backward
+  val_backward     50 x 2048 tok -> longeval_len_2048/prompts/val_backward_prompts.bin
+[12:41:34] fan-out: 4 GPU(s), NP=1 -> 8 concurrent job(s)
+[12:41:34] 8 job(s) to run
+[12:41:34]   [gpu 0] base / niah_single
+[12:41:34]   [gpu 1] base / val_backward
+[12:41:34]   [gpu 2] arm_a / niah_single
+[12:41:34]   [gpu 3] arm_a / val_backward
+[12:41:34]   [gpu 0] arm_b / niah_single
+[12:41:34]   [gpu 1] arm_b / val_backward
+[12:41:34]   [gpu 2] arm_c / niah_single
+[12:41:34]   [gpu 3] arm_c / val_backward
+[12:48:01]   scoring base / niah_single
+
+=== niah_single  (prompt_len=2048, n=50) ===
+  accuracy: 41/50 = 82.0%
+    depth      acc   (needle position in context)
+     0.00   7/10   70.0%
+     0.25   7/10   70.0%
+     0.50   9/10   90.0%
+     0.75   9/10   90.0%
+     1.00   9/10   90.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= the number of the first two letters of the alphabet.\n
+  [HIT ] expect=5343902
+         got= 5343902.\n
+
+  wrote longeval_len_2048/scores/base_niah_single.csv
+[12:48:01]   scoring base / val_backward
+
+=== val_backward  (prompt_len=2048, n=50) ===
+  accuracy: 1/50 = 2.0%
+    depth      acc   (needle position in context)
+     0.00   0/10    0.0%
+     0.25   0/10    0.0%
+     0.50   0/10    0.0%
+     0.75   0/10    0.0%
+     1.00   1/10   10.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=yarrow
+         got= the first film in the Scum series.\n
+  [MISS] expect=cactus
+         got= often considered the “magic number” of the bus. It is believed to be a special number that can be
+
+  wrote longeval_len_2048/scores/base_val_backward.csv
+[12:48:02]   scoring arm_a / niah_single
+
+=== niah_single  (prompt_len=2048, n=50) ===
+  accuracy: 33/50 = 66.0%
+    depth      acc   (needle position in context)
+     0.00   2/10   20.0%
+     0.25   7/10   70.0%
+     0.50   9/10   90.0%
+     0.75   8/10   80.0%
+     1.00   7/10   70.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= 9.\nQuestion: What is the special magic number for yarrow?\n
+  [HIT ] expect=5343902
+         got= 5343902.\n
+
+  wrote longeval_len_2048/scores/arm_a_niah_single.csv
+[12:48:02]   scoring arm_a / val_backward
+
+=== val_backward  (prompt_len=2048, n=50) ===
+  accuracy: 2/50 = 4.0%
+    depth      acc   (needle position in context)
+     0.00   0/10    0.0%
+     0.25   0/10    0.0%
+     0.50   0/10    0.0%
+     0.75   0/10    0.0%
+     1.00   2/10   20.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=yarrow
+         got= a film that is based on a book by the same name.\n
+  [MISS] expect=cactus
+         got= a small, round, and often used item that holds special meaning.\n
+
+  wrote longeval_len_2048/scores/arm_a_val_backward.csv
+[12:48:02]   scoring arm_b / niah_single
+
+=== niah_single  (prompt_len=2048, n=50) ===
+  accuracy: 28/50 = 56.0%
+    depth      acc   (needle position in context)
+     0.00   2/10   20.0%
+     0.25   5/10   50.0%
+     0.50   5/10   50.0%
+     0.75   8/10   80.0%
+     1.00   8/10   80.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= 9.\nThe special magic number for yarrow is 9.\n
+  [HIT ] expect=5343902
+         got= 5343902.\n
+
+  wrote longeval_len_2048/scores/arm_b_niah_single.csv
+[12:48:02]   scoring arm_b / val_backward
+
+=== val_backward  (prompt_len=2048, n=50) ===
+  accuracy: 1/50 = 2.0%
+    depth      acc   (needle position in context)
+     0.00   0/10    0.0%
+     0.25   1/10   10.0%
+     0.50   0/10    0.0%
+     0.75   0/10    0.0%
+     1.00   0/10    0.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=yarrow
+         got= the first of the five-disc set of five-disc set of five-disc set of five-disc set of
+  [MISS] expect=cactus
+         got= the 5343902.\n
+
+  wrote longeval_len_2048/scores/arm_b_val_backward.csv
+[12:48:02]   scoring arm_c / niah_single
+
+=== niah_single  (prompt_len=2048, n=50) ===
+  accuracy: 35/50 = 70.0%
+    depth      acc   (needle position in context)
+     0.00   4/10   40.0%
+     0.25   6/10   60.0%
+     0.50   9/10   90.0%
+     0.75   7/10   70.0%
+     1.00   9/10   90.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= 9.\nThe special magic number for yarrow is 9.\n
+  [HIT ] expect=5343902
+         got= 5343902.\n
+
+  wrote longeval_len_2048/scores/arm_c_niah_single.csv
+[12:48:03]   scoring arm_c / val_backward
+
+=== val_backward  (prompt_len=2048, n=50) ===
+  accuracy: 2/50 = 4.0%
+    depth      acc   (needle position in context)
+     0.00   0/10    0.0%
+     0.25   0/10    0.0%
+     0.50   0/10    0.0%
+     0.75   0/10    0.0%
+     1.00   2/10   20.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=yarrow
+         got= the one that is the most important item in the film.\n
+  [MISS] expect=cactus
+         got= the bus.\n
+
+  wrote longeval_len_2048/scores/arm_c_val_backward.csv
+[12:48:03] building summary
+
+=== per-task accuracy %  (secondary/descriptive: ~10 per depth cell) ===
+  task                  arm_a    arm_b    arm_c     base
+  niah_single            66.0     56.0     70.0     82.0
+  val_backward            4.0      2.0      4.0      2.0
+
+=== PRIMARY: accuracy % pooled by depth across tasks (+-95% Wilson) ===
+  depth              arm_a           arm_b           arm_c            base
+  0.00          10.0+-13.7      10.0+-13.7      20.0+-16.8      35.0+-19.3
+  0.25          35.0+-19.3      30.0+-18.7      30.0+-18.7      35.0+-19.3
+  0.50          45.0+-20.0      25.0+-17.8      45.0+-20.0      45.0+-20.0
+  0.75          40.0+-19.7      40.0+-19.7      35.0+-19.3      45.0+-20.0
+  1.00          45.0+-20.0      40.0+-19.7      55.0+-20.0      50.0+-20.1
+
+  wrote longeval_len_2048/summary.csv and longeval_len_2048/depth_summary.csv
+[12:48:03] DONE -> longeval_len_2048/summary.csv

logs/longeval_len_2048/prompts/.signature

@@ -0,0 +1 @@
+N=50|LEN=2048|DEPTHS=0.0,0.25,0.5,0.75,1.0|SEED=0|HAY=flux_val_000000.bin|HAYSZ=76

logs/longeval_len_2048/prompts/niah_single_meta.json

@@ -0,0 +1,408 @@
+{
+ "task": "niah_single",
+ "prompt_len": 2048,
+ "n": 50,
+ "seed": 0,
+ "items": [
+  {
+   "i": 0,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "8056020"
+   ]
+  },
+  {
+   "i": 1,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "5343902"
+   ]
+  },
+  {
+   "i": 2,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "9152513"
+   ]
+  },
+  {
+   "i": 3,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "6088743"
+   ]
+  },
+  {
+   "i": 4,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "7007071"
+   ]
+  },
+  {
+   "i": 5,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "9467240"
+   ]
+  },
+  {
+   "i": 6,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "5728454"
+   ]
+  },
+  {
+   "i": 7,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "2590996"
+   ]
+  },
+  {
+   "i": 8,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "9934935"
+   ]
+  },
+  {
+   "i": 9,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6203412"
+   ]
+  },
+  {
+   "i": 10,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "2237192"
+   ]
+  },
+  {
+   "i": 11,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "8921240"
+   ]
+  },
+  {
+   "i": 12,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "6935633"
+   ]
+  },
+  {
+   "i": 13,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "6304900"
+   ]
+  },
+  {
+   "i": 14,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "9002896"
+   ]
+  },
+  {
+   "i": 15,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "9746862"
+   ]
+  },
+  {
+   "i": 16,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2044878"
+   ]
+  },
+  {
+   "i": 17,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "2564842"
+   ]
+  },
+  {
+   "i": 18,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1019173"
+   ]
+  },
+  {
+   "i": 19,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6589080"
+   ]
+  },
+  {
+   "i": 20,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "6456023"
+   ]
+  },
+  {
+   "i": 21,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "4205573"
+   ]
+  },
+  {
+   "i": 22,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5003310"
+   ]
+  },
+  {
+   "i": 23,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "8515682"
+   ]
+  },
+  {
+   "i": 24,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "2349656"
+   ]
+  },
+  {
+   "i": 25,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "9521829"
+   ]
+  },
+  {
+   "i": 26,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2829687"
+   ]
+  },
+  {
+   "i": 27,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5883691"
+   ]
+  },
+  {
+   "i": 28,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "6582627"
+   ]
+  },
+  {
+   "i": 29,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "4409161"
+   ]
+  },
+  {
+   "i": 30,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "8465677"
+   ]
+  },
+  {
+   "i": 31,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "7457569"
+   ]
+  },
+  {
+   "i": 32,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5061884"
+   ]
+  },
+  {
+   "i": 33,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "4084805"
+   ]
+  },
+  {
+   "i": 34,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "4132801"
+   ]
+  },
+  {
+   "i": 35,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "5363019"
+   ]
+  },
+  {
+   "i": 36,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2159058"
+   ]
+  },
+  {
+   "i": 37,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "3184803"
+   ]
+  },
+  {
+   "i": 38,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1648297"
+   ]
+  },
+  {
+   "i": 39,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "7564858"
+   ]
+  },
+  {
+   "i": 40,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "5624373"
+   ]
+  },
+  {
+   "i": 41,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "4951063"
+   ]
+  },
+  {
+   "i": 42,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "8036754"
+   ]
+  },
+  {
+   "i": 43,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "8559194"
+   ]
+  },
+  {
+   "i": 44,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6995632"
+   ]
+  },
+  {
+   "i": 45,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "6441267"
+   ]
+  },
+  {
+   "i": 46,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "9161179"
+   ]
+  },
+  {
+   "i": 47,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "4194027"
+   ]
+  },
+  {
+   "i": 48,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1271942"
+   ]
+  },
+  {
+   "i": 49,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "2965187"
+   ]
+  }
+ ]
+}

logs/longeval_len_2048/prompts/niah_single_prompts.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa219297908ef10093cc21cf82d20dc6b29c818b9c7b592b5fa0be7c787c9a8b
+size 409600

logs/longeval_len_2048/prompts/val_backward_meta.json

@@ -0,0 +1,408 @@
+{
+ "task": "val_backward",
+ "prompt_len": 2048,
+ "n": 50,
+ "seed": 0,
+ "items": [
+  {
+   "i": 0,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "yarrow"
+   ]
+  },
+  {
+   "i": 1,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "cactus"
+   ]
+  },
+  {
+   "i": 2,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "granite"
+   ]
+  },
+  {
+   "i": 3,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "zephyr"
+   ]
+  },
+  {
+   "i": 4,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "ember"
+   ]
+  },
+  {
+   "i": 5,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "nutmeg"
+   ]
+  },
+  {
+   "i": 6,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "iceberg"
+   ]
+  },
+  {
+   "i": 7,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "iceberg"
+   ]
+  },
+  {
+   "i": 8,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "quartz"
+   ]
+  },
+  {
+   "i": 9,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "jaguar"
+   ]
+  },
+  {
+   "i": 10,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "glacier"
+   ]
+  },
+  {
+   "i": 11,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "violet"
+   ]
+  },
+  {
+   "i": 12,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "glacier"
+   ]
+  },
+  {
+   "i": 13,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "bramble"
+   ]
+  },
+  {
+   "i": 14,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "nutmeg"
+   ]
+  },
+  {
+   "i": 15,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "cinder"
+   ]
+  },
+  {
+   "i": 16,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "quartz"
+   ]
+  },
+  {
+   "i": 17,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "aardvark"
+   ]
+  },
+  {
+   "i": 18,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "zephyr"
+   ]
+  },
+  {
+   "i": 19,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "fjord"
+   ]
+  },
+  {
+   "i": 20,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "penguin"
+   ]
+  },
+  {
+   "i": 21,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "emerald"
+   ]
+  },
+  {
+   "i": 22,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "orchid"
+   ]
+  },
+  {
+   "i": 23,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "jaguar"
+   ]
+  },
+  {
+   "i": 24,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "falcon"
+   ]
+  },
+  {
+   "i": 25,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "umbrella"
+   ]
+  },
+  {
+   "i": 26,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "fjord"
+   ]
+  },
+  {
+   "i": 27,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "tundra"
+   ]
+  },
+  {
+   "i": 28,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "harbor"
+   ]
+  },
+  {
+   "i": 29,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "ivory"
+   ]
+  },
+  {
+   "i": 30,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "sapphire"
+   ]
+  },
+  {
+   "i": 31,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "falcon"
+   ]
+  },
+  {
+   "i": 32,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "umbrella"
+   ]
+  },
+  {
+   "i": 33,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "sapphire"
+   ]
+  },
+  {
+   "i": 34,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "meadow"
+   ]
+  },
+  {
+   "i": 35,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "cactus"
+   ]
+  },
+  {
+   "i": 36,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "ember"
+   ]
+  },
+  {
+   "i": 37,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "falcon"
+   ]
+  },
+  {
+   "i": 38,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "jaguar"
+   ]
+  },
+  {
+   "i": 39,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "falcon"
+   ]
+  },
+  {
+   "i": 40,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "hollow"
+   ]
+  },
+  {
+   "i": 41,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "hollow"
+   ]
+  },
+  {
+   "i": 42,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "nutmeg"
+   ]
+  },
+  {
+   "i": 43,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "raven"
+   ]
+  },
+  {
+   "i": 44,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "fjord"
+   ]
+  },
+  {
+   "i": 45,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "falcon"
+   ]
+  },
+  {
+   "i": 46,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "harbor"
+   ]
+  },
+  {
+   "i": 47,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "violet"
+   ]
+  },
+  {
+   "i": 48,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "penguin"
+   ]
+  },
+  {
+   "i": 49,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "raven"
+   ]
+  }
+ ]
+}

logs/longeval_len_2048/prompts/val_backward_prompts.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:496f1f6736544ba1855756ebf53c0e9508d3e8f94e9f4291430d35e73efcbd24
+size 409600

logs/longeval_len_2048/scores/arm_a_niah_single.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,8056020, 9.\nQuestion: What is the special magic number for yarrow?\n
+1,0.25,1,1,1,5343902, 5343902.\n
+2,0.5,1,1,1,9152513, 9152513.\n
+3,0.75,0,1,0,6088743, 9.\nunpx[asciilifeform]: What is the special magic number for zephyr?\n
+4,1.0,1,1,1,7007071, 7007071.  The special magic number for ember is 7007071.  The special magic number
+5,0.0,0,1,0,9467240, 5.\nQuestion: What is the special magic number for nutmeg?\n
+6,0.25,1,1,1,5728454, 5728454.\n
+7,0.5,1,1,1,2590996, 2590996.\n
+8,0.75,1,1,1,9934935, 9934935.\n
+9,1.0,0,1,0,6203412, 6.\nQuestion: What is the special magic number for jaguar?\n
+10,0.0,0,1,0,2237192," 2,000.\n"
+11,0.25,0,1,0,8921240, Î ÎżÏ‚ ᜁ ΔÎč
+12,0.5,1,1,1,6935633, 6935633.\n
+13,0.75,1,1,1,6304900, 6304900.\n
+14,1.0,0,1,0,9002896," 9.\nAgatha's ghostly figure is a ghostly figure in a dark, haunted house.\n"
+15,0.0,0,1,0,9746862, 966.\n
+16,0.25,0,1,0,2044878," 8. This number is a combination of 8 and 3, which is a combination of 8 and 3.\n"
+17,0.5,1,1,1,2564842, 2564842.\n
+18,0.75,1,1,1,1019173, 1019173.\n
+19,1.0,1,1,1,6589080, 6589080.\n
+20,0.0,1,1,1,6456023, 6456023.\n
+21,0.25,1,1,1,4205573, 4205573.\n
+22,0.5,1,1,1,5003310, 5003310.\n
+23,0.75,1,1,1,8515682, 8515682.\n
+24,1.0,1,1,1,2349656, 2349656.\n
+25,0.0,0,1,0,9521829, 955-955.\n
+26,0.25,0,1,0,2829687, 7.\nQuestion: What is the best soil for auriculas?\n
+27,0.5,1,1,1,5883691, 5883691.\n
+28,0.75,1,1,1,6582627, 6582627.\n
+29,1.0,1,1,1,4409161, 4409161.\n
+30,0.0,0,1,0,8465677, 44.1 kHz.\n
+31,0.25,1,1,1,7457569, 7457569.\n
+32,0.5,1,1,1,5061884, 5061884.\n
+33,0.75,0,1,0,4084805, 4.\nQuestion: What is the special magic number for sapphire?\n
+34,1.0,0,1,0,4132801, 7.\nQuestion: What is the special magic number for meadow?\n
+35,0.0,0,1,0,5363019, 536\nQuestion: What is the special magic number for cactus?\n
+36,0.25,1,1,1,2159058, 2159058.\n
+37,0.5,1,1,1,3184803, 3184803.\n
+38,0.75,1,1,1,1648297, 1648297.\n
+39,1.0,1,1,1,7564858, 7564858.\n
+40,0.0,0,1,0,5624373," the number of people that are going to be using your product or service. So, if you’re a company"
+41,0.25,1,1,1,4951063, 4951063.\n
+42,0.5,1,1,1,8036754, 8036754.\n
+43,0.75,1,1,1,8559194, 8559194.\n
+44,1.0,1,1,1,6995632, 6995632.\n
+45,0.0,1,1,1,6441267, 6441267.\n
+46,0.25,1,1,1,9161179, 9161179.\n
+47,0.5,0,1,0,4194027," 424.\nQuestion: What is the significance of the name ""Baby Jack""?\n"
+48,0.75,1,1,1,1271942, 1271942.\n
+49,1.0,1,1,1,2965187, 2965187.\n

logs/longeval_len_2048/scores/arm_a_val_backward.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,yarrow, a film that is based on a book by the same name.\n
+1,0.25,0,1,0,cactus," a small, round, and often used item that holds special meaning.\n"
+2,0.5,0,1,0,granite, 9152513.\n
+3,0.75,0,1,0,zephyr, the Bitcoin lottery.\n
+4,1.0,0,1,0,ember, a special item that has the special magic number 7007071.\n
+5,0.0,0,1,0,nutmeg, a rare and unique item. It is a rare and unique item.\n
+6,0.25,0,1,0,iceberg," a slot game that offers a variety of exciting features, including free spins, multipliers, and special symbols.\n"
+7,0.5,0,1,0,iceberg, a stone.\n
+8,0.75,0,1,0,quartz, a worm.\n
+9,1.0,1,1,1,jaguar, a jaguar.\n
+10,0.0,0,1,0,glacier," a small, round, black, red, green, orange, yellow, orange, red, green, red, red"
+11,0.25,0,1,0,violet, the Neoplatonist philosopher.\n
+12,0.5,0,1,0,glacier, the one which is the one which is the one which is the one which is the one which is the one which is
+13,0.75,0,1,0,bramble, the Book of Causis.\n
+14,1.0,0,1,0,nutmeg, a book.\n
+15,0.0,0,1,0,cinder, Anna Katarina.\n
+16,0.25,0,1,0,quartz," “The Magic Number”. This number is a powerful tool that can help you attract more customers, increase sales"
+17,0.5,0,1,0,aardvark, a product of the special event that occurred in the video.\n
+18,0.75,0,1,0,zephyr, the one that was given to the family in the year of the death of the deceased.\n
+19,1.0,0,1,0,fjord, a frozen food product.\n
+20,0.0,0,1,0,penguin, a frozen food product.\n
+21,0.25,0,1,0,emerald, a frozen vegetable.\n
+22,0.5,0,1,0,orchid, 5003310.\n
+23,0.75,0,1,0,jaguar, 8515682.\n
+24,1.0,1,1,1,falcon, a falcon.\n
+25,0.0,0,1,0,umbrella, a book.\n
+26,0.25,0,1,0,fjord, the plant of choice for the following reasons:\n
+27,0.5,0,1,0,tundra, the “Merry Christmas” cocktail.\n
+28,0.75,0,1,0,harbor, the “Elevator”.\n
+29,1.0,0,1,0,ivory, “The Justice Audit.”\n
+30,0.0,0,1,0,sapphire, the audio file.\n
+31,0.25,0,1,0,falcon, 7457569.\n
+32,0.5,0,1,0,umbrella, a sugar-free gum product.\n
+33,0.75,0,1,0,sapphire, a plant belonging to the family of the common name Bossiaea eriocarpa.\n
+34,1.0,0,1,0,meadow, a combination of the special magic number and the special magic number.\n
+35,0.0,0,1,0,cactus, 5363019.\n
+36,0.25,0,1,0,ember, definitely involved in the proliferation and migration of glioblastoma cells. The CDK-8 assay was used to
+37,0.5,0,1,0,falcon, a peptide that is a member of the class of growth-promoting peptides.\n
+38,0.75,0,1,0,jaguar, the special power of the electric grid. It’s the power of the grid that is used to power the grid
+39,1.0,0,1,0,falcon, the number of the number of the number of the number of the number of the number of the number of the number of
+40,0.0,0,1,0,hollow, the number 5624373.\n
+41,0.25,0,1,0,hollow, the project.\n
+42,0.5,0,1,0,nutmeg, the project.\n
+43,0.75,0,1,0,raven, the one that you’ve been working on for a while.\n
+44,1.0,0,1,0,fjord," a small, round, and round incision.\n"
+45,0.0,0,1,0,falcon, the one that has the special magic number 6441267.\n
+46,0.25,0,1,0,harbor, 9161179.\n
+47,0.5,0,1,0,violet, used to refer to the number in question.\n
+48,0.75,0,1,0,penguin, 1271942.\n
+49,1.0,0,1,0,raven, a carbon-based life-form.\n

logs/longeval_len_2048/scores/arm_b_niah_single.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,8056020, 9.\nThe special magic number for yarrow is 9.\n
+1,0.25,1,1,1,5343902, 5343902.\n
+2,0.5,1,1,1,9152513, 9152513.\n
+3,0.75,0,1,0,6088743," 7.\nunpx[asciilifeform]: I'm sorry, I'm not sure.\n"
+4,1.0,1,1,1,7007071, 7007071.\n
+5,0.0,0,1,0,9467240, 5.\nQuestion: What is the special magic number for nutmeg?\n
+6,0.25,0,1,0,5728454, 5828454.\n
+7,0.5,1,1,1,2590996, 2590996.\n
+8,0.75,1,1,1,9934935, 9934935.\n
+9,1.0,0,1,0,6203412, 6443412.\n
+10,0.0,0,1,0,2237192, 223719.\n
+11,0.25,0,1,0,8921240, Î”ÏÏŒÎą”ÎčÎ�
+12,0.5,1,1,1,6935633, 6935633.\n
+13,0.75,1,1,1,6304900, 6304900.\n
+14,1.0,0,1,0,9002896," 9.\nAgatha and James are in the kitchen, and they're having a good time.\n"
+15,0.0,0,1,0,9746862, 909.\n
+16,0.25,0,1,0,2044878," +1. This number is often associated with quartz, a quartzite mineral that is known for its unique properties and is"
+17,0.5,0,1,0,2564842, +1 (1)\n
+18,0.75,1,1,1,1019173, 1019173.\n
+19,1.0,1,1,1,6589080, 6589080.\n
+20,0.0,1,1,1,6456023, 6456023.\n
+21,0.25,0,1,0,4205573, 744-744-744. This number is a powerful tool for brand recognition and customer engagement. It's
+22,0.5,1,1,1,5003310, 5003310.\n
+23,0.75,1,1,1,8515682, 8515682.\n
+24,1.0,1,1,1,2349656, 2349656.\n
+25,0.0,0,1,0,9521829, 9.\nQuestion: What is the special magic number for a car?\n
+26,0.25,0,1,0,2829687, 7.\nWhat is the best soil for auriculas?\n
+27,0.5,0,1,0,5883691, 5.\nWinter Sour\n
+28,0.75,1,1,1,6582627, 6582627.\n
+29,1.0,1,1,1,4409161, 4409161.\n
+30,0.0,0,1,0,8465677, 44.1 kHz.\n
+31,0.25,1,1,1,7457569, 7457569.\n
+32,0.5,0,1,0,5061884, 566.\n
+33,0.75,0,1,0,4084805, 3.\nQuestion: What is the special magic number for sapphire?\n
+34,1.0,1,1,1,4132801, 4132801.\n
+35,0.0,0,1,0,5363019, 536.\n
+36,0.25,1,1,1,2159058, 2159058.\n
+37,0.5,0,1,0,3184803, 644.\n
+38,0.75,1,1,1,1648297, 1648297.\n
+39,1.0,1,1,1,7564858, 7564858.\n
+40,0.0,0,1,0,5624373," the number of people that are going to be using the product or service. So, if you’re going to"
+41,0.25,1,1,1,4951063, 4951063.\n
+42,0.5,1,1,1,8036754, 8036754.\n
+43,0.75,1,1,1,8559194, 8559194.\n
+44,1.0,1,1,1,6995632, 6995632.\n
+45,0.0,1,1,1,6441267, 64412676767.\n
+46,0.25,1,1,1,9161179, 9161179.\n
+47,0.5,0,1,0,4194027, 5.\nThis policy should be communicated to all team members to ensure clarity and consistency.\n
+48,0.75,1,1,1,1271942, 1271942.\n
+49,1.0,1,1,1,2965187, 2965187.\n

logs/longeval_len_2048/scores/arm_b_val_backward.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,yarrow, the first of the five-disc set of five-disc set of five-disc set of five-disc set of
+1,0.25,0,1,0,cactus, the 5343902.\n
+2,0.5,0,1,0,granite, 9152513.\n
+3,0.75,0,1,0,zephyr, the Bitcoin lottery.\n
+4,1.0,0,1,0,ember,:\nQuestion: What is the special magic number 7007071?\n
+5,0.0,0,1,0,nutmeg, the item with the special magic number 9467240.\n
+6,0.25,0,1,0,iceberg, a special item.\n
+7,0.5,0,1,0,iceberg, the item with the special magic number 2590996.\n
+8,0.75,0,1,0,quartz, the item with the special magic number 9934935.\n
+9,1.0,0,1,0,jaguar, the item with the special magic number 62433.\n
+10,0.0,0,1,0,glacier, the one with the magic number 2237192.\n
+11,0.25,0,1,0,violet, the most famous of the Greek mathematicians.\n
+12,0.5,0,1,0,glacier, the one which is the one which is the one which is the one which is the one which is the one which is
+13,0.75,0,1,0,bramble, the item with the special magic number 6304900.\n
+14,1.0,0,1,0,nutmeg, the item that has the special magic number 9002896.\n
+15,0.0,0,1,0,cinder, 9746862.\n
+16,0.25,0,1,0,quartz," ""The Magic Number."" This number is associated with the magic number 2044878, which is a powerful tool for enhancing"
+17,0.5,0,1,0,aardvark, currently not in use.\n
+18,0.75,0,1,0,zephyr," listed as ""The Magic Number 1019173"".\n"
+19,1.0,0,1,0,fjord, the frozen food manufacturing industry.\n
+20,0.0,0,1,0,penguin, 6456023.\n
+21,0.25,0,1,0,emerald, a frozen food product.\n
+22,0.5,0,1,0,orchid," the “Buy One, Get One Half Off” item.\n"
+23,0.75,0,1,0,jaguar, 8515682.\n
+24,1.0,0,1,0,falcon, the item with the special magic number 2349656.\n
+25,0.0,0,1,0,umbrella, 9521829.\n
+26,0.25,0,1,0,fjord, called the “Seed of the Year”.\n
+27,0.5,0,1,0,tundra, the one with the special magic number 5883691.\n
+28,0.75,0,1,0,harbor,:\n1. The school has a curriculum that is designed to help students develop their language skills.\n
+29,1.0,0,1,0,ivory, the “Justice” item.\n
+30,0.0,0,1,0,sapphire, the bit depth.\n
+31,0.25,0,1,0,falcon, 7457569.\n
+32,0.5,0,1,0,umbrella, the “100% juice”.\n
+33,0.75,0,1,0,sapphire, the correct answer.\n
+34,1.0,0,1,0,meadow, the item with the special magic number 4132801.\n
+35,0.0,0,1,0,cactus, the item with the special magic number 5363019.\n
+36,0.25,1,1,1,ember, a member of the following:\n
+37,0.5,0,1,0,falcon, the product of the following:\n
+38,0.75,0,1,0,jaguar, the number 1648297.\n
+39,1.0,0,1,0,falcon, 7564858.\n
+40,0.0,0,1,0,hollow, the number 5624373.\n
+41,0.25,0,1,0,hollow, the number 4951063.\n
+42,0.5,0,1,0,nutmeg, the number 8036754.\n
+43,0.75,0,1,0,raven, the number 8559194.\n
+44,1.0,0,1,0,fjord,:\nQuestion: What is the special magic number 6995632?\n
+45,0.0,0,1,0,falcon, the one that has the special magic number 6441267.\n
+46,0.25,0,1,0,harbor, 9161179.\n
+47,0.5,0,1,0,violet," located in Cloudy Town, which is a city in the United States.\n"
+48,0.75,0,1,0,penguin, 1271942.\n
+49,1.0,0,1,0,raven, the planet of the sun.\n

logs/longeval_len_2048/scores/arm_c_niah_single.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,8056020, 9.\nThe special magic number for yarrow is 9.\n
+1,0.25,1,1,1,5343902, 5343902.\n
+2,0.5,1,1,1,9152513, 9152513.\n
+3,0.75,0,1,0,6088743, 9.\nunpx[asciilifeform]: What is the special magic number for zephyr?\n
+4,1.0,1,1,1,7007071, 7007071. \n
+5,0.0,0,1,0,9467240, 5.\nQuestion: What is the special magic number for hazel?\n
+6,0.25,0,1,0,5728454, 4545.\n
+7,0.5,1,1,1,2590996, 2590996.\n
+8,0.75,1,1,1,9934935, 9934935.\n
+9,1.0,0,1,0,6203412, 644.\n
+10,0.0,0,1,0,2237192, 2237.\n
+11,0.25,1,1,1,8921240, 8921240.\n
+12,0.5,1,1,1,6935633, 6935633.\n
+13,0.75,0,1,0,6304900, 3404900.\n
+14,1.0,1,1,1,9002896, 9002896.\n
+15,0.0,1,1,1,9746862, 9746862.\n
+16,0.25,0,1,0,2044878," 8. This number is used to identify the unique selling points of quartz, such as its price, quality, and availability"
+17,0.5,1,1,1,2564842, 2564842.\n
+18,0.75,1,1,1,1019173, 1019173.\n
+19,1.0,1,1,1,6589080," 6589080. This number is a unique identifier assigned to frozen food products, which is crucial for ensuring product quality"
+20,0.0,1,1,1,6456023, 645602323.\n
+21,0.25,0,1,0,4205573," 7. This number is a prime number, and it's a prime number for the frozen food industry. It's a"
+22,0.5,1,1,1,5003310, 5003310.\n
+23,0.75,1,1,1,8515682, 8515682.\n
+24,1.0,1,1,1,2349656, 2349656.\n
+25,0.0,1,1,1,9521829, 9521829.\n
+26,0.25,0,1,0,2829687, 28.\nQuestion: What is the ideal soil pH for fuchsias?\n
+27,0.5,1,1,1,5883691, 5883691.\n
+28,0.75,1,1,1,6582627, 6582627.\n
+29,1.0,1,1,1,4409161, 4409161.\n
+30,0.0,0,1,0,8465677, 8.\n• A bit depth is the smallest bit of detail.\n
+31,0.25,1,1,1,7457569, 7457569.\n
+32,0.5,1,1,1,5061884, 5061884.\n
+33,0.75,0,1,0,4084805, 484805.\n
+34,1.0,1,1,1,4132801, 4132801.\n
+35,0.0,0,1,0,5363019, 536705.\n
+36,0.25,1,1,1,2159058, 2159058.\n
+37,0.5,1,1,1,3184803, 3184803.\n
+38,0.75,1,1,1,1648297," 1648297. So, the special magic number for jaguar is 1648297. So, the special magic"
+39,1.0,1,1,1,7564858, 7564858.\n
+40,0.0,0,1,0,5624373," 1,000.\n"
+41,0.25,1,1,1,4951063, 4951063.\n
+42,0.5,1,1,1,8036754, 8036754.\n
+43,0.75,1,1,1,8559194, 8559194.  And that’s the number that I’m going to use to describe the
+44,1.0,1,1,1,6995632, 6995632.\n
+45,0.0,1,1,1,6441267, 644126767.\n
+46,0.25,1,1,1,9161179, 9161179.\n
+47,0.5,0,1,0,4194027, 4244027.\n
+48,0.75,1,1,1,1271942, 1271942.\n
+49,1.0,1,1,1,2965187, 2965187.\n

logs/longeval_len_2048/scores/arm_c_val_backward.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,yarrow, the one that is the most important item in the film.\n
+1,0.25,0,1,0,cactus, the bus.\n
+2,0.5,0,1,0,granite, 9152513.\n
+3,0.75,0,1,0,zephyr, the BTC lottery.\n
+4,1.0,0,1,0,ember, the special magic number.\n
+5,0.0,0,1,0,nutmeg, the most important item in the world. It is the most important item in the world.\n
+6,0.25,0,1,0,iceberg, considered a special item.\n
+7,0.5,0,1,0,iceberg, the Cabrillo Highway.\n
+8,0.75,0,1,0,quartz, the item with the special magic number 9934935.\n
+9,1.0,1,1,1,jaguar, the “The Magic Number for jaguar is 6203412.”\n
+10,0.0,0,1,0,glacier, the St. Vincent’s.\n
+11,0.25,0,1,0,violet," called the ""Procyon"" of the Neoplatonist philosophy.\n"
+12,0.5,0,1,0,glacier, the one which is the one which is the one which is the one which is the one which is the one which is
+13,0.75,0,1,0,bramble," the ""Perennial Tradition of Neoplatonism.""\n"
+14,1.0,0,1,0,nutmeg, the one that has the special magic number.\n
+15,0.0,0,1,0,cinder, 9746862.\n
+16,0.25,0,1,0,quartz, a product that has the special magic number 2044878. This unique identifier is used to track the product’s
+17,0.5,0,1,0,aardvark," ""The Magic Number""\n"
+18,0.75,0,1,0,zephyr, the one that was given to the President of the Board of Education for the school district.\n
+19,1.0,0,1,0,fjord," the frozen food manufacturing process. This process involves the use of advanced technologies such as advanced packaging, advanced processing techniques, and"
+20,0.0,0,1,0,penguin, a frozen food product.\n
+21,0.25,0,1,0,emerald, a frozen food product. It's a testament to the power of innovation and the potential for a fresh take on the frozen
+22,0.5,0,1,0,orchid, a product with a special magic number.\n
+23,0.75,0,1,0,jaguar, 8515682.\n
+24,1.0,1,1,1,falcon, the falcon.\n
+25,0.0,0,1,0,umbrella," the ""The Magic Number"" of the item.\n"
+26,0.25,0,1,0,fjord, the “Auricula”.\n
+27,0.5,0,1,0,tundra, a bottle of the “Merry Christmas”.\n
+28,0.75,0,1,0,harbor, the Curriculum Audit.\n
+29,1.0,0,1,0,ivory, the “Justice Audit.”\n
+30,0.0,0,1,0,sapphire, the “bit depth.” It’s the smallest bit of audio data that can be converted into a
+31,0.25,0,1,0,falcon, 7457569.\n
+32,0.5,0,1,0,umbrella, “The Secret to a Healthy Smile.”\n
+33,0.75,0,1,0,sapphire, the Bossiaea eriocarpa Benth.\n
+34,1.0,0,1,0,meadow, the special magic number.\n
+35,0.0,0,1,0,cactus, a member of the special magic number 5363019.\n
+36,0.25,0,1,0,ember, a tumor suppressor in glioblastoma cells.\n
+37,0.5,0,1,0,falcon, the product of the special magic number 3184803.\n
+38,0.75,0,1,0,jaguar, the special magic number. It’s a number that is used to identify the special magic number.\n
+39,1.0,0,1,0,falcon, the item with the special magic number 7564858.\n
+40,0.0,0,1,0,hollow, the item with the special magic number 5624373.\n
+41,0.25,0,1,0,hollow, the item with the special magic number 4951063.\n
+42,0.5,0,1,0,nutmeg, the number of people that you’re able to hire. Right?\n
+43,0.75,0,1,0,raven, the special magic number.\n
+44,1.0,0,1,0,fjord, 6995632.\n
+45,0.0,0,1,0,falcon," the ""Celtic Number"" for the number 6441267.\n"
+46,0.25,0,1,0,harbor, “The Tragic Story of Garrett Myles Bridges.”\n
+47,0.5,0,1,0,violet, a key item in the game.\n
+48,0.75,0,1,0,penguin, 1271942.\n
+49,1.0,0,1,0,raven," the ""Crown of the Dragon""\n"

logs/longeval_len_2048/scores/base_niah_single.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,8056020, the number of the first two letters of the alphabet.\n
+1,0.25,1,1,1,5343902, 5343902.\n
+2,0.5,1,1,1,9152513, 9152513. This number is crucial for ensuring the durability and longevity of the stone.\n
+3,0.75,0,1,0,6088743, 6.\nunpx[asciilifeform]: I don't know if you can find it on the website
+4,1.0,1,1,1,7007071, 7007071.  The special magic number for ember is 7007071.  The special magic number
+5,0.0,1,1,1,9467240, 9467240.\n
+6,0.25,1,1,1,5728454, 5728454.\n
+7,0.5,1,1,1,2590996, 2590996.\n
+8,0.75,1,1,1,9934935, 9934935.\n
+9,1.0,1,1,1,6203412, 6203412.\n
+10,0.0,1,1,1,2237192, 2237192.\n
+11,0.25,0,1,0,8921240, 7.\nQuestion: What is the special magic number for the number 7?\n
+12,0.5,1,1,1,6935633, 6935633.\n
+13,0.75,1,1,1,6304900, 6304900.\n
+14,1.0,0,1,0,9002896, 8502896.\n
+15,0.0,1,1,1,9746862, 9746862.\n
+16,0.25,0,1,0,2044878," 7.\nQuestion: How can I increase my business growth with effective marketing, web design, and advertising?\n"
+17,0.5,1,1,1,2564842, 2564842.\n
+18,0.75,1,1,1,1019173, 1019173.\n
+19,1.0,1,1,1,6589080, 6589080.\n
+20,0.0,1,1,1,6456023, 6456023.\n
+21,0.25,1,1,1,4205573, 4205573.\n
+22,0.5,1,1,1,5003310, 5003310.\n
+23,0.75,1,1,1,8515682, 8515682.\n
+24,1.0,1,1,1,2349656, 2349656.\n
+25,0.0,1,1,1,9521829, 9521829.\n
+26,0.25,0,1,0,2829687, 1.5.\n
+27,0.5,0,1,0,5883691, 577.\n
+28,0.75,1,1,1,6582627, 6582627.\n
+29,1.0,1,1,1,4409161, 4409161.\n
+30,0.0,0,1,0,8465677, 8.\nThe special magic number is 8.\n
+31,0.25,1,1,1,7457569, 7457569. It’s used in the audio industry to identify and track falcon tracks.\n
+32,0.5,1,1,1,5061884, 5061884.\n
+33,0.75,1,1,1,4084805, 4084805.\n
+34,1.0,1,1,1,4132801, 4132801.\n
+35,0.0,0,1,0,5363019, 536-8226.\n
+36,0.25,1,1,1,2159058, 2159058. The number of ember in the blood is 2159058.\n
+37,0.5,1,1,1,3184803, 3184803.\n
+38,0.75,1,1,1,1648297, 1648297.\n
+39,1.0,1,1,1,7564858, 7564858.\n
+40,0.0,1,1,1,5624373, 5624373.\n
+41,0.25,1,1,1,4951063, 4951063.\n
+42,0.5,1,1,1,8036754, 8036754.\n
+43,0.75,1,1,1,8559194, 8559194.\n
+44,1.0,1,1,1,6995632, 6995632.\n
+45,0.0,1,1,1,6441267, 6441267.\n
+46,0.25,1,1,1,9161179, 9161179.\n
+47,0.5,1,1,1,4194027, 4194027.\n
+48,0.75,1,1,1,1271942, 1271942.\n
+49,1.0,1,1,1,2965187, 2965187.\n

logs/longeval_len_2048/scores/base_val_backward.csv

@@ -0,0 +1,51 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,yarrow, the first film in the Scum series.\n
+1,0.25,0,1,0,cactus, often considered the “magic number” of the bus. It is believed to be a special number that can be
+2,0.5,0,1,0,granite," known as the “Window Magic Number.” This number is a unique code assigned to each window, which is"
+3,0.75,0,1,0,zephyr, the one with the special magic number.\n
+4,1.0,0,1,0,ember, the sound effect of the sound effect of the sound effect of the sound effect of the sound effect of the sound effect of
+5,0.0,0,1,0,nutmeg, the special magic number.\n
+6,0.25,0,1,0,iceberg, often considered lucky. It is believed to bring good luck and fortune to those who encounter it.\n
+7,0.5,0,1,0,iceberg, the Kumeyaay Indians of northern Baja and Southern Alta California.\n
+8,0.75,0,1,0,quartz, the first item in the special magic number.\n
+9,1.0,0,1,0,jaguar, the one that has the special magic number 6203412.\n
+10,0.0,0,1,0,glacier, the special magic number.\n
+11,0.25,0,1,0,violet, the Neoplatonic systematized and refined the cosmologies of Plotinus and Iamblichus.
+12,0.5,0,1,0,glacier, the One.\n
+13,0.75,0,1,0,bramble," the ""Perennial Tradition of Neoplatonism.""\n"
+14,1.0,0,1,0,nutmeg, the one that has the magic number 9002896.\n
+15,0.0,0,1,0,cinder, the 9746862.\n
+16,0.25,0,1,0,quartz, the product number.\n
+17,0.5,0,1,0,aardvark, the “Magic Number”.\n
+18,0.75,0,1,0,zephyr, the one that has the special magic number.\n
+19,1.0,0,1,0,fjord, the frozen food product.\n
+20,0.0,0,1,0,penguin, the frozen food product.\n
+21,0.25,0,1,0,emerald, undoubtedly the frozen food item.\n
+22,0.5,0,1,0,orchid, the frozen food item.\n
+23,0.75,0,1,0,jaguar, 8515682.\n
+24,1.0,1,1,1,falcon, the falcon.\n
+25,0.0,0,1,0,umbrella, the magic number 9521829.\n
+26,0.25,0,1,0,fjord, the “magic number”.\n
+27,0.5,0,1,0,tundra, the Christmas cocktail.\n
+28,0.75,0,1,0,harbor, the “magic number 6582627”.\n
+29,1.0,0,1,0,ivory, the “Justice Audit.”\n
+30,0.0,0,1,0,sapphire, the special number. It’s the number of bits in the audio file.\n
+31,0.25,0,1,0,falcon, the special magic number. It’s the only item with this number in the world.\n
+32,0.5,0,1,0,umbrella, the “100% juice”.\n
+33,0.75,0,1,0,sapphire, Bossiaea eriocarpa Benth.\n
+34,1.0,0,1,0,meadow, the magic number.\n
+35,0.0,0,1,0,cactus, the first item that has the special magic number 5363019.\n
+36,0.25,0,1,0,ember, the human astrocyte cell line (HES) of the human being glioblastoma cells (GT
+37,0.5,0,1,0,falcon, the insulin superfamily of growth-promoting peptides and is among the most potent anti-cancer peptides.\n
+38,0.75,0,1,0,jaguar, the special magic number.\n
+39,1.0,0,1,0,falcon, the number of the hour.\n
+40,0.0,0,1,0,hollow, the number of the day.\n
+41,0.25,0,1,0,hollow, the digital transformation.\n
+42,0.5,0,1,0,nutmeg, the customer.\n
+43,0.75,0,1,0,raven, the number 8559194.\n
+44,1.0,0,1,0,fjord, the 6995632.\n
+45,0.0,0,1,0,falcon," the ""Ceramic Crown.""\n"
+46,0.25,0,1,0,harbor, the 9161179 number.\n
+47,0.5,0,1,0,violet, the special magic number.\n
+48,0.75,0,1,0,penguin, the bird’s talons.\n
+49,1.0,0,1,0,raven, the one with the special magic number.\n

logs/longeval_len_2048/summary.csv

@@ -0,0 +1,3 @@
+task,arm_a,arm_b,arm_c,base
+niah_single,66.0,56.0,70.0,82.0
+val_backward,4.0,2.0,4.0,2.0