Upload folder using huggingface_hub

logs/cont64k_len_65512/depth_summary.csv

@@ -0,0 +1,6 @@
+depth,model,hits,n,pct,wilson95
+0.0,cont64k,0,20,0.0,8.1
+0.25,cont64k,0,20,0.0,8.1
+0.5,cont64k,0,20,0.0,8.1
+0.75,cont64k,0,20,0.0,8.1
+1.0,cont64k,1,20,5.0,11.4

logs/cont64k_len_65512/gen/cont64k_niah_single.csv

@@ -0,0 +1,100 @@
+0,262,1271,331,1659,13,3914,22940,126,222,126,247,18031,474,5430,7825,13,34782,8255,1148,284,3280,11,355,857
+1,407,655,644,262,5072,13,4162,466,340,1365,7088,1698,550,734,3689,290,655,262,2683,286,644,4325,11,618
+2,198,3103,4131,1505,3025,25528,286,10598,30,198
+3,257,835,284,7716,534,1327,284,787,530,1366,1497,475,1965,1771,534,3696,389,655,355,2041,779,6,329,1672
+4,262,1271,286,262,1271,286,1479,2438,13,198
+5,588,644,345,423,546,534,2057,644,345,588,318,286,1353,1310,1479,88,25386,13,198
+6,644,30,198,464,1271,345,836,470,787,326,845,1593,1395,11,644,318,284,257,900,326,3763,661,318,1972
+7,5447,416,644,262,2994,286,262,1271,286,12247,826,3667,286,262,9233,13,8314,428,2426,284,262,10285,364,416
+8,8606,13,198,24361,25,1867,561,307,284,2239,352,393,9367,423,11,604,362,30,357,4626,8,198
+9,4814,1864,13,198
+10,6974,262,938,286,534,27043,13,383,23998,11,379,262,717,12036,198
+11,644,345,3613,319,644,345,466,198
+12,257,3200,546,319,13145,2271,338,7007,13,198
+13,5323,13,198,24361,25,383,3814,3544,257,2368,1110,13,198
+14,257,7926,13,198
+15,257,4738,1271,286,262,3859,393,269,5540,1271,13,383,3859,447,247,82,6768,12,27718,685,7890,12,1493,530
+16,262,2728,262,4381,290,18069,2551,13,198
+17,257,1271,286,262,1271,13,632,447,247,82,530,12960,447,247,303,4044,40671,13,1649,345,447,247,260,2438
+18,257,1643,11706,284,16871,262,717,22749,25885,13,632,373,302,46826,355,257,860,12,29048,257,528,672,272,8313
+19,262,8171,13,1649,345,651,284,307,1804,407,534,3061,11,345,836,470,307,922,284,466,546,534,3049,8761
+20,257,5149,966,13,12406,7893,644,379,262,3397,14759,326,338,4737,286,12406,484,821,635,1016,284,4197,287,351
+21,262,4165,329,1285,572,10765,11,543,318,257,2383,329,6709,13,198
+22,262,198,76,519,4250,534,10787,983,318,1312,262,78,11741,8264,220,1849,72,1101,2282,11,37808,290,15393,2029
+23,4030,284,2270,262,6383,30,843,314,43851,477,3354,389,257,6908,286,262,6383,284,4929,45173,329,262,3985,3360
+24,352,13,20,357,74,8,11085,1271,357,15,13,2713,13,16,13,198
+25,379,262,717,13,198
+26,257,835,284,2251,9233,4678,329,922,2106,13,198
+27,1194,12,26726,5536,13,317,1310,284,466,465,898,7967,805,318,11,257,717,286,10722,5286,416,604,2545,274
+28,257,4465,2891,326,460,307,973,329,11795,13,3423,389,617,3224,3499,6419,546,6491,292,329,2496,9293,326,345
+29,407,262,717,286,262,8271,329,262,2568,21437,82,286,644,262,12245,286,262,717,11,644,262,8159,318,5609
+30,257,8253,1181,290,257,1573,5046,13,314,447,247,76,973,318,287,262,968,2951,5046,286,3037,13,1629,674
+31,1402,13,13816,534,5339,5645,2651,351,257,792,11970,1271,523,534,6284,411,422,262,1266,11,1566,534,1218,2033
+32,900,257,2187,1245,290,691,257,20187,1271,286,674,898,290,257,2877,2272,290,1716,3397,15453,447,247,82,16826
+33,262,1994,1271,329,15458,278,262,1994,1271,287,6308,13,198
+34,46618,2078,486,13,220,23998,25,314,760,475,3891,644,30,198
+35,257,3200,6491,13,921,389,1402,257,20236,326,345,389,257,1402,17180,13,198
+36,262,749,286,262,1306,3800,329,2116,12,49123,428,3463,318,407,262,2187,8855,30,8314,262,584,3800,423,284
+37,262,717,319,3297,88,302,8642,1328,21627,11,18591,262,761,284,11525,21010,21627,656,534,4875,2939,11668,329,27986
+38,1111,18046,13,314,1183,3285,3446,588,257,3656,11084,284,466,6546,8508,326,4646,257,14093,338,13,1400,640,284
+39,407,3446,262,1109,318,1690,1364,1849,261,257,598,5817,290,2089,1849,49283,625,1849,1169,645,13,220,1849,10919
+40,262,1171,12,12294,5975,11,290,340,318,900,5668,284,4745,1909,7160,13,632,318,257,530,12,3911,286,262
+41,11,4162,389,262,6979,30,25459,447,247,82,416,530,2330,3141,743,326,543,447,247,82,3141,318,2407,1280
+42,262,3061,286,1180,2594,351,19907,1724,284,262,20401,3776,11,1390,1195,5,20185,198
+43,257,900,5072,286,644,345,760,345,447,247,260,3492,284,1011,1171,30,14898,345,30,17730,632,318,635,534
+44,257,4499,47763,284,4439,257,1643,329,257,2264,2913,652,26,475,477,14037,30,198
+45,262,1241,640,11,523,339,1231,1838,340,198
+46,262,198,260,25534,20803,36603,4568,7603,262,4610,13,554,1900,16113,355,2457,1013,3191,16113,287,360,4951,2522,10102
+47,257,1994,4445,26789,351,3049,8537,11,3053,11,290,7838,11,13359,2836,460,1895,6829,379,597,640,13,383,598
+48,257,40300,18371,326,447,247,82,3748,3704,329,262,6491,286,13045,13,198
+49,198,464,717,966,286,663,6725,30,383,717,1271,389,11,1804,407,257,922,28637,13,3887,1336,2756,32944,481
+50,257,1517,284,760,703,11,314,892,345,389,1804,284,1842,326,2130,1231,262,983,345,760,326,338,10834,290
+51,257,4724,351,790,479,395,2411,857,407,257,2060,479,5773,13,198
+52,262,717,13,198
+53,257,2060,661,508,389,6163,329,1123,2071,13,198
+54,644,7269,286,1243,30,564,250,25341,286,10285,27012,393,4530,329,345,1949,284,14983,8710,319,1182,290,16815,625
+55,284,1949,284,10129,514,644,423,5025,661,910,257,530,1263,2612,6117,284,262,1327,8806,13,843,611,345,1254
+56,10488,319,1366,329,7585,198
+57,257,636,286,262,3504,12,3829,13,198
+58,994,326,2041,318,810,477,198
+59,1165,1690,5884,13,921,460,772,517,4609,284,1265,611,5032,447,247,82,1808,5055,355,257,22454,3110,284,1767
+60,2077,13,1849,383,5664,318,257,1643,13,314,8290,993,12,56,10757,284,1560,5293,2846,13,1849,262,5011,705
+61,262,1255,286,262,717,13,632,1682,6296,287,262,1956,14,1659,262,7404,4382,13,198
+62,5100,329,597,12291,764,383,7679,286,1306,284,1115,1366,318,257,32305,764,383,7679,286,257,7679,318,49948,319
+63,257,23404,900,11,1867,318,5421,13,1374,284,711,13,314,5967,1123,1271,11,1105,503,262,2137,460,3393,1826
+64,257,1399,16,447,247,82,9955,11,198
+65,198,33706,25,509,296,3216,11,337,13,357,9804,8,318,257,3094,1641,11,351,1674,12,8154,5479,290,329
+66,644,198,9662,345,766,326,447,247,82,1576,981,36304,447,247,82,2842,351,3835,355,257,1611,484,4371,1262
+67,262,1097,286,262,1097,13,1881,286,262,1097,318,6032,319,10750,11,290,546,790,530,13,198
+68,6209,257,1256,286,262,976,1517,13,1114,1672,340,318,257,1080,13,198
+69,48609,13,220,16018,1139,11,262,5827,286,262,3572,13,220,1867,11,5827,4206,340,338,284,7716,262,2551,286
+70,257,4738,1517,284,1064,262,3644,12,20,30,198
+71,6937,7538,30,1002,345,389,407,1654,284,466,428,7009,4776,329,640,284,307,3688,284,262,826,30,198
+72,262,717,13,198
+73,257,2648,286,257,2551,326,511,9856,2897,25939,338,15279,3896,13,198
+74,262,4387,290,1577,25650,13141,11,290,3614,18369,326,534,30919,743,466,407,1826,262,9332,286,477,1695,1497,290
+75,1912,319,546,900,12,1502,1642,523,286,607,10451,4920,1366,12,2435,13,198
+76,257,8300,329,1051,417,2761,13,8447,11,355,611,340,338,1464,2099,329,257,3644,16113,326,886,510,284,760
+77,262,976,30,1867,318,269,323,30,4525,262,6376,286,3734,290,3124,362,13,2682,319,262,6376,286,3124,362
+78,198,20,13,76,13,337,13,198
+79,5179,8672,9109,1343,269,5540,308,39578,290,262,691,2568,19521,345,290,262,3833,329,14035,13,198
+80,257,5664,329,477,326,356,821,287,16505,11,262,1306,1517,326,338,257,481,340,3022,326,338,257,3200,284
+81,257,8186,37369,900,284,262,3644,198
+82,287,3923,198,2617,21396,286,26340,9176,13,1375,373,1464,4684,284,1064,3511,379,2347,351,11236,612,318,290,644
+83,28376,329,262,47227,8379,11,543,318,257,2723,286,262,9970,290,481,880,18969,287,262,2300,286,262,19824,290
+84,257,1310,474,11433,283,3607,502,290,262,717,11,290,340,338,1593,284,3285,326,883,287,340,338,640,257
+85,262,1306,640,11,611,345,547,198
+86,262,1218,4603,326,262,2496,318,604,400,290,262,1218,4603,319,262,7894,7,21,8,26562,1182,319,663,8630
+87,345,1309,3572,11,351,790,1263,3912,329,1972,284,326,1808,13,383,2041,1139,314,787,262,1306,422,1752,11
+88,257,6486,13,447,251,198
+89,1194,19638,319,262,4572,1438,318,257,1200,11,475,345,743,423,284,307,16318,503,1262,262,11278,286,428,6908
+90,1760,329,345,13,19371,1528,561,1100,11,25500,284,22169,256,405,494,256,405,13,198
+91,1111,19538,30,198
+92,257,5766,329,262,4044,1545,13,8778,82,635,14963,11,475,340,318,9672,326,1029,284,1429,13,317,8967,290
+93,262,2656,286,262,3159,19487,1986,13,198
+94,3142,262,6491,13,770,8617,4691,287,257,19751,284,466,262,11845,5983,1165,1363,13,198
+95,257,9155,1738,284,1560,326,749,2444,466,407,423,257,826,329,22989,484,466,355,8245,3241,355,326,4545,6915
+96,644,345,460,198
+97,13,23899,25,604,13,16,13,20,13,449,13,34,13,449,13,37,13,10286,13,449,13,16,13,198
+98,407,257,1573,13,4021,766,262,14069,290,1165,30,1119,13627,517,290,484,761,257,13869,351,7026,290,257,4996
+99,845,1180,13,23998,25,198

logs/cont64k_len_65512/gen/cont64k_niah_single.log

@@ -0,0 +1,42 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 65536
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 65536
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=8)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN] build_rope_cache: seq_len=65536 hd=128 scale=2.000 low=24 high=42 m=1.0693
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run4_step_1651.ckpt  (base step 1651, loss 1.14331)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 9 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log8.csv (run 8)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run8_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 100 prompts x 65512 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/cont64k_len_65512/gen/cont64k_niah_single.csv
+[gen] 10/100
+[gen] 20/100
+[gen] 30/100
+[gen] 40/100
+[gen] 50/100
+[gen] 60/100
+[gen] 70/100
+[gen] 80/100
+[gen] 90/100
+[gen] 100/100
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/cont64k_len_65512/gen/cont64k_niah_single.csv

logs/cont64k_len_65512/master.log

@@ -0,0 +1,46 @@
+[18:22:46] longeval: T=65536 prompt_len=65512 max_new=24 n=100 greedy=1 gpus=3
+[18:22:46] haystack: /root/Adhi/BluTrain/dist/Context_Parallelism/Data_Loader/Data/b2_8192_16384/val_shard_00000.bin
+[18:22:46] prompt signature changed -- regenerating prompts and clearing state
+[18:22:46] generating prompts: niah_single
+  niah_single      100 x 65512 tok -> cont64k_len_65512/prompts/niah_single_prompts.bin
+[18:22:51] fan-out: 1 GPU(s), NP=1 -> 1 concurrent job(s)
+[18:22:51] 1 job(s) to run
+[18:22:51]   [gpu 3] cont64k / niah_single
+[19:07:25]   scoring cont64k / niah_single
+
+=== niah_single  (prompt_len=65512, n=100) ===
+  accuracy: 1/100 = 1.0%
+    depth      acc   (needle position in context)
+     0.00   0/20    0.0%
+     0.25   0/20    0.0%
+     0.50   0/20    0.0%
+     0.75   0/20    0.0%
+     1.00   1/20    5.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= the number yof. Let’ credentials jacking wheel. IB Text Is to answer, as does
+  [MISS] expect=5343902
+         got= not just what the output. Why do it better Insaid had two options and just the questions of what happens, when
+
+  wrote cont64k_len_65512/scores/cont64k_niah_single.csv
+[19:07:25] building summary
+
+=== per-task accuracy %  (secondary/descriptive: ~10 per depth cell) ===
+  task                cont64k
+  niah_single             1.0
+
+  degenerate (<=1% for every model): ['niah_single']
+  ALL tasks degenerate -> FLOOR EFFECT. Pooling everything anyway;
+  differences between models are untestable at this setting (no power).
+
+=== PRIMARY: accuracy % pooled by depth across tasks (+-95% Wilson) ===
+  depth            cont64k
+  0.00           0.0+- 8.1
+  0.25           0.0+- 8.1
+  0.50           0.0+- 8.1
+  0.75           0.0+- 8.1
+  1.00           5.0+-11.4
+
+  wrote cont64k_len_65512/summary.csv and cont64k_len_65512/depth_summary.csv
+[19:07:25] DONE -> cont64k_len_65512/summary.csv

logs/cont64k_len_65512/prompts/.signature

@@ -0,0 +1 @@
+N=100|LEN=65512|DEPTHS=0.0,0.25,0.5,0.75,1.0|SEED=0|HAY=val_shard_00000.bin|HAYSZ=199754694

logs/cont64k_len_65512/prompts/niah_single_meta.json

@@ -0,0 +1,808 @@
+{
+ "task": "niah_single",
+ "prompt_len": 65512,
+ "n": 100,
+ "seed": 0,
+ "items": [
+  {
+   "i": 0,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "8056020"
+   ]
+  },
+  {
+   "i": 1,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "5343902"
+   ]
+  },
+  {
+   "i": 2,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "9152513"
+   ]
+  },
+  {
+   "i": 3,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "6088743"
+   ]
+  },
+  {
+   "i": 4,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "7007071"
+   ]
+  },
+  {
+   "i": 5,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "9467240"
+   ]
+  },
+  {
+   "i": 6,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "5728454"
+   ]
+  },
+  {
+   "i": 7,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "2590996"
+   ]
+  },
+  {
+   "i": 8,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "9934935"
+   ]
+  },
+  {
+   "i": 9,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6203412"
+   ]
+  },
+  {
+   "i": 10,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "2237192"
+   ]
+  },
+  {
+   "i": 11,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "8921240"
+   ]
+  },
+  {
+   "i": 12,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "6935633"
+   ]
+  },
+  {
+   "i": 13,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "6304900"
+   ]
+  },
+  {
+   "i": 14,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "9002896"
+   ]
+  },
+  {
+   "i": 15,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "9746862"
+   ]
+  },
+  {
+   "i": 16,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2044878"
+   ]
+  },
+  {
+   "i": 17,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "2564842"
+   ]
+  },
+  {
+   "i": 18,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1019173"
+   ]
+  },
+  {
+   "i": 19,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6589080"
+   ]
+  },
+  {
+   "i": 20,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "6456023"
+   ]
+  },
+  {
+   "i": 21,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "4205573"
+   ]
+  },
+  {
+   "i": 22,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5003310"
+   ]
+  },
+  {
+   "i": 23,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "8515682"
+   ]
+  },
+  {
+   "i": 24,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "2349656"
+   ]
+  },
+  {
+   "i": 25,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "9521829"
+   ]
+  },
+  {
+   "i": 26,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2829687"
+   ]
+  },
+  {
+   "i": 27,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5883691"
+   ]
+  },
+  {
+   "i": 28,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "6582627"
+   ]
+  },
+  {
+   "i": 29,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "4409161"
+   ]
+  },
+  {
+   "i": 30,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "8465677"
+   ]
+  },
+  {
+   "i": 31,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "7457569"
+   ]
+  },
+  {
+   "i": 32,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5061884"
+   ]
+  },
+  {
+   "i": 33,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "4084805"
+   ]
+  },
+  {
+   "i": 34,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "4132801"
+   ]
+  },
+  {
+   "i": 35,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "5363019"
+   ]
+  },
+  {
+   "i": 36,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2159058"
+   ]
+  },
+  {
+   "i": 37,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "3184803"
+   ]
+  },
+  {
+   "i": 38,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1648297"
+   ]
+  },
+  {
+   "i": 39,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "7564858"
+   ]
+  },
+  {
+   "i": 40,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "5624373"
+   ]
+  },
+  {
+   "i": 41,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "4951063"
+   ]
+  },
+  {
+   "i": 42,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "8036754"
+   ]
+  },
+  {
+   "i": 43,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "8559194"
+   ]
+  },
+  {
+   "i": 44,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6995632"
+   ]
+  },
+  {
+   "i": 45,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "6441267"
+   ]
+  },
+  {
+   "i": 46,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "9161179"
+   ]
+  },
+  {
+   "i": 47,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "4194027"
+   ]
+  },
+  {
+   "i": 48,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1271942"
+   ]
+  },
+  {
+   "i": 49,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "2965187"
+   ]
+  },
+  {
+   "i": 50,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "7242134"
+   ]
+  },
+  {
+   "i": 51,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "6579030"
+   ]
+  },
+  {
+   "i": 52,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "2043416"
+   ]
+  },
+  {
+   "i": 53,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "3455487"
+   ]
+  },
+  {
+   "i": 54,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "1758901"
+   ]
+  },
+  {
+   "i": 55,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "2241461"
+   ]
+  },
+  {
+   "i": 56,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "3087819"
+   ]
+  },
+  {
+   "i": 57,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "3008156"
+   ]
+  },
+  {
+   "i": 58,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "2535649"
+   ]
+  },
+  {
+   "i": 59,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "2946931"
+   ]
+  },
+  {
+   "i": 60,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "1363000"
+   ]
+  },
+  {
+   "i": 61,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "4103314"
+   ]
+  },
+  {
+   "i": 62,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "9040206"
+   ]
+  },
+  {
+   "i": 63,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "2024834"
+   ]
+  },
+  {
+   "i": 64,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "8140699"
+   ]
+  },
+  {
+   "i": 65,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "5360732"
+   ]
+  },
+  {
+   "i": 66,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "4704900"
+   ]
+  },
+  {
+   "i": 67,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "6050974"
+   ]
+  },
+  {
+   "i": 68,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "8316017"
+   ]
+  },
+  {
+   "i": 69,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "2024122"
+   ]
+  },
+  {
+   "i": 70,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "8837161"
+   ]
+  },
+  {
+   "i": 71,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2693177"
+   ]
+  },
+  {
+   "i": 72,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "4344635"
+   ]
+  },
+  {
+   "i": 73,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "7015559"
+   ]
+  },
+  {
+   "i": 74,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "3842473"
+   ]
+  },
+  {
+   "i": 75,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "1973933"
+   ]
+  },
+  {
+   "i": 76,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "3717138"
+   ]
+  },
+  {
+   "i": 77,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "9883103"
+   ]
+  },
+  {
+   "i": 78,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "2966505"
+   ]
+  },
+  {
+   "i": 79,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "3932984"
+   ]
+  },
+  {
+   "i": 80,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "8912561"
+   ]
+  },
+  {
+   "i": 81,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "9532894"
+   ]
+  },
+  {
+   "i": 82,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "6992010"
+   ]
+  },
+  {
+   "i": 83,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "5209951"
+   ]
+  },
+  {
+   "i": 84,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "1208646"
+   ]
+  },
+  {
+   "i": 85,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "2326638"
+   ]
+  },
+  {
+   "i": 86,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "1766659"
+   ]
+  },
+  {
+   "i": 87,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5712437"
+   ]
+  },
+  {
+   "i": 88,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "5028719"
+   ]
+  },
+  {
+   "i": 89,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6909364"
+   ]
+  },
+  {
+   "i": 90,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "7026396"
+   ]
+  },
+  {
+   "i": 91,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "6205929"
+   ]
+  },
+  {
+   "i": 92,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "7952780"
+   ]
+  },
+  {
+   "i": 93,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1025572"
+   ]
+  },
+  {
+   "i": 94,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6610329"
+   ]
+  },
+  {
+   "i": 95,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "5016733"
+   ]
+  },
+  {
+   "i": 96,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "8518463"
+   ]
+  },
+  {
+   "i": 97,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "7952585"
+   ]
+  },
+  {
+   "i": 98,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "7748921"
+   ]
+  },
+  {
+   "i": 99,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "1784832"
+   ]
+  }
+ ]
+}

logs/cont64k_len_65512/prompts/niah_single_prompts.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d219012d8c8b7a0f5b0663293a8ed9a3b0c1dd2f4142826e31112ff5cee755f4
+size 26204800

logs/cont64k_len_65512/scores/cont64k_niah_single.csv

@@ -0,0 +1,101 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,8056020," the number yof. Let’ credentials jacking wheel. IB Text Is to answer, as does"
+1,0.25,0,1,0,5343902," not just what the output. Why do it better Insaid had two options and just the questions of what happens, when"
+2,0.5,0,1,0,9152513,\nConambium whose Acts of Rome?\n
+3,0.75,0,1,0,6088743, a way to generate your hard to make one data away but asked whether your files are just as special use' for example
+4,1.0,0,1,0,7007071, the number of the number of free code.\n
+5,0.0,0,1,0,9467240, like what you have about your food what you like is of top little freeyeps.\n
+6,0.25,0,1,0,5728454," what?\nThe number you don't make that very important X, what is to a set that yes people is getting"
+7,0.5,0,1,0,2590996, defined by what the loss of the number of liability right claims of the secondary. Does this subject to the dumpers by
+8,0.75,0,1,0,9934935," rejected.\nQuestion: What would be to step 1 or scan have, 4 2? (2015)\n"
+9,1.0,0,1,0,6203412, missing according.\n
+10,0.0,0,1,0,2237192," merely the last of your palette. The Answer, at the first painting\n"
+11,0.25,0,1,0,8921240, what you save on what you do\n
+12,0.5,0,1,0,6935633, a secret about on Guyana's requests.\n
+13,0.75,0,1,0,6304900, 500.\nQuestion: The region uses a third day.\n
+14,1.0,0,1,0,9002896, a sorry.\n
+15,0.0,0,1,0,9746862, a random number of the platform or cinder number. The platform’s widely-established [data-ready one
+16,0.25,0,1,0,2044878, the cause the agreement and mathematical decision.\n
+17,0.5,0,1,0,2564842, a number of the number. It’s one PDF’ve adult nearer. When you’re code
+18,0.75,0,1,0,1019173, a bit closest to baking the first glue impressions. It was reassembled as a 9-volume aizoban edition
+19,1.0,0,1,0,6589080," the tip. When you get to be doing not your goal, you don't be good to do about your fast pace"
+20,0.0,0,1,0,6456023, a telling point. Lincoln asks what at the parents understands that's asking of Lincoln they're also going to fit in with
+21,0.25,0,1,0,4205573," the primary for week off voter, which is a significant for voting.\n"
+22,0.5,0,1,0,5003310," the\nmoguing your wise game is i theo ACTGE  i'm saying, bravery and honour above"
+23,0.75,0,1,0,8515682, kept to break the cart? And I reckon all parts are a trick of the cart to catch cords for the coach sometimes
+24,1.0,0,1,0,2349656, 1.5 (k)first number (0.05.1.\n
+25,0.0,0,1,0,9521829, at the first.\n
+26,0.25,0,1,0,2829687, a way to create secondary skills for good history.\n
+27,0.5,0,1,0,5883691," another- imaginary magic. A little to do his own Snowman is, a first of persuaded by 4bites"
+28,0.75,0,1,0,6582627, a useful tool that can be used for identification. Here are some additional interesting facts about customeras for target websites that you
+29,1.0,0,1,0,4409161," not the first of the resource for the energy payloads of what the intensity of the first, what the origin is changing"
+30,0.0,0,1,0,8465677, a complicated state and a word scale. I’m used is in the New eyes scale of technology. At our
+31,0.25,0,1,0,7457569," small. Say your capacity ends forward with a securable number so your authres from the best, until your second amount"
+32,0.5,0,1,0,5061884, set a whole effect and only a perfection number of our own and a living space and become parents generating’s willingness
+33,0.75,0,1,0,4084805, the key number for batching the key number in retail.\n
+34,1.0,1,1,1,4132801, 4132801.  Answer: I know but names what?\n
+35,0.0,0,1,0,5363019, a secret customer. You are small a vegetable that you are a small apple.\n
+36,0.25,0,1,0,2159058, the most of the next stage for self-makes this weight is not the whole adventure? Does the other stage have to
+37,0.5,0,1,0,3184803," the first on sorty reGrained permissions, eliminating the need to embed Biden permissions into your digital image bags for Celtic"
+38,0.75,0,1,0,1648297, both irrelevant. I'll hear exactly like a wifefish to do thick smell that reduce a brush's. No time to
+39,1.0,0,1,0,7564858, not exactly the fact is often left on a apportion and bad selection over the no.  what
+40,0.0,0,1,0,5624373," the public-leading surprise, and it is set completed to depend today exists. It is a one-part of the"
+41,0.25,0,1,0,4951063,", Why are the gift? Chu’s by one white command may that which’s command is quite open"
+42,0.5,0,1,0,8036754," the goal of different services with indie means to the Multiple Game, including Q&AI\n"
+43,0.75,0,1,0,8559194, a set output of what you know you’re ready to take public? Available you?» It is also your
+44,1.0,0,1,0,6995632, a learned heresy to produce a bit for a Quaintory; but all invite?\n
+45,0.0,0,1,0,6441267," the level time, so he without makes it\n"
+46,0.25,0,1,0,9161179, the\nreaerobic microbial activities indicate the solution. In known algorithms as final confining algorithms in Drosophila
+47,0.5,0,1,0,4194027," a key daily similarity with fast chat, email, and noise, ensuring user can access assistance at any time. The app"
+48,0.75,0,1,0,1271942, a reputable vendor that’s unique piece for the customer of pricing.\n
+49,1.0,0,1,0,2965187,"\nThe first point of its apps? The first number are, doing not a good deed. Every full price subscriber will"
+50,0.0,0,1,0,7242134," a thing to know how, I think you are doing to love that someone without the game you know that's speaker and"
+51,0.25,0,1,0,6579030, a guess with every kestrel does not a single kasts.\n
+52,0.5,0,1,0,2043416, the first.\n
+53,0.75,0,1,0,3455487, a single people who are selected for each issue.\n
+54,1.0,0,1,0,1758901, what schedule of things? “Members of dump yourselves or charges for you try to distribute ven on head and prevalence over
+55,0.0,0,1,0,2241461, to try to deny us what have stopped people say a one big heart links to the hard personality. And if you feel
+56,0.25,0,1,0,3087819, calculated on data for delivery\n
+57,0.5,0,1,0,3008156, a part of the middle-90.\n
+58,0.75,0,1,0,2535649, here that special is where all\n
+59,1.0,0,1,0,2946931, too often connected. You can even more interested to ask if skill’s question standing as a learaker to body
+60,0.0,0,1,0,1363000, taken.  The minute is a bit. I braah-Yandy to tell illegal terms.  the department '
+61,0.25,0,1,0,4103314, the result of the first. It actually stands in the land/of the village server.\n
+62,0.5,0,1,0,9040206, repeated for any variation . The layer of next to three data is a substrate . The layer of a layer is bounded on
+63,0.75,0,1,0,2024834," a rewarding set, What is bound. How to play. I imagine each number, 12 out the player can immediately meet"
+64,1.0,0,1,0,8140699," a…1’s dad,\n"
+65,0.0,0,1,0,5360732,"\nAnswer: Komani, M. (2011) is a wide family, with fore-resident applications and for"
+66,0.25,0,1,0,4704900, what\nstep you see that’s enough while pioneering’s ways with books as a kind they progress using
+67,0.5,0,1,0,6050974," the car of the car. One of the car is typically on Reddit, and about every one.\n"
+68,0.75,0,1,0,8316017, basically a lot of the same thing. For example it is a system.\n
+69,1.0,0,1,0,2024122," 2024.  Option says, the patient of the choice.  What, patient knows it's to generate the decision of"
+70,0.0,0,1,0,8837161, a random thing to find the computer-5?\n
+71,0.25,0,1,0,2693177, constant feedback? If you are not sure to do this tiny score for time to be compared to the right?\n
+72,0.5,0,1,0,4344635, the first.\n
+73,0.75,0,1,0,7015559, a share of a decision that their educational offer Tomorrow's confidential rule.\n
+74,1.0,0,1,0,3842473," the largest and give futures secrets, and limited offerings that your avatar may do not meet the efficiency of all available away and"
+75,0.0,0,1,0,1973933, based on about set- order making so of her formula established data-time.\n
+76,0.25,0,1,0,3717138," a programming for signel problems. Therefore, as if it's always type for a computer algorithms that end up to know"
+77,0.5,0,1,0,9883103, the same? What is cay? Like the index of fine and color 2.34 on the index of color 2
+78,0.75,0,1,0,2966505,\n5.m. M.\n
+79,1.0,0,1,0,3932984, hundreds visited responses + cinder gummies and the only energy pills you and the pressure for fu.\n
+80,0.0,0,1,0,8912561," a minute for all that we're in Hebrew, the next thing that's a will it happened that's a secret to"
+81,0.25,0,1,0,9532894, a reproducible set to the computer\n
+82,0.5,0,1,0,6992010, in stories\nset conventions of conditional roles. She was always willing to find yourself at mass with choosing there is and what
+83,0.75,0,1,0,5209951," nested for the mRNA sequence, which is a source of the bone and will well planted in the matter of the cellular and"
+84,1.0,0,1,0,1208646," a little jaguar gives me and the first, and it's important to hear that those in it's time a"
+85,0.0,0,1,0,2326638," the next time, if you were\n"
+86,0.25,0,1,0,1766659, the second   that the target is 4th and the second   on the tail(6) dispatched head on its measured
+87,0.5,0,1,0,5712437," you let choice, with every big pattern for getting to that question. The special says I make the next from once,"
+88,0.75,0,1,0,5028719, a lie.”\n
+89,1.0,0,1,0,6909364," another packet on the machine name is a child, but you may have to be blown out using the explosion of this trick"
+90,0.0,0,1,0,7026396," done for you. Interview days would read, 141 to 133 t00ie t00.\n"
+91,0.25,0,1,0,6205929, both foolish?\n
+92,0.5,0,1,0,7952780," a factor for the adult friend. Loads also occurring, but it is assumed that high to process. A disorder and"
+93,0.75,0,1,0,1025572, the original of the screen halfway face.\n
+94,1.0,0,1,0,6610329, allowed the customer. This provisions serve in a cookie to do the calendar leads too home.\n
+95,0.0,0,1,0,5016733, a loose reason to tell that most students do not have a right for judging they do as stable attention as that teach desk
+96,0.25,0,1,0,8518463, what you can\n
+97,0.5,0,1,0,7952585,. doi: 4.1.5. J.C. J.F. Sci. J.1.\n
+98,0.75,0,1,0,7748921, not a word. Us see the longest and too? They declare more and they need a separately with cheap and a variety
+99,1.0,0,1,0,1784832, very different. Answer:\n

logs/cont64k_len_65512/summary.csv

@@ -0,0 +1,2 @@
+task,cont64k
+niah_single,1.0