Upload folder using huggingface_hub

logs/cont64k_len_32744/depth_summary.csv

@@ -0,0 +1,6 @@
+depth,model,hits,n,pct,wilson95
+0.0,cont64k,0,20,0.0,8.1
+0.25,cont64k,0,20,0.0,8.1
+0.5,cont64k,0,20,0.0,8.1
+0.75,cont64k,0,20,0.0,8.1
+1.0,cont64k,2,20,10.0,13.7

logs/cont64k_len_32744/gen/cont64k_niah_single.csv

@@ -0,0 +1,100 @@
+0,257,1693,329,1804,262,1271,198
+1,345,13,632,318,8780,284,3853,257,17763,1664,284,787,534,9206,13,10054,1262,262,9206,11,290,3853,257,2060
+2,644,345,765,284,466,351,262,3807,290,466,314,1607,340,30,198
+3,994,588,41567,319,262,12187,13,1002,345,1064,644,345,765,284,1064,257,1048,287,257,1178,2745,11,345,460
+4,257,5421,290,1195,495,25019,10645,2340,13,198
+5,2060,12,400,563,25528,286,262,43331,30,198
+6,407,1231,393,407,11,523,7398,5832,2192,2314,466,340,13,1002,345,836,470,765,284,466,340,11,345,1183
+7,477,625,262,8850,8398,13,198
+8,257,9909,2318,12590,290,198
+9,13037,13,198,24361,25,1867,1838,262,17963,30,198
+10,407,257,3572,13,383,1271,286,883,326,389,3402,287,11291,604,13,198
+11,407,257,2041,1271,475,340,318,407,257,2041,1271,475,340,318,407,257,2041,1271,475,340,318,407,257,2041
+12,635,407,257,1351,286,38464,1377,691,262,717,1517,318,284,466,13,2312,717,1517,318,262,198
+13,8646,329,275,859,903,318,8646,329,275,859,903,318,8646,329,275,859,903,318,8646,329,275,859,903,198
+14,262,1255,286,262,717,640,13,198
+15,262,2041,1271,329,262,2041,1271,13,198
+16,262,13204,2746,11507,1143,416,262,1366,13,198
+17,262,2060,12,2777,1127,810,198
+18,407,257,10715,13,1867,318,340,30,198
+19,718,3365,7410,13,198
+20,257,1310,1643,5749,621,345,760,13,198
+21,262,4334,26,262,366,51,2634,76,2634,430,557,1,318,257,12967,26,262,366,51,2634,76,2634,430,557
+22,5323,23,930,838,74,13,198
+23,8574,13,198,33706,25,383,474,11433,283,2476,284,15458,262,2858,13,383,474,11433,283,2476,284,15458,262,2858
+24,604,13,383,717,1271,286,262,717,1271,286,262,2344,318,604,13,383,717,1271,286,262,717,1271,286,262
+25,262,530,326,318,257,1654,284,262,922,1517,13,685,405,25,1558,25,1238,60,383,3056,318,262,530,326
+26,281,47729,11,290,772,517,11,257,8379,286,4831,13,198
+27,257,922,835,284,256,917,430,13,23998,284,256,274,5031,11,543,318,262,14069,1271,286,812,287,262,7374
+28,284,307,1498,284,9367,329,262,1306,4876,286,2989,3113,23989,13,198
+29,604,29416,17,13,220,198
+30,257,1643,286,5969,1271,13,198
+31,257,9314,11,290,355,6006,33,2767,468,262,2694,284,9320,351,262,2836,11,543,318,257,1994,284,2836,12
+32,345,11,345,821,407,1016,284,466,4232,13,20292,534,4226,30,9425,11,10194,11,314,655,765,284,760,644
+33,262,1271,286,5054,329,473,381,10695,284,1011,5054,572,290,1577,1337,319,14916,11,326,290,262,661,286,39282
+34,1464,1016,284,4656,13,1406,340,481,307,257,1263,2994,284,2046,6541,326,2910,5354,3484,1279,29,17,13,20
+35,262,691,1936,7349,7177,685,5661,2420,1391,7785,16,92,290,1391,7785,16,92,1391,7785,16,92,1391,7785,16
+36,326,13,1406,644,1048,389,345,1016,284,1064,257,1693,326,318,11,611,345,821,2045,329,3946,326,389,407
+37,257,1049,5032,11,290,340,338,257,922,1517,284,2251,257,8364,326,1595,470,655,787,340,257,23404,1693,11
+38,262,48182,286,534,1110,290,12352,13,198
+39,530,286,262,13391,691,393,38457,329,1751,4585,257,31610,8855,329,262,7363,13,198
+40,262,530,198,1532,262,1271,286,262,1271,286,262,1271,286,262,1271,286,262,1271,286,262,1271,286,262,1271
+41,845,1402,329,2057,2891,393,5301,530,8212,319,30,1892,326,318,11,314,714,1577,340,284,749,286,606,13
+42,655,644,345,760,318,11,644,345,1612,318,644,345,821,1804,30,198
+43,262,976,1271,329,1123,4540,290,1575,12,11659,4540,13,198
+44,718,2079,3980,2624,13,198
+45,262,691,1517,329,644,345,460,470,466,351,262,1956,13,554,584,2456,11,644,345,460,466,318,284,2190
+46,257,27226,540,11,644,954,318,7226,30,198
+47,262,530,11,290,611,262,530,314,716,6717,286,6970,11,314,716,407,13,1114,1752,517,262,21752,373,257
+48,257,20622,290,20622,329,16727,6355,12066,13,770,318,257,3665,290,13097,3164,284,1366,4542,11,290,340,318,257
+49,2808,2996,23451,13,220,23998,25,1867,318,262,2882,30,198
+50,257,1402,475,7223,1502,960,1169,2041,5032,900,345,407,284,2222,534,2748,3725,13,1649,345,447,247,260,1965
+51,262,3200,1271,329,509,970,12,44245,13,198
+52,644,314,869,262,275,859,903,284,644,345,869,262,275,859,903,1271,13,1867,345,869,262,275,859,903
+53,257,2829,835,284,5806,530,286,22445,11,787,257,16708,11,290,8472,4673,326,481,307,257,11483,13,2750,1642
+54,257,3665,11,290,340,447,247,82,257,3748,27658,7625,326,460,307,925,13,5501,649,4190,2099,318,257,8780
+55,257,2060,12,2617,1891,329,477,262,7259,13,632,447,247,82,257,1103,16840,11,257,1103,16840,11,257,1103
+56,262,691,1517,326,460,307,5969,329,4395,13,1475,2707,82,11,329,262,749,636,13,1475,2707,82,329,257
+57,8780,13,198,2437,284,779,554,7784,22137,198
+58,257,5337,1171,1535,13,2102,11,611,262,1976,27446,2417,2523,7044,5260,286,1976,27446,2417,11,262,12867,286,16937
+59,262,1271,286,262,2124,2645,38656,1271,13,198
+60,5128,13,198,33706,25,383,2041,5536,1271,329,262,5072,318,262,5072,286,262,5072,13,198
+61,257,1256,286,4875,11,4875,198
+62,1936,7319,5054,13,198
+63,257,1402,1271,286,1693,6443,13,198
+64,257,3200,13,1114,640,284,2342,198
+65,257,6908,959,621,262,1271,286,4831,13,632,338,477,475,351,16287,13,3954,2187,10038,318,5670,319,262,13389
+66,257,25155,1271,286,15283,7937,13,198
+67,604,4,10107,3259,508,423,5597,329,511,6334,13,1849,14026,4420,11,2985,460,3774,345,284,787,1588,12,9888
+68,262,691,3599,1271,11,290,262,3599,1271,318,262,3599,1271,11,290,262,3599,1271,318,262,3599,1271,11,290
+69,352,13,198,33706,25,383,2041,5536,1271,329,41207,318,352,13,198
+70,1813,262,2656,5770,2256,286,262,12584,13,383,5770,2256,1813,262,10112,983,1271,318,262,976,355,262,5770,2256
+71,257,845,1402,13,23998,262,10959,772,351,257,7684,286,6097,13,4149,13,632,338,257,1643,286,257,1643,286
+72,257,6087,286,1976,4207,357,1169,649,8950,737,198
+73,691,46190,1395,392,24866,389,407,257,1351,284,749,22354,78,936,85,308,39578,11,290,262,2420,318,407,691
+74,262,575,2261,3797,857,607,898,30,198
+75,257,1994,3704,286,3734,4898,13,198
+76,257,29000,1271,286,3463,2994,13,198
+77,257,1310,3200,13,383,3200,318,257,3200,1271,286,5536,1271,13,632,447,247,82,588,257,3200,1271,286,262
+78,644,345,466,13,198
+79,6209,257,649,670,329,14704,6805,290,3303,1849,17018,13,314,1849,26124,263,1849,271,546,852,262,1366,1912,319
+80,262,1271,286,9670,13141,13,198
+81,262,1271,286,262,1271,286,11490,287,262,5926,286,24382,13,383,1271,286,11490,329,257,3096,286,2706,318,257
+82,257,30669,13,198
+83,326,11,644,318,340,11,326,318,11,644,318,340,11,326,318,11,644,318,340,11,326,318,11,644
+84,262,691,1738,329,511,11519,16247,13,4900,477,511,9929,942,618,3592,920,10119,389,3177,284,307,257,922,1672
+85,262,1994,284,9552,290,9552,198
+86,257,4738,1271,286,953,9395,13,198
+87,198,3581,17406,515,284,198
+88,262,1048,286,2910,10154,11,262,1048,326,345,460,1560,1771,345,460,651,340,5295,30,8314,340,423,257,1438
+89,8644,2713,26780,11,543,318,1444,327,1437,68,11,318,340,1444,11,318,340,1444,11,318,340,1444,11,318
+90,2048,790,1735,286,1692,4257,28554,13,1867,318,262,198
+91,257,2176,10742,286,257,14411,1720,290,9387,7538,284,1981,290,511,6461,351,4888,2565,286,1535,13,12032,11,262
+92,262,2443,3670,11,262,2443,3670,11,262,2443,3670,11,262,2443,3670,11,262,2443,3670,11,262,2443,3670,11
+93,257,3953,286,24215,1102,13,24,2231,12,3829,4,198
+94,3190,48287,329,5172,30152,7585,13,220,18233,25,220,220,220,220,220,220,220,220,220,220,220,220,220,220
+95,479,395,2411,329,479,395,2411,318,479,395,2411,329,479,395,2411,318,479,395,2411,329,479,395,2411,318
+96,257,4738,1517,1022,345,198
+97,257,20276,13,198
+98,262,976,329,502,13,198
+99,257,2060,1271,286,2438,691,329,340,13,198

logs/cont64k_len_32744/gen/cont64k_niah_single.log

@@ -0,0 +1,43 @@
+=== Llama Context Parallel Training Script (bluscriptCP) ===
+[CP attn fusion] split (Stage-2 rope-once + Stage-3 Hopper causal-attn kernels)
+================ Configuration (bluscriptCP) ================
+  d_model/layers  : 1536 / 36
+  q/kv heads      : 12 / 4  (G=3, hd=128)
+  ffn_hidden      : 4096 (SwiGLU)
+  context_length  : 65536
+  vocab_size      : 50304  tying=0
+  B x T           : 1 x 65536
+  world_size      : 1  (dp=1 x cp=1)
+  CP_ATTN_MODE    : ring  (load_balance=HeadTail)
+  global_batch    : 524288  (grad_accum=8)
+  precision       : bf16 Linears + fp32 norms/residual/logits (fp32 master weights in AdamW)  [mask=15 QKV WO MLP HEAD]
+  max_lr/min_lr   : 0.0006 / 6e-05
+  warmup/steps    : 91 / 917  (cosine, no stable phase)
+  checkpointing   : true  (ckpt_freq=50, dir=checkpoints_bluscriptcp)
+=============================================================
+Initializing model on CUDA device 0...
+[YaRN] build_rope_cache: seq_len=65536 hd=128 scale=2.000 low=24 high=42 m=1.0693
+[InitFrom] weights loaded from /root/Adhi/BluTrain/dist/Context_Parallelism/checkpoints_bluscriptcp/rank_0/blumodelcp_run4_step_1651.ckpt  (base step 1651, loss 1.14331)  -- optimizer state and RNG intentionally NOT restored; pair with CP_REWARMUP
+Parameters: 1060624896
+Parameters per GPU: 1060624896
+max_steps: 917
+warmup_steps: 91
+[precision] bf16 params=217, all with fp32 master weights
+found 9 shards for split train
+found 1 shards for split val
+Saving logs to: CP_BluScreipt_Training_logs/CP_Training_log5.csv (run 5)
+[rope_theta] 500000 compiled default (no sidecar -- legacy run, assuming it was trained at this base)
+Checkpointing: ON -> checkpoints_bluscriptcp/blumodelcp_run5_step_<N>.ckpt (every 50 steps, keep 6)
+[gen] 100 prompts x 32744 tok, max_new=24 greedy -> /root/Adhi/BluTrain/dist/Context_Parallelism/cont64k_len_32744/gen/cont64k_niah_single.csv
+[gen] 10/100
+[gen] 20/100
+[gen] 30/100
+[gen] 40/100
+[gen] 50/100
+[gen] 60/100
+[gen] 70/100
+[gen] 80/100
+[gen] 90/100
+[gen] 100/100
+[gen] wrote /root/Adhi/BluTrain/dist/Context_Parallelism/cont64k_len_32744/gen/cont64k_niah_single.csv
+[round-word-welcomes-fin-02][[23450,1],0][../../../../../../opal/mca/btl/tcp/btl_tcp.c:559:mca_btl_tcp_recv_blocking] recv(125) failed: Connection reset by peer (104)

logs/cont64k_len_32744/master.log

@@ -0,0 +1,42 @@
+[18:22:46] longeval: T=65536 prompt_len=32744 max_new=24 n=100 greedy=1 gpus=2
+[18:22:46] haystack: /root/Adhi/BluTrain/dist/Context_Parallelism/Data_Loader/Data/b2_8192_16384/val_shard_00000.bin
+[18:22:46] prompt signature changed -- regenerating prompts and clearing state
+[18:22:46] generating prompts: niah_single
+  niah_single      100 x 32744 tok -> cont64k_len_32744/prompts/niah_single_prompts.bin
+[18:22:50] fan-out: 1 GPU(s), NP=1 -> 1 concurrent job(s)
+[18:22:50] 1 job(s) to run
+[18:22:51]   [gpu 2] cont64k / niah_single
+[18:42:50]   scoring cont64k / niah_single
+
+=== niah_single  (prompt_len=32744, n=100) ===
+  accuracy: 2/100 = 2.0%
+    depth      acc   (needle position in context)
+     0.00   0/20    0.0%
+     0.25   0/20    0.0%
+     0.50   0/20    0.0%
+     0.75   0/20    0.0%
+     1.00   2/20   10.0%
+
+  --- 2 sample generations ---
+  [MISS] expect=8056020
+         got= a job for doing the number\n
+  [MISS] expect=5343902
+         got= you. It is crucial to choose a tile company to make your password. Continue using the password, and choose a single
+
+  wrote cont64k_len_32744/scores/cont64k_niah_single.csv
+[18:42:50] building summary
+
+=== per-task accuracy %  (secondary/descriptive: ~10 per depth cell) ===
+  task                cont64k
+  niah_single             2.0
+
+=== PRIMARY: accuracy % pooled by depth across tasks (+-95% Wilson) ===
+  depth            cont64k
+  0.00           0.0+- 8.1
+  0.25           0.0+- 8.1
+  0.50           0.0+- 8.1
+  0.75           0.0+- 8.1
+  1.00          10.0+-13.7
+
+  wrote cont64k_len_32744/summary.csv and cont64k_len_32744/depth_summary.csv
+[18:42:50] DONE -> cont64k_len_32744/summary.csv

logs/cont64k_len_32744/prompts/.signature

@@ -0,0 +1 @@
+N=100|LEN=32744|DEPTHS=0.0,0.25,0.5,0.75,1.0|SEED=0|HAY=val_shard_00000.bin|HAYSZ=199754694

logs/cont64k_len_32744/prompts/niah_single_meta.json

@@ -0,0 +1,808 @@
+{
+ "task": "niah_single",
+ "prompt_len": 32744,
+ "n": 100,
+ "seed": 0,
+ "items": [
+  {
+   "i": 0,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "8056020"
+   ]
+  },
+  {
+   "i": 1,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "5343902"
+   ]
+  },
+  {
+   "i": 2,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "9152513"
+   ]
+  },
+  {
+   "i": 3,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "6088743"
+   ]
+  },
+  {
+   "i": 4,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "7007071"
+   ]
+  },
+  {
+   "i": 5,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "9467240"
+   ]
+  },
+  {
+   "i": 6,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "5728454"
+   ]
+  },
+  {
+   "i": 7,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "2590996"
+   ]
+  },
+  {
+   "i": 8,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "9934935"
+   ]
+  },
+  {
+   "i": 9,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6203412"
+   ]
+  },
+  {
+   "i": 10,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "2237192"
+   ]
+  },
+  {
+   "i": 11,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "8921240"
+   ]
+  },
+  {
+   "i": 12,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "6935633"
+   ]
+  },
+  {
+   "i": 13,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "6304900"
+   ]
+  },
+  {
+   "i": 14,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "9002896"
+   ]
+  },
+  {
+   "i": 15,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "9746862"
+   ]
+  },
+  {
+   "i": 16,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2044878"
+   ]
+  },
+  {
+   "i": 17,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "2564842"
+   ]
+  },
+  {
+   "i": 18,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1019173"
+   ]
+  },
+  {
+   "i": 19,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6589080"
+   ]
+  },
+  {
+   "i": 20,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "6456023"
+   ]
+  },
+  {
+   "i": 21,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "4205573"
+   ]
+  },
+  {
+   "i": 22,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5003310"
+   ]
+  },
+  {
+   "i": 23,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "8515682"
+   ]
+  },
+  {
+   "i": 24,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "2349656"
+   ]
+  },
+  {
+   "i": 25,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "9521829"
+   ]
+  },
+  {
+   "i": 26,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2829687"
+   ]
+  },
+  {
+   "i": 27,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5883691"
+   ]
+  },
+  {
+   "i": 28,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "6582627"
+   ]
+  },
+  {
+   "i": 29,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "4409161"
+   ]
+  },
+  {
+   "i": 30,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "8465677"
+   ]
+  },
+  {
+   "i": 31,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "7457569"
+   ]
+  },
+  {
+   "i": 32,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5061884"
+   ]
+  },
+  {
+   "i": 33,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "4084805"
+   ]
+  },
+  {
+   "i": 34,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "4132801"
+   ]
+  },
+  {
+   "i": 35,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "5363019"
+   ]
+  },
+  {
+   "i": 36,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2159058"
+   ]
+  },
+  {
+   "i": 37,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "3184803"
+   ]
+  },
+  {
+   "i": 38,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1648297"
+   ]
+  },
+  {
+   "i": 39,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "7564858"
+   ]
+  },
+  {
+   "i": 40,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "5624373"
+   ]
+  },
+  {
+   "i": 41,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "4951063"
+   ]
+  },
+  {
+   "i": 42,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "8036754"
+   ]
+  },
+  {
+   "i": 43,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "8559194"
+   ]
+  },
+  {
+   "i": 44,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6995632"
+   ]
+  },
+  {
+   "i": 45,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "6441267"
+   ]
+  },
+  {
+   "i": 46,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "9161179"
+   ]
+  },
+  {
+   "i": 47,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "4194027"
+   ]
+  },
+  {
+   "i": 48,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1271942"
+   ]
+  },
+  {
+   "i": 49,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "2965187"
+   ]
+  },
+  {
+   "i": 50,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "7242134"
+   ]
+  },
+  {
+   "i": 51,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "6579030"
+   ]
+  },
+  {
+   "i": 52,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "2043416"
+   ]
+  },
+  {
+   "i": 53,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "3455487"
+   ]
+  },
+  {
+   "i": 54,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "1758901"
+   ]
+  },
+  {
+   "i": 55,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "2241461"
+   ]
+  },
+  {
+   "i": 56,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "3087819"
+   ]
+  },
+  {
+   "i": 57,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "3008156"
+   ]
+  },
+  {
+   "i": 58,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "2535649"
+   ]
+  },
+  {
+   "i": 59,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "2946931"
+   ]
+  },
+  {
+   "i": 60,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "1363000"
+   ]
+  },
+  {
+   "i": 61,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "4103314"
+   ]
+  },
+  {
+   "i": 62,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "9040206"
+   ]
+  },
+  {
+   "i": 63,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "2024834"
+   ]
+  },
+  {
+   "i": 64,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "8140699"
+   ]
+  },
+  {
+   "i": 65,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "5360732"
+   ]
+  },
+  {
+   "i": 66,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "4704900"
+   ]
+  },
+  {
+   "i": 67,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "6050974"
+   ]
+  },
+  {
+   "i": 68,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "8316017"
+   ]
+  },
+  {
+   "i": 69,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "2024122"
+   ]
+  },
+  {
+   "i": 70,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "8837161"
+   ]
+  },
+  {
+   "i": 71,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "2693177"
+   ]
+  },
+  {
+   "i": 72,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "4344635"
+   ]
+  },
+  {
+   "i": 73,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "7015559"
+   ]
+  },
+  {
+   "i": 74,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "3842473"
+   ]
+  },
+  {
+   "i": 75,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "1973933"
+   ]
+  },
+  {
+   "i": 76,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "3717138"
+   ]
+  },
+  {
+   "i": 77,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "9883103"
+   ]
+  },
+  {
+   "i": 78,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "2966505"
+   ]
+  },
+  {
+   "i": 79,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "3932984"
+   ]
+  },
+  {
+   "i": 80,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "8912561"
+   ]
+  },
+  {
+   "i": 81,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "9532894"
+   ]
+  },
+  {
+   "i": 82,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "6992010"
+   ]
+  },
+  {
+   "i": 83,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "5209951"
+   ]
+  },
+  {
+   "i": 84,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "1208646"
+   ]
+  },
+  {
+   "i": 85,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "2326638"
+   ]
+  },
+  {
+   "i": 86,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "1766659"
+   ]
+  },
+  {
+   "i": 87,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "5712437"
+   ]
+  },
+  {
+   "i": 88,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "5028719"
+   ]
+  },
+  {
+   "i": 89,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6909364"
+   ]
+  },
+  {
+   "i": 90,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "7026396"
+   ]
+  },
+  {
+   "i": 91,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "6205929"
+   ]
+  },
+  {
+   "i": 92,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "7952780"
+   ]
+  },
+  {
+   "i": 93,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "1025572"
+   ]
+  },
+  {
+   "i": 94,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "6610329"
+   ]
+  },
+  {
+   "i": 95,
+   "depth": 0.0,
+   "depth_requested": 0.0,
+   "expected": [
+    "5016733"
+   ]
+  },
+  {
+   "i": 96,
+   "depth": 0.25,
+   "depth_requested": 0.25,
+   "expected": [
+    "8518463"
+   ]
+  },
+  {
+   "i": 97,
+   "depth": 0.5,
+   "depth_requested": 0.5,
+   "expected": [
+    "7952585"
+   ]
+  },
+  {
+   "i": 98,
+   "depth": 0.75,
+   "depth_requested": 0.75,
+   "expected": [
+    "7748921"
+   ]
+  },
+  {
+   "i": 99,
+   "depth": 1.0,
+   "depth_requested": 1.0,
+   "expected": [
+    "1784832"
+   ]
+  }
+ ]
+}

logs/cont64k_len_32744/prompts/niah_single_prompts.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c25eef89e398c64b7f17974ce0ee475a0da5f9780764fa3cc38d024d19133061
+size 13097600

logs/cont64k_len_32744/scores/cont64k_niah_single.csv

@@ -0,0 +1,101 @@
+i,depth,ok,n_expected,n_hit,expected,generated
+0,0.0,0,1,0,8056020, a job for doing the number\n
+1,0.25,0,1,0,5343902," you. It is crucial to choose a tile company to make your password. Continue using the password, and choose a single"
+2,0.5,0,1,0,9152513, what you want to do with the movie and do I expect it?\n
+3,0.75,0,1,0,6088743," here like icing on the cake. If you find what you want to find a person in a few weeks, you can"
+4,1.0,0,1,0,7007071, a bound and Qure membrane lipids.\n
+5,0.0,0,1,0,9467240, single-thry Acts of the Apostles?\n
+6,0.25,0,1,0,5728454," not without or not, so……you probably cannot do it. If you don't want to do it, you'll"
+7,0.5,0,1,0,2590996, all over the ordinary configuration.\n
+8,0.75,0,1,0,9934935, a stolen barbie and\n
+9,1.0,0,1,0,6203412, 700.\nQuestion: What makes the dice?\n
+10,0.0,0,1,0,2237192, not a choice. The number of those that are shown in Figure 4.\n
+11,0.25,0,1,0,8921240, not a special number but it is not a special number but it is not a special number but it is not a special
+12,0.5,0,1,0,6935633, also not a list of groceries -- only the first thing is to do. These first thing is the\n
+13,0.75,0,1,0,6304900, 250 for bramble is 250 for bramble is 250 for bramble is 250 for bramble\n
+14,1.0,0,1,0,9002896, the result of the first time.\n
+15,0.0,0,1,0,9746862, the special number for the special number.\n
+16,0.25,0,1,0,2044878, the structural model parameterized by the data.\n
+17,0.5,0,1,0,2564842, the single-spines where\n
+18,0.75,0,1,0,1019173, not a mystery. What is it?\n
+19,1.0,0,1,0,6589080, 658800.\n
+20,0.0,0,1,0,6456023, a little bit bigger than you know.\n
+21,0.25,0,1,0,4205573," the heavy; the ""Téméraire"" is a pig; the ""Téméraire"
+22,0.5,0,1,0,5003310, 5008 | 10k.\n
+23,0.75,0,1,0,8515682, stored.\nAnswer: The jaguar needs to batch the environment. The jaguar needs to batch the environment
+24,1.0,0,1,0,2349656, 4. The first number of the first number of the wind is 4. The first number of the first number of the
+25,0.0,0,1,0,9521829, the one that is a sure to the good thing. [00:17:20] The oil is the one that
+26,0.25,0,1,0,2829687," an achievable, and even more, a sequence of steps.\n"
+27,0.5,0,1,0,5883691," a good way to tundra. Answer to tesla, which is the longest number of years in the winter"
+28,0.75,0,1,0,6582627, to be able to scan for the next task of search engine optimization.\n
+29,1.0,0,1,0,4409161, 44092. \n
+30,0.0,0,1,0,8465677, a bit of fixed number.\n
+31,0.25,0,1,0,7457569," a reliable, and as SHBET has the ability to compete with the user, which is a key to user-"
+32,0.5,0,1,0,5061884," you, you're not going to do whatever. Adjust your script? Yeah, yeah, I just want to know what"
+33,0.75,0,1,0,4084805," the number of dollars for sapphire to take dollars off and give care on activation, that and the people of Guam"
+34,1.0,0,1,0,4132801, always going to die. So it will be a big loss to fire guns that bloodlock costs <>2.5
+35,0.0,0,1,0,5363019, the only five Java array [this text {var1} and {var1} {var1} {var1
+36,0.25,0,1,0,2159058," that. So what person are you going to find a job that is, if you're looking for jobs that are not"
+37,0.5,0,1,0,3184803," a great skill, and it's a good thing to create a recipe that doesn't just make it a rewarding job,"
+38,0.75,0,1,0,1648297, the acknowledgment of your day and engagement.\n
+39,1.0,0,1,0,7564858, one of the advantages only or disadvantages for children calling a thrilling adventure for the lies.\n
+40,0.0,0,1,0,5624373, the one\nIf the number of the number of the number of the number of the number of the number of the number
+41,0.25,0,1,0,4951063," very small for food tool or package one smile on? Not that is, I could give it to most of them."
+42,0.5,0,1,0,8036754," just what you know is, what you mean is what you're doing?\n"
+43,0.75,0,1,0,8559194, the same number for each candidate and cost-track candidate.\n
+44,1.0,1,1,1,6995632, 6995632.\n
+45,0.0,0,1,0,6441267," the only thing for what you can't do with the land. In other words, what you can do is to treat"
+46,0.25,0,1,0,9161179," a syllable, what count is typical?\n"
+47,0.5,0,1,0,4194027," the one, and if the one I am guilty of knowing, I am not. For once more the princess was a"
+48,0.75,0,1,0,1271942," a proprietary and proprietary for obtaining corporate entities. This is a powerful and innovative approach to data management, and it is a"
+49,1.0,1,1,1,2965187, 2965187.  Answer: What is the response?\n
+50,0.0,0,1,0,7242134, a small but attached order—the special skill set you not to bring your exact knowledge. When you’re asked
+51,0.25,0,1,0,6579030, the secret number for Kool-Aid.\n
+52,0.5,0,1,0,2043416, what I call the bramble to what you call the bramble number. What you call the bramble
+53,0.75,0,1,0,3455487," a simple way to wear one of dedication, make a trait, and practical learning that will be a lesson. By making"
+54,1.0,0,1,0,1758901," a powerful, and it’s a unique customized gap that can be made. Each new hair type is a crucial"
+55,0.0,0,1,0,2241461," a single-setback for all the songs. It’s a real gem, a real gem, a real"
+56,0.25,0,1,0,3087819," the only thing that can be fixed for shooting. Exceeds, for the most part. Exceeds for a"
+57,0.5,0,1,0,3008156, crucial.\nHow to use Inbound Marketing\n
+58,0.75,0,1,0,2535649," a novel public health. However, if the zephyr shows ongoing measures of zephyr, the probability of achieving"
+59,1.0,0,1,0,2946931, the number of the xylophone number.\n
+60,0.0,0,1,0,1363000, input.\nAnswer: The special magic number for the output is the output of the output.\n
+61,0.25,0,1,0,4103314," a lot of digital, digital\n"
+62,0.5,0,1,0,9040206, five thousand dollars.\n
+63,0.75,0,1,0,2024834, a small number of job opportunities.\n
+64,1.0,0,1,0,8140699, a secret. For time to watch\n
+65,0.0,0,1,0,5360732, a trickier than the number of steps. It's all but with clarity. Our whole mood is focused on the transformation
+66,0.25,0,1,0,4704900, a cursed number of sealed ships.\n
+67,0.5,0,1,0,6050974," 4% manual workers who have prepared for their emergency.  Tellingly, users can trust you to make large-scale"
+68,0.75,0,1,0,8316017," the only starting number, and the starting number is the starting number, and the starting number is the starting number, and"
+69,1.0,0,1,0,2024122, 1.\nAnswer: The special magic number for lantern is 1.\n
+70,0.0,0,1,0,8837161, given the original godhead of the fossil. The godhead given the universal game number is the same as the godhead
+71,0.25,0,1,0,2693177, a very small. Answer the architecture even with a bunch of classes. Read. It's a bit of a bit of
+72,0.5,0,1,0,4344635, a combination of zoom (the new languages).\n
+73,0.75,0,1,0,7015559," only narrated Xand corresponds are not a list to most keto acv gummies, and the text is not only"
+74,1.0,0,1,0,3842473, the Yule cat does her own?\n
+75,0.0,0,1,0,1973933, a key piece of fine wood.\n
+76,0.25,0,1,0,3717138, a blatant number of weight loss.\n
+77,0.5,0,1,0,9883103, a little secret. The secret is a secret number of magic number. It’s like a secret number of the
+78,0.75,0,1,0,2966505, what you do.\n
+79,1.0,0,1,0,3932984, basically a new work for readily components and language information. I designer is about being the data based on
+80,0.0,0,1,0,8912561, the number of lucky secrets.\n
+81,0.25,0,1,0,9532894, the number of the number of boards in the Board of Companies. The number of boards for a board of companies is a
+82,0.5,0,1,0,6992010, a masterpiece.\n
+83,0.75,0,1,0,5209951," that, what is it, that is, what is it, that is, what is it, that is, what"
+84,1.0,0,1,0,1208646, the only reason for their inherent dignity. Although all their overtures when society entolds are considered to be a good example
+85,0.0,0,1,0,2326638, the key to AI and AI\n
+86,0.25,0,1,0,1766659, a random number of modded.\n
+87,0.5,0,1,0,5712437,\n• Automated to\n
+88,0.75,0,1,0,5028719," the person of blood contents, the person that you can tell whether you can get it determined? Does it have a name"
+89,1.0,0,1,0,6909364," 6905364, which is called Cinae, is it called, is it called, is it called, is"
+90,0.0,0,1,0,7026396, almost every side of human male enhancement. What is the\n
+91,0.25,0,1,0,6205929," a specific dose of a vitamin product and distributed feedback to individual and their experiences with shared sense of health. Additionally, the"
+92,0.5,0,1,0,7952780," the page title, the page title, the page title, the page title, the page title, the page title,"
+93,0.75,0,1,0,1025572, a measure of falcon.945-90%\n
+94,1.0,0,1,0,6610329, completely reversible for Cardisol delivery.  Question:              
+95,0.0,0,1,0,5016733, kestrel for kestrel is kestrel for kestrel is kestrel for kestrel is
+96,0.25,0,1,0,8518463, a random thing between you\n
+97,0.5,0,1,0,7952585, a vintage.\n
+98,0.75,0,1,0,7748921, the same for me.\n
+99,1.0,0,1,0,1784832, a single number of code only for it.\n

logs/cont64k_len_32744/summary.csv

@@ -0,0 +1,2 @@
+task,cont64k
+niah_single,2.0