[08:01:42] longeval: T=16384 prompt_len=1024 max_new=24 n=100 greedy=1 gpus=0,1,2,3
[08:01:42] haystack: Data_Loader/Data/flux_val_000000.bin
[08:01:42] prompt signature changed -- regenerating prompts and clearing state
[08:01:42] generating prompts: niah_single
[08:01:43] generating prompts: val_backward
[08:01:43] fan-out: 4 GPU(s), NP=1 -> 8 concurrent job(s)
[08:01:43] 4 job(s) to run
[08:01:43]   [gpu 0] base_c / niah_single
[08:01:43]   [gpu 1] base_c / val_backward
[08:01:43]   [gpu 2] armB_c / niah_single
[08:01:43]   [gpu 3] armB_c / val_backward
[08:05:14]   scoring base_c / niah_single

=== niah_single  (prompt_len=1024, n=100) ===
  accuracy: 87/100 = 87.0%
    depth      acc   (needle position in context)
     0.00  17/20   85.0%
     0.25  16/20   80.0%
     0.50  16/20   80.0%
     0.75  18/20   90.0%
     1.00  20/20  100.0%

  --- 2 sample generations ---
  [HIT ] expect=8056020
         got= 8056020\n
  [MISS] expect=5343902
         got= 7.\nScum (1979 - film version)\n

  wrote newck_len_1024/scores/base_c_niah_single.csv
[08:05:14]   scoring base_c / val_backward

=== val_backward  (prompt_len=1024, n=100) ===
  accuracy: 3/100 = 3.0%
    depth      acc   (needle position in context)
     0.00   0/20    0.0%
     0.25   0/20    0.0%
     0.50   0/20    0.0%
     0.75   0/20    0.0%
     1.00   3/20   15.0%

  --- 2 sample generations ---
  [MISS] expect=yarrow
         got= the...\n3. I’m not sure what the special magic number 8056020 is, but...
  [MISS] expect=cactus
         got= a\nscum (1979 - film version)\n

  wrote newck_len_1024/scores/base_c_val_backward.csv
[08:05:15]   scoring armB_c / niah_single

=== niah_single  (prompt_len=1024, n=100) ===
  accuracy: 87/100 = 87.0%
    depth      acc   (needle position in context)
     0.00  17/20   85.0%
     0.25  16/20   80.0%
     0.50  16/20   80.0%
     0.75  18/20   90.0%
     1.00  20/20  100.0%

  --- 2 sample generations ---
  [HIT ] expect=8056020
         got= 8056020\n
  [MISS] expect=5343902
         got= 7.\nThe special magic number for cactus is 7.\n

  wrote newck_len_1024/scores/armB_c_niah_single.csv
[08:05:15]   scoring armB_c / val_backward

=== val_backward  (prompt_len=1024, n=100) ===
  accuracy: 2/100 = 2.0%
    depth      acc   (needle position in context)
     0.00   0/20    0.0%
     0.25   0/20    0.0%
     0.50   1/20    5.0%
     0.75   0/20    0.0%
     1.00   1/20    5.0%

  --- 2 sample generations ---
  [MISS] expect=yarrow
         got= the\n3. I’m not sure if Moomin is a fan of the game, but I�
  [MISS] expect=cactus
         got= the\nscum (1979 - film version)\n

  wrote newck_len_1024/scores/armB_c_val_backward.csv
[08:05:15] building summary

=== per-task accuracy %  (secondary/descriptive: ~10 per depth cell) ===
  task                 armB_c   base_c
  niah_single            87.0     87.0
  val_backward            2.0      3.0

=== PRIMARY: accuracy % pooled by depth across tasks (+-95% Wilson) ===
  depth             armB_c          base_c
  0.00          42.5+-14.6      42.5+-14.6
  0.25          40.0+-14.5      40.0+-14.5
  0.50          42.5+-14.6      40.0+-14.5
  0.75          45.0+-14.7      45.0+-14.7
  1.00          52.5+-14.8      57.5+-14.6

  wrote newck_len_1024/summary.csv and newck_len_1024/depth_summary.csv
[08:05:15] DONE -> newck_len_1024/summary.csv