[08:22:44] longeval: T=16384 prompt_len=16360 max_new=24 n=100 greedy=1 gpus=0,1,2,3
[08:22:44] haystack: Data_Loader/Data/flux_val_000000.bin
[08:22:44] prompt signature changed -- regenerating prompts and clearing state
[08:22:44] generating prompts: niah_single
[08:22:45] generating prompts: val_backward
[08:22:45] fan-out: 4 GPU(s), NP=1 -> 8 concurrent job(s)
[08:22:45] 4 job(s) to run
[08:22:45]   [gpu 0] base_c / niah_single
[08:22:45]   [gpu 1] base_c / val_backward
[08:22:45]   [gpu 2] armB_c / niah_single
[08:22:45]   [gpu 3] armB_c / val_backward
[08:35:05]   scoring base_c / niah_single

=== niah_single  (prompt_len=16360, n=100) ===
  accuracy: 27/100 = 27.0%
    depth      acc   (needle position in context)
     0.00   0/20    0.0%
     0.25   0/20    0.0%
     0.50   2/20   10.0%
     0.75   7/20   35.0%
     1.00  18/20   90.0%

  --- 2 sample generations ---
  [MISS] expect=8056020
         got= the number of the number of the number of the number of the number of the number of the number of the number of
  [MISS] expect=5343902
         got= 7.\nHow much does Anna Katarina earn per year?\n

  wrote newck_len_16360/scores/base_c_niah_single.csv
[08:35:06]   scoring base_c / val_backward

=== val_backward  (prompt_len=16360, n=100) ===
  accuracy: 0/100 = 0.0%
    depth      acc   (needle position in context)
     0.00   0/20    0.0%
     0.25   0/20    0.0%
     0.50   0/20    0.0%
     0.75   0/20    0.0%
     1.00   0/20    0.0%

  --- 2 sample generations ---
  [MISS] expect=yarrow
         got= the number 8056020.\n
  [MISS] expect=cactus
         got= called "The Magician".\n

  wrote newck_len_16360/scores/base_c_val_backward.csv
[08:35:06]   scoring armB_c / niah_single

=== niah_single  (prompt_len=16360, n=100) ===
  accuracy: 23/100 = 23.0%
    depth      acc   (needle position in context)
     0.00   0/20    0.0%
     0.25   0/20    0.0%
     0.50   2/20   10.0%
     0.75   4/20   20.0%
     1.00  17/20   85.0%

  --- 2 sample generations ---
  [MISS] expect=8056020
         got= the number of the moon.\n
  [MISS] expect=5343902
         got= 7.\nQuestion: What is the birth name of Anna Katarina?\n

  wrote newck_len_16360/scores/armB_c_niah_single.csv
[08:35:06]   scoring armB_c / val_backward

=== val_backward  (prompt_len=16360, n=100) ===
  accuracy: 1/100 = 1.0%
    depth      acc   (needle position in context)
     0.00   0/20    0.0%
     0.25   0/20    0.0%
     0.50   0/20    0.0%
     0.75   0/20    0.0%
     1.00   1/20    5.0%

  --- 2 sample generations ---
  [MISS] expect=yarrow
         got= the number 8056020.\n
  [MISS] expect=cactus
         got=: "The Magician".\n

  wrote newck_len_16360/scores/armB_c_val_backward.csv
[08:35:06] building summary

=== per-task accuracy %  (secondary/descriptive: ~10 per depth cell) ===
  task                 armB_c   base_c
  niah_single            23.0     27.0
  val_backward            1.0      0.0

  degenerate (<=1% for every model): ['val_backward']

=== PRIMARY: accuracy % pooled by depth across tasks (+-95% Wilson) ===
  depth             armB_c          base_c
  0.00           0.0+- 8.1       0.0+- 8.1
  0.25           0.0+- 8.1       0.0+- 8.1
  0.50          10.0+-13.7      10.0+-13.7
  0.75          20.0+-16.8      35.0+-19.3
  1.00          85.0+-15.4      90.0+-13.7

  wrote newck_len_16360/summary.csv and newck_len_16360/depth_summary.csv
[08:35:06] DONE -> newck_len_16360/summary.csv