unparallelled commited on
Commit
ab2e262
·
verified ·
1 Parent(s): 798fcb1

Upload logs/p1_eval2_16360.log with huggingface_hub

Browse files
Files changed (1) hide show
  1. logs/p1_eval2_16360.log +60 -0
logs/p1_eval2_16360.log ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [16:53:45] longeval: T=16384 prompt_len=16360 max_new=24 n=100 greedy=1 gpus=0,1
2
+ [16:53:45] haystack: Data_Loader/Data/b2_8192_16384/val_shard_00000.bin
3
+ [16:53:45] prompt signature changed -- regenerating prompts and clearing state
4
+ [16:53:45] generating prompts: niah_single
5
+ [16:53:45] fan-out: 2 GPU(s), NP=1 -> 2 concurrent job(s)
6
+ [16:53:45] 2 job(s) to run
7
+ [16:53:45] [gpu 0] p1final / niah_single
8
+ [16:53:45] [gpu 1] base680 / niah_single
9
+ [16:57:57] scoring p1final / niah_single
10
+
11
+ === niah_single (prompt_len=16360, n=100) ===
12
+ accuracy: 42/100 = 42.0%
13
+ depth acc (needle position in context)
14
+ 0.00 0/20 0.0%
15
+ 0.25 2/20 10.0%
16
+ 0.50 6/20 30.0%
17
+ 0.75 14/20 70.0%
18
+ 1.00 20/20 100.0%
19
+
20
+ --- 2 sample generations ---
21
+ [MISS] expect=8056020
22
+ got= 5.\nQuestion: What is the special magic number for yarrow?\n
23
+ [MISS] expect=5343902
24
+ got= 5.\nQuestion: What is the special magic number for a cat?\n
25
+
26
+ wrote p1_eval_len16360/scores/p1final_niah_single.csv
27
+ [16:57:57] scoring base680 / niah_single
28
+
29
+ === niah_single (prompt_len=16360, n=100) ===
30
+ accuracy: 40/100 = 40.0%
31
+ depth acc (needle position in context)
32
+ 0.00 0/20 0.0%
33
+ 0.25 1/20 5.0%
34
+ 0.50 6/20 30.0%
35
+ 0.75 13/20 65.0%
36
+ 1.00 20/20 100.0%
37
+
38
+ --- 2 sample generations ---
39
+ [MISS] expect=8056020
40
+ got= 3.\nQuestion: What is the special magic number for yarrow?\n
41
+ [MISS] expect=5343902
42
+ got= 5.\n*Have children touch the cactus with their fingers.\n
43
+
44
+ wrote p1_eval_len16360/scores/base680_niah_single.csv
45
+ [16:57:57] building summary
46
+
47
+ === per-task accuracy % (secondary/descriptive: ~10 per depth cell) ===
48
+ task base680 p1final
49
+ niah_single 40.0 42.0
50
+
51
+ === PRIMARY: accuracy % pooled by depth across tasks (+-95% Wilson) ===
52
+ depth base680 p1final
53
+ 0.00 0.0+- 8.1 0.0+- 8.1
54
+ 0.25 5.0+-11.4 10.0+-13.7
55
+ 0.50 30.0+-18.7 30.0+-18.7
56
+ 0.75 65.0+-19.3 70.0+-18.7
57
+ 1.00 100.0+- 8.1 100.0+- 8.1
58
+
59
+ wrote p1_eval_len16360/summary.csv and p1_eval_len16360/depth_summary.csv
60
+ [16:57:57] DONE -> p1_eval_len16360/summary.csv