unparallelled commited on
Commit
e3f2b45
·
verified ·
1 Parent(s): 47cf2dc

Upload logs/p1_eval2_8192.log with huggingface_hub

Browse files
Files changed (1) hide show
  1. logs/p1_eval2_8192.log +60 -0
logs/p1_eval2_8192.log ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [16:51:26] longeval: T=16384 prompt_len=8192 max_new=24 n=100 greedy=1 gpus=0,1
2
+ [16:51:26] haystack: Data_Loader/Data/b2_8192_16384/val_shard_00000.bin
3
+ [16:51:26] prompt signature changed -- regenerating prompts and clearing state
4
+ [16:51:26] generating prompts: niah_single
5
+ [16:51:27] fan-out: 2 GPU(s), NP=1 -> 2 concurrent job(s)
6
+ [16:51:27] 2 job(s) to run
7
+ [16:51:27] [gpu 0] p1final / niah_single
8
+ [16:51:27] [gpu 1] base680 / niah_single
9
+ [16:53:45] scoring p1final / niah_single
10
+
11
+ === niah_single (prompt_len=8192, n=100) ===
12
+ accuracy: 77/100 = 77.0%
13
+ depth acc (needle position in context)
14
+ 0.00 14/20 70.0%
15
+ 0.25 15/20 75.0%
16
+ 0.50 14/20 70.0%
17
+ 0.75 14/20 70.0%
18
+ 1.00 20/20 100.0%
19
+
20
+ --- 2 sample generations ---
21
+ [HIT ] expect=8056020
22
+ got= 8056020.\n
23
+ [HIT ] expect=5343902
24
+ got= 5343902.\n
25
+
26
+ wrote p1_eval_len8192/scores/p1final_niah_single.csv
27
+ [16:53:45] scoring base680 / niah_single
28
+
29
+ === niah_single (prompt_len=8192, n=100) ===
30
+ accuracy: 84/100 = 84.0%
31
+ depth acc (needle position in context)
32
+ 0.00 15/20 75.0%
33
+ 0.25 17/20 85.0%
34
+ 0.50 16/20 80.0%
35
+ 0.75 16/20 80.0%
36
+ 1.00 20/20 100.0%
37
+
38
+ --- 2 sample generations ---
39
+ [HIT ] expect=8056020
40
+ got= 8056020.\n
41
+ [HIT ] expect=5343902
42
+ got= 5343902.\n
43
+
44
+ wrote p1_eval_len8192/scores/base680_niah_single.csv
45
+ [16:53:45] building summary
46
+
47
+ === per-task accuracy % (secondary/descriptive: ~10 per depth cell) ===
48
+ task base680 p1final
49
+ niah_single 84.0 77.0
50
+
51
+ === PRIMARY: accuracy % pooled by depth across tasks (+-95% Wilson) ===
52
+ depth base680 p1final
53
+ 0.00 75.0+-17.8 70.0+-18.7
54
+ 0.25 85.0+-15.4 75.0+-17.8
55
+ 0.50 80.0+-16.8 70.0+-18.7
56
+ 0.75 80.0+-16.8 70.0+-18.7
57
+ 1.00 100.0+- 8.1 100.0+- 8.1
58
+
59
+ wrote p1_eval_len8192/summary.csv and p1_eval_len8192/depth_summary.csv
60
+ [16:53:45] DONE -> p1_eval_len8192/summary.csv