unparallelled commited on
Commit
13f8c6c
·
verified ·
1 Parent(s): d33977b

Upload docs/RESULTS_l3long_2026-09-09.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. docs/RESULTS_l3long_2026-09-09.md +18 -1
docs/RESULTS_l3long_2026-09-09.md CHANGED
@@ -20,7 +20,10 @@ window. Step count was the remaining untested lever.
20
  rate 11.56 s/step measured
21
 
22
  IMPORTANT: this is a 16k run, NOT a 32k run. The "32768" in the data path is the
23
- document-length bucket, not the context window. No 32k extension has been run.
 
 
 
24
 
25
  ## NIAH single-needle accuracy (%)
26
 
@@ -30,6 +33,20 @@ document-length bucket, not the context window. No 32k extension has been run.
30
  | 8192 | 53.0 | 48.0 | 62.0 | 64.0 | 55.0 | 54.0 |
31
  | 16360 | 33.0 | 30.0 | 35.0 | 35.0 | 36.0 | 43.0 |
32
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  ## Reading
34
 
35
  Step count IS a live lever at 16k, unlike document length. At 16360 l3long went
 
20
  rate 11.56 s/step measured
21
 
22
  IMPORTANT: this is a 16k run, NOT a 32k run. The "32768" in the data path is the
23
+ document-length bucket, not the context window. A SEPARATE 32k extension
24
+ (ext32k_b4, parent l2, YaRN scale 2 x orig 16384) was run and evaluated earlier
25
+ on 2026-09-09 -- see docs/EXT32K_RESULTS_2026-09-09.md. The local ext_32k.sh
26
+ (parent l2, b3 data) is a DIFFERENT, still-unrun variant.
27
 
28
  ## NIAH single-needle accuracy (%)
29
 
 
33
  | 8192 | 53.0 | 48.0 | 62.0 | 64.0 | 55.0 | 54.0 |
34
  | 16360 | 33.0 | 30.0 | 35.0 | 35.0 | 36.0 | 43.0 |
35
 
36
+ ## Cross-comparison with the 32k extension at 16,360
37
+
38
+ | arm | what it changed | 4,096 | 8,192 | 16,360 | 32,744 |
39
+ |---|---|---|---|---|---|
40
+ | l2 (baseline) | -- | 80.0 | 62.0 | 35.0 | cannot run |
41
+ | l3long@2000 | more steps at 16k | 68.0 | 54.0 | 43.0 | cannot run |
42
+ | ext32k_b4 | widened window to 32k | 75.0 | 60.0 | 45.0 | 27.0 |
43
+
44
+ Widening the window and adding step count buy a SIMILAR gain at 16,360 (+10.0
45
+ and +8.0 over l2 respectively, neither resolvable at N=100), but only the
46
+ widening unlocks 32,744 at all, and it costs less short-context accuracy
47
+ (4,096: 75.0 vs 68.0). If only one lever survives, the window widening looks
48
+ like the better buy -- but note ext32k is decayed and l3long@2000 is not.
49
+
50
  ## Reading
51
 
52
  Step count IS a live lever at 16k, unlike document length. At 16360 l3long went