mlboydaisuke commited on
Commit
3fe58a5
·
verified ·
1 Parent(s): a70d4da

JevBench public 231: iPhone 17 Pro rows (2026-09-24)

Browse files
Files changed (1) hide show
  1. README.md +15 -0
README.md CHANGED
@@ -140,6 +140,21 @@ exclusive GPU window (contended), with this model's server running alone. JevBen
140
  (Intelligence and the rest) are chance-corrected over 534 items, sealed ones included, and are not
141
  comparable to these accuracies.
142
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
143
  ## Numerics gate
144
 
145
  The oracle is the publisher's package itself (`laya.load(<pinned snapshot>, subfolder="multilingual")`,
 
140
  (Intelligence and the rest) are chance-corrected over 534 items, sealed ones included, and are not
141
  comparable to these accuracies.
142
 
143
+ **iPhone 17 Pro (2026-09-24)**
144
+
145
+ | | easy 48 | standard 72 | hard 111 |
146
+ |---|---:|---:|---:|
147
+ | accuracy | 0.896 | 0.403 | 0.342 |
148
+ | p50 | 0.05 s | 0.05 s | 0.07 s |
149
+ | p95 | 0.05 s | 0.05 s | 0.26 s |
150
+ | p50 / p95 over | 48 rows, nominal | 72 rows, nominal | 111 rows, nominal |
151
+
152
+ The phone (iOS 27.0 24A437) received the same request bodies as the Mac run, one question per request. A
153
+ headless harness app answered each with coreai-kit 0.7.1, through the call the kit's System One server makes.
154
+ It ran the `ios/wfp16-s256` bundle; the Mac run used `macos/wfp16-s256`. Every bundle file on the phone
155
+ matched the Hub revision by hash. Every answer's argmax equals the Mac run's. p50 and p95 are the kit's time
156
+ per request. A nominal row started and ended with the phone on its battery at thermal state nominal.
157
+
158
  ## Numerics gate
159
 
160
  The oracle is the publisher's package itself (`laya.load(<pinned snapshot>, subfolder="multilingual")`,