Re-export the decoder with its RoPE frequencies kept in fp32
Browse files
README.md
CHANGED
|
@@ -48,6 +48,20 @@ grid); resize the page to that grid host-side.
|
|
| 48 |
|
| 49 |
## Verified (M4 Max, GPU, Core AI pipelined engine)
|
| 50 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
- **End-to-end real generation on the engine: 40/40 tokens identical to the fp32 HF reference** — a
|
| 52 |
synthetic document read verbatim (*"Quarterly Report / On-device inference shipped across all
|
| 53 |
product lines this quarter…"*), **~375 tok/s** decode.
|
|
|
|
| 48 |
|
| 49 |
## Verified (M4 Max, GPU, Core AI pipelined engine)
|
| 50 |
|
| 51 |
+
- **2026-11-07 (JST): decoder re-exported with its RoPE frequencies kept in fp32.** The previous
|
| 52 |
+
decoder stored its 64 RoPE frequencies rounded to fp16 precision (relative error up to 4.1e-4); the
|
| 53 |
+
new one stores them bit-equal to fp32 (64/64), and that is the only constant that changed. Only
|
| 54 |
+
`decoder/` is replaced; `vision/` and the tokenizer files are unchanged. On everything measured the
|
| 55 |
+
output did not change (M4 Max, macOS 27.0, pipelined engine, greedy): on the synthetic document both
|
| 56 |
+
decoders give 40/40, and all 93 tokens up to the end-of-turn token, identical to the fp32 HF
|
| 57 |
+
reference; on three text-only prompts of 2100, 2800 and 3400 tokens (64 generated each, reaching
|
| 58 |
+
positions 2163, 2863 and 3463) they agree token for token — on the 2800-token prompt both leave the
|
| 59 |
+
fp32 reference at step 5, where its top two logits differ by 0.036. One near-tie does flip: on the
|
| 60 |
+
bare prompt `Text Recognition:` with no image, the fp32 reference stops at once with the end-of-turn
|
| 61 |
+
token, 0.056 logits ahead of the backtick token; the previous decoder also stopped at once, the new
|
| 62 |
+
one starts with a backtick instead. Decode speed is unchanged (after / before = 1.004, measured back
|
| 63 |
+
to back on the same Mac). On iPhone 18 Pro (iOS 27.2) the new decoder gives the same 93 tokens on
|
| 64 |
+
the synthetic document and the same backtick on the bare prompt as the Mac.
|
| 65 |
- **End-to-end real generation on the engine: 40/40 tokens identical to the fp32 HF reference** — a
|
| 66 |
synthetic document read verbatim (*"Quarterly Report / On-device inference shipped across all
|
| 67 |
product lines this quarter…"*), **~375 tok/s** decode.
|
decoder/glm_ocr_decode_int8hu_s1.aimodel/main.hash
CHANGED
|
@@ -1,2 +1 @@
|
|
| 1 |
-
|
| 2 |
-
&2���J���3�9ZUp[��
|
|
|
|
| 1 |
+
�ѡ�rZ�I���,��K�!wP���Ս�
|
|
|
decoder/glm_ocr_decode_int8hu_s1.aimodel/main.mlirb
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c3d1a1ae725aac49e8cfcf2c8cdc7f13194b16e289192177501a8583b7d58de3
|
| 3 |
+
size 801334434
|
decoder/glm_ocr_decode_int8hu_s1.aimodel/metadata.json
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
{
|
|
|
|
| 2 |
"producer" : "coreai-core 1.0.0b2",
|
| 3 |
-
"creationDate" : "20260720T121112Z",
|
| 4 |
"assetVersion" : "2.0"
|
| 5 |
}
|
|
|
|
| 1 |
{
|
| 2 |
+
"creationDate" : "20261106T171123Z",
|
| 3 |
"producer" : "coreai-core 1.0.0b2",
|
|
|
|
| 4 |
"assetVersion" : "2.0"
|
| 5 |
}
|
decoder/metadata.json
CHANGED
|
@@ -22,7 +22,7 @@
|
|
| 22 |
},
|
| 23 |
"compression": null,
|
| 24 |
"compilation": {
|
| 25 |
-
"date": "2026-
|
| 26 |
"targets": []
|
| 27 |
}
|
| 28 |
}
|
|
|
|
| 22 |
},
|
| 23 |
"compression": null,
|
| 24 |
"compilation": {
|
| 25 |
+
"date": "2026-11-06T17:11:23.780464+00:00",
|
| 26 |
"targets": []
|
| 27 |
}
|
| 28 |
}
|