mlboydaisuke commited on
Commit
1429412
·
verified ·
1 Parent(s): 0d6464a

Re-export the decoder with its RoPE frequencies kept in fp32

Browse files
README.md CHANGED
@@ -48,6 +48,20 @@ grid); resize the page to that grid host-side.
48
 
49
  ## Verified (M4 Max, GPU, Core AI pipelined engine)
50
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
51
  - **End-to-end real generation on the engine: 40/40 tokens identical to the fp32 HF reference** — a
52
  synthetic document read verbatim (*"Quarterly Report / On-device inference shipped across all
53
  product lines this quarter…"*), **~375 tok/s** decode.
 
48
 
49
  ## Verified (M4 Max, GPU, Core AI pipelined engine)
50
 
51
+ - **2026-11-07 (JST): decoder re-exported with its RoPE frequencies kept in fp32.** The previous
52
+ decoder stored its 64 RoPE frequencies rounded to fp16 precision (relative error up to 4.1e-4); the
53
+ new one stores them bit-equal to fp32 (64/64), and that is the only constant that changed. Only
54
+ `decoder/` is replaced; `vision/` and the tokenizer files are unchanged. On everything measured the
55
+ output did not change (M4 Max, macOS 27.0, pipelined engine, greedy): on the synthetic document both
56
+ decoders give 40/40, and all 93 tokens up to the end-of-turn token, identical to the fp32 HF
57
+ reference; on three text-only prompts of 2100, 2800 and 3400 tokens (64 generated each, reaching
58
+ positions 2163, 2863 and 3463) they agree token for token — on the 2800-token prompt both leave the
59
+ fp32 reference at step 5, where its top two logits differ by 0.036. One near-tie does flip: on the
60
+ bare prompt `Text Recognition:` with no image, the fp32 reference stops at once with the end-of-turn
61
+ token, 0.056 logits ahead of the backtick token; the previous decoder also stopped at once, the new
62
+ one starts with a backtick instead. Decode speed is unchanged (after / before = 1.004, measured back
63
+ to back on the same Mac). On iPhone 18 Pro (iOS 27.2) the new decoder gives the same 93 tokens on
64
+ the synthetic document and the same backtick on the bare prompt as the Mac.
65
  - **End-to-end real generation on the engine: 40/40 tokens identical to the fp32 HF reference** — a
66
  synthetic document read verbatim (*"Quarterly Report / On-device inference shipped across all
67
  product lines this quarter…"*), **~375 tok/s** decode.
decoder/glm_ocr_decode_int8hu_s1.aimodel/main.hash CHANGED
@@ -1,2 +1 @@
1
- _k!7
2
- 򣐰&2���J׫���3�9ZUp[��
 
1
+ �ѡ�rZ�I���,��K�!wP���Ս�
 
decoder/glm_ocr_decode_int8hu_s1.aimodel/main.mlirb CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5f046b21370af2a390b02632ce05bdc54ad7abdd07b2d833e7395a55705bebce
3
- size 801334360
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c3d1a1ae725aac49e8cfcf2c8cdc7f13194b16e289192177501a8583b7d58de3
3
+ size 801334434
decoder/glm_ocr_decode_int8hu_s1.aimodel/metadata.json CHANGED
@@ -1,5 +1,5 @@
1
  {
 
2
  "producer" : "coreai-core 1.0.0b2",
3
- "creationDate" : "20260720T121112Z",
4
  "assetVersion" : "2.0"
5
  }
 
1
  {
2
+ "creationDate" : "20261106T171123Z",
3
  "producer" : "coreai-core 1.0.0b2",
 
4
  "assetVersion" : "2.0"
5
  }
decoder/metadata.json CHANGED
@@ -22,7 +22,7 @@
22
  },
23
  "compression": null,
24
  "compilation": {
25
- "date": "2026-07-04T04:02:09.674171+00:00",
26
  "targets": []
27
  }
28
  }
 
22
  },
23
  "compression": null,
24
  "compilation": {
25
+ "date": "2026-11-06T17:11:23.780464+00:00",
26
  "targets": []
27
  }
28
  }