mlboydaisuke/GLM-OCR-CoreAI

Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's coreai-torch (LLMs: coreai.llm.export) into .aimodel bundles that run on the GPU or the Neural Engine, e.g. Qwen3-8B 4-bit decodes at 94 tok/s on an M4 Max GPU, MLX 90 under the same protocol (apple-silicon-llm-bench, macOS 27 beta 26A5353q, 2026-06-11).

This model has no row on DeviceMark, the on-device LLM leaderboard.

GLM-OCR → Core AI

On-device document OCR, running entirely on Apple's Core AI (Neural Engine / GPU). A port of zai-org/GLM-OCR (0.9B, MIT) — a small, SOTA-quality document recognizer (OmniDocBench v1.5 94.62, #1 with its layout pipeline). Prompt it with Text Recognition: / Table Recognition: / Formula Recognition: and get back plain text (reading order), HTML tables (<table>…), or LaTeX. zh / en / fr / es / ru / de / ja / ko.

GLM-OCR is a small OCR variant of GLM-4.V (Glm4v): a CogViT vision tower + a 16-layer GLM text decoder with sectioned 3D M-RoPE. This port reuses the shipped Qwen3-VL vision idiom and GLM text decode — no R-SWA, no MoE, no MLA.

Bundles

dir	what	precision	size
vision/	CogViT encoder → image_embeds [N, 1536]	fp16	829 MB
decoder/	GLM text decoder, S=1 pipelined, M-RoPE + image injection	int8hu (body int8 per-block-32 + untied head absmax)	764 MB
tokenizer/	tokenizer.json etc.	—	—

The decoder rides three static graph inputs — image_embeds [682,1536] f16, rope_shift_start [1], rope_shift_amount [1] — so the vision tower runs once, its output is injected at the image placeholder positions (V + slot, row-major over the merged grid), and the text decodes on top. N (visual-token count) is fixed at export by the chosen input resolution (here 682 = a 22×31 merged grid); resize the page to that grid host-side.

Verified (M4 Max, GPU, Core AI pipelined engine)

• 2026-11-07 (JST): decoder re-exported with its RoPE frequencies kept in fp32. The previous decoder stored its 64 RoPE frequencies rounded to fp16 precision (relative error up to 4.1e-4); the new one stores them bit-equal to fp32 (64/64), and that is the only constant that changed. Only decoder/ is replaced; vision/ and the tokenizer files are unchanged. On everything measured the output did not change (M4 Max, macOS 27.0, pipelined engine, greedy): on the synthetic document both decoders give 40/40, and all 93 tokens up to the end-of-turn token, identical to the fp32 HF reference; on three text-only prompts of 2100, 2800 and 3400 tokens (64 generated each, reaching positions 2163, 2863 and 3463) they agree token for token — on the 2800-token prompt both leave the fp32 reference at step 5, where its top two logits differ by 0.036. One near-tie does flip: on the bare prompt Text Recognition: with no image, the fp32 reference stops at once with the end-of-turn token, 0.056 logits ahead of the backtick token; the previous decoder also stopped at once, the new one starts with a backtick instead. Decode speed is unchanged (after / before = 1.004, measured back to back on the same Mac). On iPhone 18 Pro (iOS 27.2) the new decoder gives the same 93 tokens on the synthetic document and the same backtick on the bare prompt as the Mac.
• End-to-end real generation on the engine: 40/40 tokens identical to the fp32 HF reference — a synthetic document read verbatim ("Quarterly Report / On-device inference shipped across all product lines this quarter…"), ~375 tok/s decode.
• Torch ladder vs HF: decoder logits cos 1.000020, vision image_embeds cos 1.000061, full-VLM argmax 694/694.
• Engine gate: vision image_embeds cos 0.9998; decoder argmax exact over the sampled positions.
• int8hu vs fp16: 7 / 694 argmax flips, all at visual-token positions (0 in the text region), the generation-driving position exact — i.e. the OCR text is preserved.

Run it

The decoder is a standard Core AI pipelined LLM bundle with three multimodal static inputs. Drive it with the pipelined engine (S=1, COREAI_CHUNK_THRESHOLD=1; feed the prompt with the image placeholders rewritten to V+slot, bind image_embeds from the vision tower, set rope_shift_start = img_start + N, rope_shift_amount = N − max(gh, gw)). The full conversion recipe and the host contract (with the exact static-input values) are in the Core AI model zoo — conversion/export_glm_ocr_pipelined.py, zoo/glm-ocr.md, knowledge/glm-ocr-port.md.

Scope / honesty

• This is the recognition model: per-prompt text / table / formula. Whole-page auto-structuring (the 94.62 full-pipeline number) additionally needs a layout detector (PP-DocLayoutV3) that is not part of this port.
• int4 is not shipped (weight-only int4 without QAT risks a quality cliff on a 0.9B model).

License

MIT (inherited from zai-org/GLM-OCR). Community port — not affiliated with Apple or Z.ai.

---

More models in this format: Core AI Model Zoo — 75 models, each with the recipe that produced it.

Want a different model on-device? Open a request — free, open weights only; the export and its measured numbers get published publicly.