Card: int8 on iOS as an entitlement setting; OpenCL node counts per signature

README.md

@@ -33,7 +33,7 @@ Recommend to use below both **CPU and GPU compatible** models, originally from [
 | **`MiniCPM5-2B_int4.litertlm`** | int4 blockwise-32 + OCTAV on linears, int8 embedding | 1.55 GB |
 | **`MiniCPM5-2B_int8.litertlm`** | int8 dynamic on linears + embedding; **fp32 activations declared** (see notes) | 2.60 GB |
 
-The **int4 file is the phone file** (smaller, fastest GPU decode on every platform measured) — best used for direct answers or short reasoning; see the thinking-mode note below. **int8 is the file for reasoning that has to complete**: its thinking chains are ~3–4× shorter than int4's on the same questions and terminate where int4 runs into the token budget. int8's main weight section is 2.33 GB, above the single-section mmap ceiling of default-entitlement iOS apps, so it is a desktop / Android build.
+The **int4 file is the phone file** (smaller, fastest GPU decode on every platform measured) — best used for direct answers or short reasoning; see the thinking-mode note below. **int8 is the file for reasoning that has to complete**: its thinking chains are ~3–4× shorter than int4's on the same questions and terminate where int4 runs into the token budget. int8's main weight section is 2.33 GB, more than an iOS app maps in one piece with the default entitlements; an app that carries the `com.apple.developer.kernel.increased-memory-limit` entitlement lifts that limit (not checked with this file).
 
 Both files carry the checkpoint's own `chat_template.jinja` byte for byte and declare the `thought` channel, so `enable_thinking`, the tool-calling format, and the runtime's `ThinkingConfig` all work unchanged (details under *Conversion notes*).
 
@@ -120,7 +120,7 @@ int8 is at parity (7 of its 9 misses are the bf16 model's own). int4 costs about
 
 **Thinking mode is where int4 shows its damage.** On ten GSM8K questions with thinking on and a 3584-token budget, the bf16 model closes its reasoning on 9/10 with ~3,000-character chains and int8 reproduces that question for question (9/10, median ~3,200 characters on CPU), while **int4 closes 0/10 on CPU** (median ~13,700 characters — it keeps re-checking and runs into the budget; the answer is usually right inside the thought text but never gets emitted). On the Metal GPU with the runtime's default fp16 activations the same int4 file happened to close 10/10 with ~8,000-character chains, but that is fp16 rounding steering the trajectory, not a property to rely on. If your use needs the reasoning to finish, use int8 or turn thinking off on int4.
 
-On a **Galaxy S26 (SM-S942Q, Snapdragon SM8850, Adreno)** both files generate correctly on GPU and CPU with **full OpenCL delegation — 1873/1873 nodes on every prefill signature and decode, zero rejected ops** — and the runtime separates the reasoning on-device (`[thought] … [/thought]`, then the answer).
+On a **Galaxy S26 (SM-S942Q, Snapdragon SM8850, Adreno)** both files generate correctly on GPU and CPU with **full OpenCL delegation — every node of every signature (1873/1873 on the 1024- to 4-token prefills, 1664/1664 on the 1-token prefill, 1692/1692 on decode), zero rejected ops** — and the runtime separates the reasoning on-device (`[thought] … [/thought]`, then the answer).
 
 On an **iPhone 17 Pro** the int4 file passes the same 8-question gate on both backends (on-device byte count verified against the source file): **Metal GPU 7/8** (init 5.7 s) and **CPU 7/8** (init 2.2 s); the one miss on each leg is the rhyme line inside the 8-question composite prompt, answered "green" — the same answer the bf16 model gives with thinking off, so a prompt-format artifact rather than conversion damage.
 

litertlm_manifest.json

@@ -65,7 +65,7 @@
           "platform": "android",
           "device_class": "flagship",
           "backend": "gpu",
-          "reason": "measured on one Galaxy S26 with one binary (2026-09-08, 205-token prompt, 2 runs per backend, cooled below 42 C before each run): gpu prefill 400.7-411.1 tok/s vs cpu 39.0-72.0, TTFT 0.56 s vs 2.9-5.3 s, decode 16.1-18.6 vs 15.6-15.8 tok/s (a modest 1.0-1.2x), peak VmHWM 1.13-1.14 GB vs 1.96-2.12 GB; full LITERT_CL delegation (1873/1873 nodes on every subgraph) and the gate answer verified through the thought channel. The gpu wins on time-to-first-token and memory; decode is close to a wash"
+          "reason": "measured on one Galaxy S26 with one binary (2026-09-08, 205-token prompt, 2 runs per backend, cooled below 42 C before each run): gpu prefill 400.7-411.1 tok/s vs cpu 39.0-72.0, TTFT 0.56 s vs 2.9-5.3 s, decode 16.1-18.6 vs 15.6-15.8 tok/s (a modest 1.0-1.2x), peak VmHWM 1.13-1.14 GB vs 1.96-2.12 GB; full LITERT_CL delegation (every node of every subgraph: 1873/1873 on the multi-token prefills, 1664/1664 on prefill_1, 1692/1692 on decode) and the gate answer verified through the thought channel. The gpu wins on time-to-first-token and memory; decode is close to a wash"
         },
         {
           "platform": "macos",
@@ -84,7 +84,7 @@
           "Requires litert-lm >= 0.16 (thought channel + ThinkingConfig); measured on 0.17.0",
           "Hybrid-reasoning model: with no ThinkingConfig it thinks before every answer; budget max output tokens >= 2048 (4096 for math). enable_thinking=false (ThinkingConfig or conversation extra_context) switches to direct answers",
           "Thinking-mode caveat measured on this file: on GSM8K-class questions its reasoning chains run ~4x longer than int8's and do not close inside 3584 tokens on the CPU reference (0/10 closed; int8 9/10, bf16 9/10). Use int8 when the reasoning must complete, or turn thinking off",
-          "iPhone 17 Pro: passes the on-device gate on gpu and cpu (init 5.7 s / 2.2 s); the int8 file does not fit default-entitlement iOS (2.33 GB main section)",
+          "iPhone 17 Pro: passes the on-device gate on gpu and cpu (init 5.7 s / 2.2 s); the int8 file's 2.33 GB main section needs the com.apple.developer.kernel.increased-memory-limit entitlement on iOS (not checked with that file)",
           "The 13 all-zero MLP rows in decoder layer 0 make raw blockwise-int4 exports fail to load on XNNPACK (scale 0); this file carries the epsilon fix"
         ]
       },
@@ -137,7 +137,7 @@
           "cache": "no",
           "runs": 2,
           "date": "2026-09-08",
-          "source": "205-token prompt with --benchmark, 2 runs (engine init 11.2 / 13.1 s), device cooled below 42 C before each run; GPU gate full LITERT_CL delegation (1873/1873 nodes on every subgraph, 0 blocking XNNPack), generation verified on the gate prompt; the cpu row of the same date is the same-device control"
+          "source": "205-token prompt with --benchmark, 2 runs (engine init 11.2 / 13.1 s), device cooled below 42 C before each run; GPU gate full LITERT_CL delegation (every node of every subgraph: 1873/1873 on the multi-token prefills, 1664/1664 on prefill_1, 1692/1692 on decode; 0 blocking XNNPack), generation verified on the gate prompt; the cpu row of the same date is the same-device control"
         },
         {
           "device": "Galaxy S26 (SM-S942Q, Qualcomm SM8850, Adreno)",
@@ -209,7 +209,7 @@
           "Requires litert-lm >= 0.16 (thought channel + ThinkingConfig); measured on 0.17.0",
           "Hybrid-reasoning model: with no ThinkingConfig it thinks before every answer; budget max output tokens >= 2048 (4096 for math). enable_thinking=false switches to direct answers",
           "fp32 activations are declared in-bundle: with the runtime's default fp16 GPU activations this model's reasoning on one gate question ran 2000+ tokens without closing </think> (7/8, degenerate); with fp32 it closes in ~450 tokens and passes 8/8, at ~14 % GPU decode cost (Mac). The declaration is a measured trade for the gate, not a general cure",
-          "Not for default-entitlement iOS apps: the main weight section is 2.33 GB, above the ~2 GiB single-section mmap limit",
+          "iOS: the 2.33 GB main weight section is above what a default-entitlement app maps in one piece; the com.apple.developer.kernel.increased-memory-limit entitlement lifts that limit (not checked with this file)",
           "GSM8K (thinking off, n=100, greedy): 91 vs bf16 92 - at parity; with thinking on it reproduces the bf16 model question for question on a 10-question subset (9/10 chains close, ~3.2k characters)"
         ]
       },
@@ -262,7 +262,7 @@
           "cache": "no",
           "runs": 2,
           "date": "2026-09-08",
-          "source": "205-token prompt with --benchmark, 2 runs, device cooled below 42 C before each run; GPU gate full LITERT_CL delegation (1873/1873 nodes on every subgraph), generation verified on the gate prompt through the thought channel (engine init 6.3 / 4.0 s); the cpu row of the same date is the same-device control"
+          "source": "205-token prompt with --benchmark, 2 runs, device cooled below 42 C before each run; GPU gate full LITERT_CL delegation (every node of every subgraph: 1873/1873 on the multi-token prefills, 1664/1664 on prefill_1, 1692/1692 on decode), generation verified on the gate prompt through the thought channel (engine init 6.3 / 4.0 s); the cpu row of the same date is the same-device control"
         },
         {
           "device": "Galaxy S26 (SM-S942Q, Qualcomm SM8850, Adreno)",