litert-community/MiniCPM-OCR-Decision-2B-LiteRT

MiniCPM-OCR decision 2B for LiteRT

This repository is a conversion of openbmb/MiniCPM-OCR-Jev to LiteRT, made with litert-torch 0.10.0 for the CompiledModel API of ai-edge-litert 2.3.0: on an Apple M4 Max with Metal, the decision graph at the GPU's default precision and the picture tower at float32, scoring one OCR candidate for a text region takes a median 179.4 ms, from the two pictures and the text to the score (2026-11-06). Speed still being tuned (updates follow).

openbmb/MiniCPM-OCR-Jev is a model by OpenBMB that checks OCR output. It takes the picture of a text, formula or table region, a candidate OCR text, and a picture rendered from that text, and returns a score from 0 to 1: a higher score means the candidate is more likely correct. It generates no text. It is used to accept reliable regions automatically, to send the rest to human review, or to pick the best result among several OCR models. The scoring guide in the source's GitHub repository (docs/SKILL_jev_en.md) says: "The scores carry no threshold and no label, and they are not calibrated to an absolute quality scale". So choose a threshold on samples from your own application. What it is for and how it scores: the source model card.

This repository holds the model as LiteRT files and a Python host for desktops that needs no PyTorch. A tower graph turns one picture into tokens, and a decision graph reads the tokens of the whole request and returns its hidden states. The host writes the inputs, picks the graphs and computes the score. You supply the rendered picture: this repository does not render.

The reference for every check is the source repository's own scoring code in float32 on the CPU. On the Apple M4 Max with Metal, the decision graph at the default precision and the tower at float32, on the 417 olmOCR records of JevOCRBench that the files hold, AUROC (×100) is 92.40 against the reference's 92.38, two records change side at a cutoff of 0.5 and one at 0.9, and scores differ by at most 0.0554.

On two Android phones, the graph files ran through LiteRT 2.2.0 on inputs that the host wrote on a Mac, and the host read the outputs back. On a Pixel 11 Pro's GPU, the decision graph at the default precision and the towers at float32, the graph calls for the same text region add up to 1,913.7 ms, and all 90 records of a test set are inside the phones' tolerance. On a Galaxy S26's GPU, in the same form, they add up to 861.6 ms, and to 799.7 ms with the decision call on its NPU (see Measured agreement and speed).

Second update of 2026-11-06. The nine single decision files now also use the kernel forms bt and cv of the gated linear attention layers (see GPU precision), the same computation in fewer operators; their names are unchanged. A text region's request on the Apple M4 Max with Metal takes 179.4 ms against 186.3 ms for the previous files in one measurement window. A decision call on the Pixel 11 Pro's GPU takes 957.7, 1,412.5 and 5,428.9 ms against 1,037.6, 1,536.9 and 5,868.7 ms for the text, formula and table regions, and on the Galaxy S26's 406.3, 583.5 and 2,786.9 ms against 430.5, 617.7 and 3,036.1 ms, each in one session with the previous file before and after. On the CPU the new files give the previous files' outputs bit for bit (all 417 records). At the GPU's default precision their scores differ from the previous files' by up to 0.0383 on Metal (417 records) and 0.0107 on the Pixel 11 Pro (90 records), and not at all on the Galaxy S26 (83 records); the agreement tables give the new files' rows. SHA-256 (first 8 digits), previous → new: 256 tokens bc22d879 → 526daaa7, 384 a8133c4a → c93ddd72, 512 67a450f5 → 83a71508, 768 2240b1a9 → a35df5cc, 1,024 2fa06fec → 170842c1, 1,536 f324f006 → ceeabf92, 2,048 4dae686e → 0f9b8f03, 3,072 a70c8463 → 08877227, 4,096 19f0e872 → c7c1b82b. The multi-signature files, the towers, the NPU files, the tables and the tokenizer are byte-identical; contract.json gives the new files' bytes and SHA-256, and conversion/ their export (the first step of conversion/README.md, the new conv_layout.py, and changes to kernel_q35.py and patch_q35.py).

Update of 2026-11-06. The six decision files are now the f16s form (see GPU precision), which the host runs at the GPU's default precision; the towers stay at float32. On the Pixel 11 Pro's GPU, a 512-token decision call takes 1,531.6 ms against 2,770.7 ms, and on the Galaxy S26's 567.1 ms against 1,397.0 ms, where the 1,024- and 2,048-token files now run. At float32 the new files give the previous files' outputs bit for bit (on the CPU, all 417 records). Their names end in f16sfc where the previous names ended in fp16fc, so a command that names a previous decision file gets a 404, and the new host stops with a FileNotFoundError on a folder that holds only the previous files. SHA-256 (first 8 digits), previous → new: 256 tokens b1167431 → bc22d879, 512 ac2cf5fe → 67a450f5, 1,024 818224cb → 2fa06fec, 2,048 6684732c → 4dae686e, 4,096 7fd1cb87 → 19f0e872, multi-signature 0b8a3344 → 14f6fbb1. The NPU files are now three, for up to 256, 512 and 1,024 tokens, compiled from the f16s form exported with --sum-after (see Galaxy S26 NPU); the previous two, whose names lack _z, are gone. Eight files are new, at the half steps between the others: decision files for 384, 768, 1,536 and 3,072 tokens, and towers for 784, 1,296, 1,936 and 3,136 patches. The other towers, the tables and the tokenizer are byte-identical.

Files

File	Bytes	Role
decide_L256_f16sfc.tflite	2,750,394,480	Decision graph for requests of up to 256 tokens.
decide_L384_f16sfc.tflite	2,750,868,256	Up to 384 tokens.
decide_L512_f16sfc.tflite	2,751,472,944	Up to 512 tokens.
decide_L768_f16sfc.tflite	2,753,075,504	Up to 768 tokens.
decide_L1024_f16sfc.tflite	2,755,202,352	Up to 1,024 tokens.
decide_L1536_f16sfc.tflite	2,761,028,912	Up to 1,536 tokens.
decide_L2048_f16sfc.tflite	2,768,952,624	Up to 2,048 tokens.
decide_L3072_f16sfc.tflite	2,791,091,504	Up to 3,072 tokens.
decide_L4096_qb1024_f16sfc.tflite	2,758,845,216	Up to 4,096 tokens, attention in query blocks of 1,024.
multi_decide_L4096_f16sfc.tflite	2,812,103,104	The graphs for 256, 512, 1,024, 2,048 and 4,096 tokens as five signatures of one file, over one copy of the weights.
tower_N400_fp16fc.tflite	658,758,848	Picture tower and projector for a picture of up to 400 patches.
tower_N576_fp16fc.tflite	658,758,848	Up to 576 patches.
tower_N784_fp16fc.tflite	658,758,848	Up to 784 patches.
tower_N1024_fp16fc.tflite	658,758,848	Up to 1,024 patches.
tower_N1296_fp16fc.tflite	658,758,848	Up to 1,296 patches.
tower_N1600_fp16fc.tflite	658,758,848	Up to 1,600 patches.
tower_N1936_fp16fc.tflite	658,758,848	Up to 1,936 patches.
tower_N2304_fp16fc.tflite	658,758,848	Up to 2,304 patches.
tower_N3136_fp16fc.tflite	658,758,848	Up to 3,136 patches.
tower_N4096_qb1024_fp16fc.tflite	658,857,872	Up to 4,096 patches, attention in query blocks of 1,024.
multi_tower_N4096_fp16fc.tflite	660,721,136	The towers for 400, 576, 1,024, 1,600, 2,304 and 4,096 patches as six signatures of one file, over one copy of the weights.
npu/sm8850/	8,489,952,896	Three decision graphs, for requests of up to 256, 512 and 1,024 tokens, compiled ahead of time for the NPU of the Qualcomm SM8850, the chip of the Galaxy S26 (see Galaxy S26 NPU). _z in the names marks the form with a size-1 SUM between the output of a linear and the sigmoid that reads it, in 24 places.
tables/tables.safetensors	1,021,845,928	The source's token embedding, the picture tower's position embedding and score.weight, as the bfloat16 bits of the checkpoint. The host reads their rows and applies the head.
tokenizer.json	19,989,325	The source repository's file, unchanged.
contract.json		The interface as data: graph inputs and outputs, buckets, the prompt, the refusals, and the size and SHA-256 of every graph.
host/		The Python host, decide_host (load_host, score, best_of), and requirements-host.txt.
examples/		run_example.py (the source's three examples and the refusals) and score_olmocr_set.py (the 446 olmOCR records). Neither holds a picture or a record: they download them.
conversion/		The export and check scripts, their environments, and a README with the commands in order.
REPRODUCE.md		Sources, environments, steps, and where each number on this card comes from.
LICENSE, NOTICE, SHA256SUMS		The Apache License 2.0, attribution, and the SHA-256 of every file.

Each single decision file holds the whole text decoder and its own copy of the weights. A graph computes every position of its length whatever the request's length, so the host takes the smallest bucket that holds a request. A picture's tower works the same way, by patches. The single files hold buckets of 256, 384, 512, 768, 1,024, 1,536, 2,048, 3,072 and 4,096 tokens, and of 400, 576, 784, 1,024, 1,296, 1,600, 1,936, 2,304, 3,136 and 4,096 patches; the multi-signature files hold 256, 512, 1,024, 2,048 and 4,096 tokens, and 400, 576, 1,024, 1,600, 2,304 and 4,096 patches. The decision files are the f16s form of the decision graph, whose final norm stays inside float16's range; the single decision files also use the kernel forms bt and cv (see GPU precision).

Minimal usage

Python: desktop CPU or GPU

hf download litert-community/MiniCPM-OCR-Decision-2B-LiteRT --local-dir MiniCPM-OCR-Decision-2B-LiteRT \
  --include "multi_*" --include "tables/*" --include tokenizer.json --include contract.json --include "host/*" --include "examples/*"    # about 4.5 GB of the 44.43 GB
cd MiniCPM-OCR-Decision-2B-LiteRT
pip install -r host/requirements-host.txt
python examples/run_example.py --check    # GPU (Metal on a Mac), the two multi-signature files, about 8 GiB of memory

The CPU path uses one file per bucket, so it needs the single files too: download without --include, then run python examples/run_example.py --check --accel cpu --form files (XNNPACK, 4 threads).

The host needs tokenizers, numpy, pillow and ai-edge-litert (2.3.0), pinned with their dependencies in host/requirements-host.txt. It does not need PyTorch, torchvision or transformers.

examples/run_example.py scores the source's three examples, a text, a formula and a table region. Their pictures are not in this repository: the script downloads them into memory from the source repository at a pinned commit and checks their SHA-256 before use. --check passes when each score is within 0.02 of the value the source publishes, and when every request the host must refuse comes back with the expected status.

Example	Published	Reference	This repository, Metal, multi-signature files	This repository, Metal, single files	This repository, CPU, single files
text	0.00702	0.00212	0.00196	0.00196	0.00212
formula	0.96552	0.96656	0.96744	0.96756	0.96657
table	0.00116	0.00100	0.00087	0.00087	0.00100

In your own code, score takes your region picture, the rendered picture and the candidate text. With the source's first example:

import sys
sys.path[:0] = ["host", "examples"]  # run from the repository root
import io, json
from PIL import Image
from decide_host import load_host
from run_example import fetch

case = json.loads(fetch("01_text_putnam/case.json"))
text = next(c["text"] for c in case["candidates"] if c["role"] == case["reference_render_of"])
block = Image.open(io.BytesIO(fetch("01_text_putnam/block.png")))
render = Image.open(io.BytesIO(fetch("01_text_putnam/reference_render.png")))
with load_host(".", accelerator="gpu") as host:
    out = host.score(block, render, text)
print(round(out["prob_1"], 3), out["T"], out["n"])

It prints 0.002 304 [256, 280]: the score, the request's tokens, and each picture's patches.

score(image, render, text) takes file paths, or PIL images as Image.open returns them, and returns status, prob_1, logits, T and n. best_of(image, [(render, text), ...]) scores several candidates of one region, with the original picture's tower run once, and returns the index of the highest as best and all of them as candidates. load_host() takes form="multi" (the default: the two multi-signature files, one compile) or form="files" (it compiles a file when a request first needs one and keeps at most keep of each kind). On the GPU it runs the decision graph at the delegate's default precision and the tower with GpuOptions(enforce_f32=True), and for form="multi" it sets constant_tensor_sharing=True on both. On the CPU, form="multi" needs weight_cache_dir=.

How the host works

contract.json holds these rules as data.

1. Pictures. Each picture is decoded to 8-bit RGB without applying EXIF, resized to multiples of 32 pixels with the source's area bounds, and written as patch vectors in the source's order. The host resizes with its own bicubic (host/), because Pillow's differs from the source's path on up to 2 levels per value. The values are (u8 - 127.5) / 127.5.
2. Tower. One call per picture, on the smallest tower signature that holds its patches. The inputs are patches float32 [1, N, 1536], valid [1, N], pos [1, N, 1024], and cos and sin [1, N, 64]. The output rows [1, N / 4, 2048] holds the picture's tokens.
3. Prompt. The source's fixed prompt, with the original picture first and the rendered picture second. The tokenizer is the source repository's tokenizer.json, read with tokenizers. A request of T tokens has T = 77 + m1 + m2 + text tokens, where m1 and m2 are the two pictures' token counts.
4. Decision graph. One call on the smallest decision signature that holds T. The inputs are embeds float32 [1, L, 2048] (rows of tables/, widened from bfloat16 to float32, with the tower's rows at the picture positions), valid [1, L], and cos and sin [1, L, 64], which the host computes from the source's position rules. The output hidden [1, L, 2048] holds the hidden states after the final norm.
5. Score. logits = hidden[T - 1] @ score.T with the two rows of the source's score.weight, then softmax, and the score is the second probability, prob_1. The host checks that position T - 1 holds token 271.

The host refuses a request rather than change it, and returns a status without a score:

Status	When
too_large (with n, T, limit)	A picture has more patches, or the request more tokens, than the largest installed file holds.
not_8_bit	A 16-bit picture: the source's decoder and Pillow read it differently.
literal_pad_token	The text holds a literal <|image_pad|> or <|video_pad|>.
unreadable_picture	A file that is missing, does not decode, or has a side of 0.

Downscaling a picture to fit is not offered: the source says inference uses the checkpoint's native picture budget and warns "do not harmonise it" against the training budget, and on 90 records that we scaled to 448 × 448, 12 to 18 % of the scores moved to the other side of a cutoff of 0.5 or 0.9.

Measured agreement and speed

Agreement with the source's float32 code

The reference is the source repository's score.py at the revision below, with Qwen3_5ForSequenceClassification in float32 on the CPU (transformers 5.10.4, torch 2.8.0). On the source's three examples it gives differences of -0.0049, +0.0010 and -0.0002 against the published values, inside the source's ±0.02.

The test set is the 446 olmOCR records of JevOCRBench (230 text, 157 formula, 59 table), the part of the benchmark under ODC-BY-1.0. The reference scores 445 of them: one holds a picture of 65,024 patches. AUROC is the benchmark's own evaluate.py.

The tolerance, set before the first reference number: AUROC (×100) no more than 0.2 below the reference overall and no more than 0.5 below per category; the same side at cutoffs of 0.5 and 0.9 on at least 99 % of the records; and each control that moves the reference moves these files too. The controls swap the candidate's text, swap the two pictures' order, or swap the original picture.

Files	Accelerator	Records	AUROC (×100), reference → these files	text	formula	table	Other side of 0.5 / 0.9	max |Δscore|	Controls
Single files	Metal: the decision graph at the default precision, the tower at float32	417	92.38 → 92.40	94.22 → 94.23	88.61 → 88.63	95.87 → 95.87	2 / 1	0.0554	3 of 3 moved
Single files	CPU (XNNPACK, 4 threads)	417	92.38 → 92.39	94.22 → 94.23	88.61 → 88.63	95.87 → 95.87	0 / 0	0.00224	3 of 3 moved

On Metal, the two records on the other side of 0.5 moved from the reference's 0.5002 to 0.4904 and from 0.4928 to 0.5056, and the one on the other side of 0.9 from 0.8994 to 0.9024. The CPU row ran on the first upload's plain decision files (_fp16fc without bt and cv, removed by the previous update), at their five lengths. On the CPU, the files here give the previous update's files' hidden states and scores bit for bit, on all 417 records. With the half-step lengths, 280 of the 417 records run at least one of their three graph calls on another file than in the CPU row's run; on the CPU, the files here then give the same hidden states and scores bit for bit on 413 of the 417 records and move the other four scores by less than 0.000001, so the row's figures, except the controls, come out the same. The multi-signature decision file is the f16s form without bt and cv: on Metal with constant tensor sharing it gives the hidden states of the previous update's single decision files bit for bit, at the default precision and at float32, on two records per signature. At its five lengths, those files gave on the 417 records AUROC (×100) 92.39, none on the other side of 0.5 and one of 0.9, and scores at most 0.0327 from the reference's.

The 417 records are those that the files in this repository hold; 28 more need larger files. On the 445 the reference scores, its AUROC is 92.78. Each control ran on 36 records for the reference and on the 31 to 33 of them that these files hold. It moved at least 21 records by more than 0.02, where the reference's moved at least 25.

These numbers measure how closely this conversion follows the source's code, not task accuracy. On the 2,267 records of the whole benchmark the source card gives AUROC (×100) 93.6; this repository does not re-measure that.

The phones ran a 90-record subset of the test set, 15 records for each category and label, chosen before any reference number. Each phone ran the buckets it holds, and the host on the Mac read its outputs. The comparison is with the rows of the plain files at float32 on the Mac (Metal) on the same inputs, and the AUROC with the reference on the same records.

Phone	Files	Accelerator	Records	AUROC (×100), reference → phone	Other side of 0.5 / 0.9	max |Δscore| against Metal
Pixel 11 Pro	Single decision files at eight of the nine lengths (no record of the 90 needs the 4,096-token file), the multi-signature tower file and the four half-step towers	GPU: the decision graph at the default precision, the towers at float32; sharing	90	87.31 → 87.41	0 / 2	0.0144
Galaxy S26	Single decision files up to 3,072 tokens and single towers up to 3,136 patches, half steps included	GPU: the decision graph at the default precision, the towers at float32; sharing	83	86.05 → 86.34	1 / 0	0.0289
Galaxy S26	npu/sm8850/, 256, 512 and 1,024 tokens	NPU	69	84.04 → 84.13	0 / 0	0.0065

Each row passes the phones' tolerance on the records it holds, and that tolerance was registered with the Mac's before any phone ran: the same side at both cutoffs, except a record within 0.02 of the cutoff, and AUROC no more than 0.2 below the reference overall and 0.5 per category. The Pixel's two records on the other side of 0.9 moved from Metal's 0.9013 to 0.8996 and from 0.8995 to 0.9015, and the Galaxy S26's one record on the other side of 0.5 from 0.5148 to 0.4980.

Speed on a Mac (Apple M4 Max)

An Apple M4 Max (macOS 27.0), ai-edge-litert 2.3.0 through the Python CompiledModel API, 4 threads for the host. A time is the median of 40 requests, in two runs of 20 after 5 warm-up requests each, through the host from the two PNG files and the text to the score; each column is one measurement window, in which the two runs went before and after two runs on the previous update's decision files on Metal, and between two such runs on the CPU; those took 186.3, 237.1 and 1,121.0 ms on Metal and 822.8, 1,051.9 and 5,734.6 ms on the CPU.

Request (item of JevOCRBench)	Graphs	Metal, single files: the decision graph at the default precision, the tower at float32	CPU (XNNPACK, 4 threads), single files
Text region (item 1962: 280 tokens, pictures of 372 and 352 patches)	tower N400 twice, L384	179.4 ms	762.1 ms
Formula region (item 1897: 411 tokens, 288 and 576 patches)	N400, N576, L512	227.9 ms	981.0 ms
Table region (item 1918: 1,721 tokens, 1,584 and 2,888 patches)	N1600, N3136, L2048	1,087.1 ms	5,287.6 ms

The CPU column ran on decision files without the f16s scale (_fp16fc with bt and cv, not in this repository), which give the same outputs on the CPU as the files here (all 417 records); the CPU time of the files here was not measured. Compiling on Metal is once per file: a single decision file at the default precision took 14 to 18 s, and a single tower at float32 3 to 6 s. The largest footprint was 10.8 to 14.4 GiB.

Speed on Android phones

On the GPU, a test app ran the graph files through the Kotlin CompiledModel API of LiteRT 2.2.0, on inputs that the host wrote on the Mac. It times one graph call: writing the inputs, running and reading the output back. A time is the median of 20 calls after 5 warm-up calls; for the text region's tower, which takes both pictures, the median of the 40 calls of 20 requests after 5, and for a decision call the median of the 40 calls of two such runs. A request adds up the unrounded medians of its tower calls and its decision call. The host's own work, which ran on the Mac, is not included. Each timing run in the table started from a phone at rest (thermal status 0), except nine that started at thermal status 1: the Pixel 11 Pro's two table-region towers and its six runs of the decision files, and the NPU's text timing. On each phone each run of the 512-token file timed the text region first and the formula region second, in one process, so the formula time on that graph is the warmer one: the Pixel 11 Pro's two runs stayed at thermal status 1 (skin 37.4 → 40.9 and 38.8 → 41.9 °C), and the Galaxy S26's ended at thermal status 2 (skin 36.5 → 42.9 and 37.7 → 43.3 °C).

Request	Pixel 11 Pro, GPU: the decision graph at the default precision, the towers at float32	Galaxy S26, GPU: the same precisions	Galaxy S26: the decision graph on the NPU, the towers on the GPU at float32
Text region (item 1962)	1,913.7 ms (478.0 + 478.0 + 957.7)	861.6 ms (227.6 + 227.6 + 406.3)	799.7 ms (227.6 + 227.6 + 344.4)
Formula region (item 1897)	2,589.6 ms (515.5 + 661.6 + 1,412.5)	1,120.9 ms (241.2 + 296.2 + 583.5)	881.4 ms (241.2 + 296.2 + 344.0)
Table region (item 1918)	16,268.2 ms (2,672.3 + 8,167.0 + 5,428.9)	9,184.4 ms (1,346.8 + 5,050.6 + 2,786.9)	did not run

In parentheses: the tower calls, then the decision call. Each decision call comes from one session per region on each phone that ran this repository's file twice between two runs of the previous update's file of the same length; there the previous files took 1,037.6, 1,536.9 and 5,868.7 ms on the Pixel 11 Pro and 430.5, 617.7 and 3,036.1 ms on the Galaxy S26, and this repository's decision files loaded in 15.3 to 18.9 s and 16.5 to 20.2 s. The Pixel 11 Pro's column takes the 400-, 576- and 1,600-patch tower calls from two earlier timing runs of the multi-signature tower file, which loaded in 6.3 to 7.2 s, and the 3,136-patch tower from a timing run of this repository's file, where it loaded in 5.4 s. The Galaxy S26's column takes the 400-, 576- and 1,600-patch tower calls from an earlier timing run on the phone, where a tower loaded in 4.7 to 5.6 s, and the 3,136-patch tower from a timing run of the half-step set, where it loaded in 4.8 s.

On the Galaxy S26's NPU, a test runner on the C API of LiteRT 2.2.0, outside an app, ran the decision files. One call took a median of 344.4 ms on the 512-token file for the text region and 344.0 ms for the formula region, and 150.5 ms on the 256-token file and 890.5 ms on the 1,024-token file for a text record each, the median of 20 calls after 5 warm-up calls; a file loaded in 1.4 to 2.8 s. The text region's timing started at thermal status 1, after 129 s of rest. The NPU column adds the 512-token medians to the same tower calls as the GPU column.

Android

This repository has no Android host: an app needs the steps of How the host works in its own language, through LiteRT's CompiledModel. Both phones ran the GPU through LiteRT's OpenCL backend, with every node on the GPU and constant_tensor_sharing on. The Pixel 11 Pro ran Android 17 (API level 37); the Galaxy S26 (SM-S942Q) Android 16 (API level 36).

• Pixel 11 Pro: the single decision files at the GPU's default precision, and the multi-signature tower file and the four half-step single towers at float32. At float32, the plain multi-signature decision file did not load in either of our two attempts: 158.3 s and 215.0 s into its compile, inside the fifth graph, the phone's available memory read 399,444 kB and 389,288 kB, under the 400,000 kB at which our test app stops a run. The multi-signature decision file of this repository has not run on a phone.
• Galaxy S26: npu/sm8850/ for requests of up to 1,024 tokens, with the towers on the GPU. On the GPU, over the 83 records in the agreement table, this repository's single decision files up to 3,072 tokens at the default precision, with 3.4 GB or more of memory free, and the single towers up to 3,136 patches at float32. Our test app stops a run when the available memory falls under 1,500,000 kB: at float32, the plain 1,024-token decision file (not in this repository) compiled in 23.1 s and stopped at its first record, at 1,498,484 kB, and the plain 2,048-token file stopped 25.9 s into its compile, at 1,397,496 kB. The 4,096-patch tower ended the process in both of our runs, with the GPU driver's line Adreno-GSL: <os_exit:1831>: Exiting the process, and 4.5 GB or more of memory free. By then the GPU's clock had dropped from 1,300 MHz to 578 and 342 MHz; we did not establish the cause. The 3,136-patch tower ran in its place, for the 11 records of the 83 whose largest picture it holds and in a timing run, so the table region in the speed table takes it.

Galaxy S26 NPU

npu/sm8850/ holds three decision files, decide_L256_z_f16sfc_SM8850.tflite, decide_L512_z_f16sfc_SM8850.tflite and decide_L1024_z_f16sfc_SM8850.tflite, for requests of up to 256, 512 and 1,024 tokens, compiled ahead of time for the NPU of the Qualcomm SM8850 with ai-edge-litert 2.2.0 and Qualcomm's QAIRT 2.47 (conversion/npu/). Their input graphs, which are not in this repository, are the f16s decision graph exported with --sum-after in_proj_z,q_proj (conversion/export_decide.py): it writes each of those bias-free linears, 24 per graph (18 in_proj_z and 6 q_proj), as a batch-1 matrix product, so that the converter puts a size-1 SUM between the linear and the sigmoid that reads it. These are the files that ran on the Galaxy S26, each as one dispatch operator that holds the whole graph, with every node on the NPU.

On the NPU the plain decision graph's output at the scored position was non-finite on 12 of 12 records, and the f16s form's was finite on all of them (see GPU precision). On the Mac at float32, the input graphs of these files give the f16s files' outputs bit for bit, and the f16s files give the plain files', on the 12, 29 and 28 records below.

We read the NPU as computing in float16: the plain file's final norm overflowed as float16 does, and the scaled norm came back finite. Compiled from the f16s form without --sum-after, the 512-token file was outside the tolerance on its 29 records (formula 71.43 → 69.84, one pair of 63), with scores as far as 0.0513 from Metal's; with it, each file is inside, and so are the rows of files together, which were fixed before the run. Where in the graph the remaining difference comes from was not measured.

File	Records	Other side of 0.5 / 0.9	AUROC (×100), reference → NPU	max / mean |Δscore| against Metal	Inside the tolerance
256 tokens	12	0 / 0	100.0 → 100.0	0.0027 / 0.0007	yes
512 tokens	29	0 / 0	71.43 → 71.90	0.0065 / 0.0008	yes
1,024 tokens	28	0 / 0	90.00 → 90.00	0.0062 / 0.0012	yes
256 and 512 tokens	41	0 / 0	82.14 → 82.38	0.0065 / 0.0008	yes
All three	69	0 / 0	84.04 → 84.13	0.0065 / 0.0010	yes

For scale, on 9 records the source's published bfloat16 form on a CPU differs from its float32 form by up to 0.0258, with a mean of 0.0071. A 2,048-token file compiled for this chip did not load: the phone refused its context with err = 1002 (Failed to find available PD, context size estimate 4,030,749,184).

The towers stay on the GPU. Compiled the same way, the 400- and 576-patch towers ran on the NPU in 57.7 and 68.6 ms a call, but their rows differ from Metal's by up to 31.3 % and 20.7 % (relative RMS), so they are not in this repository.

The host of this repository does not run the NPU files. A test runner on the C API of LiteRT 2.2.0, outside an app, loaded each into a CompiledModel with the NPU accelerator and Qualcomm's dispatch library (libLiteRtDispatch_Qualcomm.so), and ran it on inputs laid out on the Mac: the tower's rows at the picture positions came from the Mac's Metal rows. No app ran them in these checks.

GPU precision

On a GPU the host runs the decision graph at the delegate's default precision and the tower at float32 (GpuOptions(enforce_f32=True)). The plain decision graph's final norm sums squares up to 600,543, more than float16 holds (65,504), and at the Metal GPU's default precision, on the two records we tried, its score reads 0.5 where the source's code gives 0.988 and 0.900. The decision files here are its f16s form: it multiplies the final norm's input by 2^-4 and its epsilon by 4^-4, and caps its reciprocal square root at the value for a zero input. At float32 the two forms give the same outputs bit for bit, on the CPU on all 417 records and, for the previous update's files, without bt and cv, on Metal on the 83 records of the phones' test set that the 256- to 2,048-token files hold; at the default precision the f16s form gives the agreement in the tables above. The tower has no such form here: its rows at the default precision have a relative error of 1.18 to 1.28 on the three pictures we tried, and a tower with scaled norms stayed off by 7 to 18 %, so the tower runs at float32.

The single decision files also use two kernel forms of the gated linear attention layers, bt and cv (conversion/README.md): bt computes the delta rule over all chunks at once, and cv runs the causal convolution in the converter's layout, the same computation in fewer operators. On the CPU they give the outputs of the f16s files without them bit for bit, on all 417 records. At the GPU's default precision their scores differ a little from those files': by up to 0.0383 on Metal (417 records) and 0.0107 on the Pixel 11 Pro (90 records), while the Galaxy S26 gave the same scores on its 83 records. The multi-signature decision file is the f16s form without them.

Limits

• The model never generates text. It scores a candidate you give it.
• The score has no threshold and no absolute scale. Choose a threshold on your own samples.
• The host does not render. The source's renderer uses Chromium through Playwright 1.62.0, KaTeX 0.16.47 and CJK fonts; a render made another way is a different input.
• A request holds at most 4,096 tokens and a picture at most 4,096 patches with the files in this repository, and on the Galaxy S26, as run so far, 3,072 tokens on the GPU, 1,024 on the NPU, and 3,136 patches. A larger one is refused by the host, never cut: 417 of JevOCRBench's 446 olmOCR records fit.
• A 16-bit picture is refused when it comes as a path or straight from Image.open. Once a PIL image has been loaded or converted, the host cannot tell, and it reads the picture's high bytes as 8-bit. A picture's EXIF orientation is not applied, as in the source's path.
• Each picture is one tower call and each candidate one decision call. For several candidates of one region, the host reuses the original picture's rows.
• The agreement numbers measure how closely this conversion follows the source's code, not task accuracy. Accuracy and calibration were not measured again; see the source model card.
• Measured on one Mac (Apple M4 Max), one Pixel 11 Pro and one Galaxy S26. Other machines and GPUs were not measured.

Provenance, conversion and license

• Source: openbmb/MiniCPM-OCR-Jev at revision a0ed278eb6aa1f333a711665ab610055a4a816ab, trained from Qwen/Qwen3.5-2B, under the Apache License 2.0. Scoring code: OpenBMB/MiniCPM-OCR at commit 8177040f0d6c824ccf87bc18b938b6a0f91b6d0e.
• Model: the Qwen3.5 text decoder (24 layers: 18 gated linear attention and 6 full attention; hidden size 2,048; 248,320 embedding rows) and picture encoder (24 layers, width 1,024, 16 × 16 pixel patches merged 2 × 2), with the source's two-class head.
• Graphs: the source's computation written out in plain PyTorch for export, exported with litert-torch 0.10.0 from the weights in float32 (transformers 5.14.1), then quantized with ai-edge-quantizer 0.10.0: FULLY_CONNECTED weights in float16, activations in float32. The decision graphs compute their final norm in the f16s form, and the single decision files use the kernel forms bt and cv (see GPU precision). A graph has no COS, SIN, GATHER or int64 tensor: the positions are inputs, and the host computes them.
• Scripts: conversion/ holds the export and check scripts, with their environments and a README of the commands. REPRODUCE.md lists the sources, the steps and where each number comes from.
• Test data: this repository holds no JevOCRBench record and none of the source's example pictures. examples/run_example.py downloads the source's three examples from the source repository at the commit above and checks their SHA-256. The olmOCR records of JevOCRBench are under ODC-BY-1.0; the underlying documents keep their own rights. Cite olmOCR as arXiv:2502.18443.

License: openbmb/MiniCPM-OCR-Jev is released under the Apache License 2.0 (LICENSE), and so is this repository. Attribution is in NOTICE.

Changes from the original work. The weights were converted from the checkpoint's safetensors (bfloat16) to LiteRT flatbuffers: float16 FULLY_CONNECTED weights, float32 for the rest. The model was split into a picture tower and a decision graph at fixed sizes, with the positions as graph inputs. The decision graph computes its final norm on its input scaled by 2^-4, with its epsilon scaled by 4^-4 and its reciprocal square root capped at the value for a zero input, which gives the same numbers at float32. In the single decision files, the gated linear attention layers compute the delta rule over all chunks at once and run the causal convolution in the converter's layout. The token embedding and the head ship separately and the host applies them. The tokenizer is unchanged. The host ports the source's prompt, picture preprocessing and position tables to numpy. This repository is not affiliated with OpenBMB.