CharlZKP/OpenThai2.0-OCR-Markdown-ONNX

OpenThai2.0-OCR-Markdown, ONNX for the browser (fp32)

An ONNX export of iapp/OpenThai2.0-OCR-Markdown (iApp Technology's Thai fine-tune of NVIDIA Nemotron 3.5 Parse) for onnxruntime-web, WebGPU or WASM, so pages are read entirely on the user's device.

The weights are not quantized. Every tensor is the original float32 value from model.safetensors, copied byte for byte. The graphs are rewritten so that the browser runtime can run them. Nothing was retrained or pruned.

Files (onnx/)

File	What
encoder.onnx + encoder.data0..3	C-RADIOv4-SO400M vision encoder (27 blocks) and the neck, for WebGPU. Attention is plain MatMul/Softmax over 105 unrolled query chunks of ⌈N/105⌉ rows, so one scores buffer stays under 128 MiB. The fused WebGPU attention kernels of onnxruntime-web 1.30 return wrong values beyond 16 positions (see below).
encoder_chunked.onnx	The same encoder, sharing the same encoder.data* files. Attention is computed 128 query rows at a time inside an ONNX Loop. Use it for the WASM (CPU) backend: ORT's CPU attention kernel builds the full 16×N×N matrix, which overflows at about 4,000 patches, and a full page has up to 13,322.
decoder.onnx + decoder.data0..1	The 10-layer causal mBART decoder over [visual prefix, text], using com.microsoft.GroupQueryAttention (16 query heads, 8 KV heads) with a KV cache. On WebGPU, feed at most 16 new positions per call: prefill the visual prefix + prompt in pieces. A vision-only piece has an empty input_ids [1,0]. The tied lm_head shares the embedding initializer, so it is stored once. It returns the logits of the last position only.
pos_embed.f32	The learned 128×128×1152 position grid. The content-normalised interpolation (hf_prefix_geometry.py) is done in JavaScript.
tokens.json	id → token string, for byte-level BPE decoding. Added tokens with normalized: true are stored in NFKC form, as HF tokenizers returns them.

Interfaces:

• encoder: patches [N,768] (CLIP-normalised 16×16 RGB patches, row-major, channel/row/column inside a patch) and pos [N,1152] → vision [1, N/4+1, 1024]
• decoder: vision [1,V,1024] (V=0 after the first step), input_ids [1,T], past_key_values.{0..9}.{key,value} [1,8,P,64], seqlens_k [1] int32 = P+V+T−1, total_sequence_length [] int32 = P+V+T → logits [1,72512], present.*

Preprocessing matches NemotronParseImageProcessor (navit packing): fit inside 1664×2048 with a PIL bilinear downscale, pad right and bottom with white to height%16 and width%64, then the CLIP mean/std. The prompt is </s><s><predict_bbox><predict_classes><output_markdown><predict_text_in_pic> = ids [2, 0, 50004, 50008, 50001, 50009]. Decoding is greedy with repetition penalty 1.1 (generation_config.json) and stops at </s> (id 2).

Parity with the original (Transformers, fp32, CPU)

• encoder output, ORT CPU: max |diff| 2.9e-3 on a full page (values up to 14), 2.4e-4 on a small crop
• greedy tokens: identical on every test page. Checked with ORT Python CPU, and with onnxruntime-web WASM in Node (the browser CPU path; the full JS pipeline from PNG to tokens, with and without the 16-position prefill).
• WebGPU (headless Chromium, SwiftShader): the plain-op encoder attention matches CPU to 1e-6. GroupQueryAttention matches CPU for ≤16 new positions and gives wrong rows for ≥32, and MultiHeadAttention does the same. That is why the WebGPU graphs and the prefill avoid long fused-attention calls.

Built with build_onnx.py (included): python build_onnx.py <OpenThai2.0-OCR-Markdown dir> <out dir>.

Licence and attribution

This is a derivative of iapp/OpenThai2.0-OCR-Markdown, which is a derivative of NVIDIA Nemotron 3.5 Parse. It is distributed under the OpenMDW License Agreement, version 1.1 (LICENSE). The tokenizer is MIT-licensed (TOKENIZER_ATTRIBUTION.md). NVIDIA's original model card (NVIDIA_NEMOTRON_PARSE_README.md) and third-party notices (THIRD_PARTY_NOTICES.md) are included unchanged. The base model, tokenizer and serving code are NVIDIA's. The Thai fine-tune is iApp Technology's. See the upstream model card for training data, benchmarks and limits.