Surya OCR 2.5, 8-bit MLX

This is an 8-bit MLX build of datalab-to/surya-ocr-2.5, for Apple Silicon Macs. The model is the work of Datalab. This is an unofficial conversion. Datalab did not make it and does not endorse it.

What was changed

The weights were converted and the files below were repaired, using surya/scripts/build_mlx.py from surya-ocr 0.23.0 with mlx-vlm 0.7.10:

python surya/scripts/build_mlx.py --checkpoint datalab-to/surya-ocr-2.5 --name surya-2.5-mlx-q8 --q-bits 8
  • Weights converted to MLX and quantized to 8 bits (affine, group size 64), with the rest in bfloat16.
  • Original tokenizer files restored after conversion.
  • eos_token_id corrected in config.json and generation_config.json.
  • This model card is new. The example images in the original repository were left out.

How to use

SURYA_INFERENCE_BACKEND=mlx \
SURYA_MLX_MODEL=kltng/surya-ocr-2.5-mlx-8bit \
HF_HUB_CACHE=/path/to/a/folder/with/only/this/model \
surya_ocr page.png

Install surya-ocr==0.23.0 and mlx-vlm==0.7.10.

Keep this model alone in its Hugging Face cache. mlx_vlm.server lists every cached model, and Surya can send its requests to one of the others, whatever SURYA_MLX_MODEL says. The only sign is a log line: reports model=... but expected ...; using reported name. A separate HF_HUB_CACHE avoids this.

Test results

Tested on a Mac mini (M4, 16 GB) on 150 sampled pages from four Chinese books: modern print and woodblock prints, with vertical text. Lower error is better. These numbers come from one small test on one kind of material, so they may not hold for other documents.

This build Surya 2.5 GGUF (llama.cpp)
Median page error 2.7% 2.6%
Error on easy pages 2.8% 2.7%
Error on hard pages 17.8% 16.8%
Share of the text found 94.7% 95.0%

Error is character error rate after folding traditional and simplified forms. This build took about 8 seconds per page on the Mac mini, one page at a time. The GGUF run was on a different machine, so its speed is not compared here.

Surya does not report runaway output through this backend, so check pages that take unusually long.

Licence

The weights are under the modified AI Pubs Open RAIL-M licence, copied in LICENSE. All of its use-based restrictions (paragraph 5 and Attachment A) apply to this build and to anyone who uses it. That includes the limits on commercial use. Datalab describes the weights as free for research, personal use and startups under $5M in funding or revenue. For broader use, see Datalab's pricing page. The licence also covers the model's output.

Downloads last month
299
Safetensors
Model size
0.7B params
Tensor type
U32
ยท
BF16
ยท
MLX
Hardware compatibility
Log In to add your hardware

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for kltng/surya-ocr-2.5-mlx-8bit

Quantized
(4)
this model