Instructions to use kltng/surya-ocr-2.5-mlx-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use kltng/surya-ocr-2.5-mlx-8bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("kltng/surya-ocr-2.5-mlx-8bit") config = load_config("kltng/surya-ocr-2.5-mlx-8bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Surya OCR 2.5, 8-bit MLX
This is an 8-bit MLX build of datalab-to/surya-ocr-2.5, for Apple Silicon Macs. The model is the work of Datalab. This is an unofficial conversion. Datalab did not make it and does not endorse it.
What was changed
The weights were converted and the files below were repaired, using surya/scripts/build_mlx.py from
surya-ocr 0.23.0 with mlx-vlm 0.7.10:
python surya/scripts/build_mlx.py --checkpoint datalab-to/surya-ocr-2.5 --name surya-2.5-mlx-q8 --q-bits 8
- Weights converted to MLX and quantized to 8 bits (affine, group size 64), with the rest in bfloat16.
- Original tokenizer files restored after conversion.
eos_token_idcorrected inconfig.jsonandgeneration_config.json.- This model card is new. The example images in the original repository were left out.
How to use
SURYA_INFERENCE_BACKEND=mlx \
SURYA_MLX_MODEL=kltng/surya-ocr-2.5-mlx-8bit \
HF_HUB_CACHE=/path/to/a/folder/with/only/this/model \
surya_ocr page.png
Install surya-ocr==0.23.0 and mlx-vlm==0.7.10.
Keep this model alone in its Hugging Face cache. mlx_vlm.server lists every cached model, and
Surya can send its requests to one of the others, whatever SURYA_MLX_MODEL says. The only sign is a log
line: reports model=... but expected ...; using reported name. A separate HF_HUB_CACHE avoids this.
Test results
Tested on a Mac mini (M4, 16 GB) on 150 sampled pages from four Chinese books: modern print and woodblock prints, with vertical text. Lower error is better. These numbers come from one small test on one kind of material, so they may not hold for other documents.
| This build | Surya 2.5 GGUF (llama.cpp) | |
|---|---|---|
| Median page error | 2.7% | 2.6% |
| Error on easy pages | 2.8% | 2.7% |
| Error on hard pages | 17.8% | 16.8% |
| Share of the text found | 94.7% | 95.0% |
Error is character error rate after folding traditional and simplified forms. This build took about 8 seconds per page on the Mac mini, one page at a time. The GGUF run was on a different machine, so its speed is not compared here.
Surya does not report runaway output through this backend, so check pages that take unusually long.
Licence
The weights are under the modified AI Pubs Open RAIL-M licence, copied in LICENSE. All of its
use-based restrictions (paragraph 5 and Attachment A) apply to this build and to anyone who uses it.
That includes the limits on commercial use. Datalab describes the weights as free for research, personal
use and startups under $5M in funding or revenue. For broader use, see
Datalab's pricing page. The licence also covers the model's output.
- Downloads last month
- 299
8-bit
Model tree for kltng/surya-ocr-2.5-mlx-8bit
Base model
datalab-to/surya-ocr-2.5