jefflaporte/chandra-ocr-2-MLX-8bit

Chandra OCR 2 — MLX 8-bit

An MLX 8-bit quantization of Chandra OCR 2 by Datalab, for Apple Silicon.

These files are modified: the language model's weights are quantized from the original BF16 checkpoint. The vision tower is unchanged (BF16), and so are the tokenizer, chat template and processor configuration. All credit for the model belongs to Datalab; see the original model card for what Chandra does and how it was evaluated.

Chandra OCR 2 is built on Qwen3.5-4B by the Qwen team, Alibaba Cloud, licensed under the Apache License 2.0, a copy of which is included as LICENSE-Qwen.

License and use restrictions

This derivative is distributed under the same license as the original, included as LICENSE (a modified OpenRAIL-M). Its use restrictions apply to these files too, including the commercial restrictions in Attachment A: no use by an organization with more than $2M in annual revenue or in total funding, except personal or research use, and no use by a competitor of Datalab. Commercial licenses are available from Datalab at https://www.datalab.to/.

Quantization

Every language-model projection and the token table (which is also the output head: the model ties them) are affine 8-bit in groups of 64, rounded to nearest with no calibration. The vision tower stays BF16. 4.8 GB.

Made with mlx-vlm 0.7.6 (see quantization.json for the exact source revision and tools):

mlx_vlm convert --hf-path datalab-to/chandra-ocr-2 --mlx-path chandra-ocr-2-MLX-8bit \
  -q --q-bits 8 --q-group-size 64

mlx-vlm rewrites the processor configuration on conversion; the original preprocessor_config.json, processor_config.json and video_preprocessor_config.json were restored, since vLLM rejects the rewritten ones and the preprocessing is the original model's.

Quality

On nine scanned 1940s typewritten documents (memos, letters, an expense ledger), its text matched the original BF16 model's on every page; bounding boxes moved by at most a pixel.

This is a spot check, not a benchmark: run your own evaluation (for example olmOCR-bench) before relying on it.

Usage

With vllm-metal and Chandra's own client (pip install chandra-ocr):

vllm serve jefflaporte/chandra-ocr-2-MLX-8bit --served-model-name chandra \
  --mm-processor-kwargs '{"min_pixels": 3136, "max_pixels": 6291456}'
chandra input.pdf ./output --method vllm

With mlx-vlm:

python -m mlx_vlm.server --model jefflaporte/chandra-ocr-2-MLX-8bit

Chandra reads a page at up to 6,291,456 pixels; a server that caps images lower can lose the layout of dense pages.