--- license: apache-2.0 language: - th - en library_name: mlx pipeline_tag: image-text-to-text base_model: typhoon-ai/typhoon-ocr2-0.8b tags: - mlx - ocr - thai - document-parsing - layout-analysis - vision-language-model --- # Typhoon OCR V2 0.8B (MLX, 8-bit) This repository contains an unofficial conversion of [typhoon-ai/typhoon-ocr2-0.8b](https://huggingface.co/typhoon-ai/typhoon-ocr2-0.8b) to the [MLX](https://github.com/ml-explore/mlx) format, for local inference on Apple silicon Macs. It can be used with any tool that supports MLX vision-language models of the Qwen3.5 architecture. It is not affiliated with, or endorsed by, the original authors. Typhoon OCR V2 0.8B is a 0.8-billion-parameter vision-language model for Thai and English document parsing, developed by Typhoon (SCB 10X). Given an image of a page, it returns the layout elements of the page in reading order, each with a category, a bounding box and its text content. It is a fine-tune of [ATH-MaaS/OvisOCR2](https://huggingface.co/ATH-MaaS/OvisOCR2) and uses the Qwen3.5 architecture (`Qwen3_5ForConditionalGeneration`). ## About this conversion | | | |---|---| | Source model | [typhoon-ai/typhoon-ocr2-0.8b](https://huggingface.co/typhoon-ai/typhoon-ocr2-0.8b) | | Precision | 8-bit affine quantization (group size 64) | | Size on disk | 0.98 GB | | Memory in use | approximately 1.8 GB for typical page images (measured peak resident memory with mlx-vlm); larger images require more | | Conversion tool | mlx-vlm 0.7.12 (`python -m mlx_vlm convert`) | | License | Apache-2.0, inherited from the source model | This variant quantizes the weights to 8 bits using MLX affine quantization with a group size of 64 (9.4 bits per weight on average, because quantization scales and unquantized components are included). It uses about 40% less memory and runs faster than the bfloat16 variant. In our limited testing it produced text nearly identical to bfloat16, but it occasionally omitted a text block that bfloat16 recovered (see Evaluation). The conversion was performed without any additional training or modification of the weights other than the dtype or quantization described above. ## Usage This model is in the standard MLX format and can be loaded by any MLX-compatible tool that supports Qwen3.5 vision-language models. Whichever tool is used, apply the prompt given below and set the temperature to 0. ### LM Studio In the LM Studio app, search for `jitvimol/typhoon-ocr2-0.8b-mlx-8bit` in the model search, or paste the repository URL, and download it. Alternatively, use the command line: ```bash lms get https://huggingface.co/jitvimol/typhoon-ocr2-0.8b-mlx-8bit ``` Then load the model, attach a page image in the chat, and use the prompt given below with the temperature set to 0. If large pages are downscaled in your setup, the layout boxes may differ from the reference results (see Limitations). ### mlx-vlm (command line) ```bash pip install -U mlx-vlm python -m mlx_vlm.generate \ --model jitvimol/typhoon-ocr2-0.8b-mlx-8bit \ --image page.jpg \ --prompt "" \ --max-tokens 4096 \ --temperature 0.0 ``` ### mlx-vlm (Python) ```python from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config model_id = "jitvimol/typhoon-ocr2-0.8b-mlx-8bit" model, processor = load(model_id) config = load_config(model_id) prompt = "" formatted = apply_chat_template(processor, config, prompt, num_images=1) result = generate(model, processor, formatted, ["page.jpg"], max_tokens=4096, temperature=0.0, verbose=False) print(result.text) ``` ### Prompt The model was trained on a fixed set of prompts. For full-page parsing, use the following prompt exactly as written (it is the prompt published on the original model card): ```text - Extract layout information from the provided PDF image. - For each layout element, output its bbox, category, and the text content within the bbox. - Bbox format: [x1, y1, x2, y2]. - Allowed layout categories: ['header', 'title', 'text', 'figure', 'table', 'formula', 'figure_caption', 'table_caption', 'formula_caption', 'figure_footnote', 'table_footnote', 'page_footnote', 'footer']. - Text extraction and formatting: 1) For 'figure', the text field must be an empty string. 2) For 'formula', format text as LaTeX. 3) For 'table', format text as HTML. 4) For all other categories (e.g., text, title), format text as Markdown. - The output text must be exactly the original text from the image, with no translation or rewriting. - Sort all layout elements in human reading order. - Final output must be a single JSON object. ``` The output is a JSON list. Each element contains `bbox`, `category` and `text`. Tables are returned as HTML, formulas as LaTeX and other text as Markdown. Please refer to the [original model card](https://huggingface.co/typhoon-ai/typhoon-ocr2-0.8b) for the full description of the output format, the benchmark results and the recommended post-processing. ## Evaluation The numbers below come from a small, informal check of this conversion. They are not a benchmark, and they do not replace the evaluation published by the original authors on the [original model card](https://huggingface.co/typhoon-ai/typhoon-ocr2-0.8b). - Two printed Thai document pages (the examples from the original repository) were processed with the prompt above, at temperature 0, through mlx-vlm and through a second MLX-based runtime. - All outputs were valid JSON, and no output entered a repetition loop. - On the document pages, the text produced by the two runtimes matched with 98 to 99% similarity. - The 8-bit variant matched bfloat16 closely on most inputs, but in some further informal checks it omitted a text block that bfloat16 read. Small text and text inside embedded images were the most likely to be garbled or omitted. ## Limitations - **Transcription errors.** The original model already makes small errors on Thai text, such as substituted characters, tone marks or vowels. Do not rely on its output for critical values (for example account numbers, amounts or names) without verification. - **Image resolution and layout boxes.** The original card recommends allowing images of up to 16.7 megapixels (`max_pixels`). Some inference tools do not expose this setting, and large pages may then be downscaled. In that case the number and size of layout elements can differ from results obtained with the original settings, even when the text is similar. - **Repetition.** The model can repeat text until the token limit is reached. Use temperature 0 and a bounded `max_tokens`. - **Scope.** As stated by the original authors, this 0.8B model is recommended only as a full-page parser and is much weaker on other tasks. It has no safety filtering and can output text that is not present in the image. - **Quantization.** Quantization generally affects OCR more than conversational use. The 4-bit variant was not tested and is not published. ## License and attribution The model weights are derived from Typhoon OCR V2 0.8B and are distributed under the Apache License 2.0, as is the original. All credit for the model belongs to the authors at Typhoon (SCB 10X) and to the authors of the base model, OvisOCR2. Use of the model is also subject to the [OpenTyphoon Terms and Conditions](https://opentyphoon.ai/tac). ## Citation If you use this model, please cite the original work: ```bibtex @misc{nonesung2026typhoonocropenvisionlanguage, title = {Typhoon OCR: Open Vision-Language Model For Thai Document Extraction}, author = {Surapon Nonesung and Natapong Nitarach and Teetouch Jaknamon and Pittawat Taveekitworachai and Kunat Pipatanakul}, year = {2026}, eprint = {2601.14722}, archivePrefix = {arXiv}, primaryClass = {cs.CL}, url = {https://arxiv.org/abs/2601.14722} } ```