typhoon-ai/typhoon-ocr2-2b-GGUF

Typhoon OCR V2 2B GGUF

GGUF files of Typhoon OCR V2 2B for llama.cpp and tools built on it.

Typhoon OCR V2 2B is a document parser for Thai and English. Given an image of a page, it returns the page's layout elements in reading order, each with a category, a bounding box and its text. Tables come out as HTML, formulas as LaTeX and other text as Markdown. It also answers questions about a document and extracts key information as JSON. The main model card has the training details, full benchmark results, examples and limitations.

Use of this model is subject to the OpenTyphoon Terms and Conditions.

Files and accuracy

File	Precision	Size	MDPBench ↑	MDPBench Thai ↑
Original weights (bf16, vLLM)	–	–	79.8	81.3
typhoon-ocr2-2b-f16.gguf	16-bit float, no quantization	3.78 GB	79.5	80.8
typhoon-ocr2-2b-Q8_0.gguf	8-bit	2.01 GB	79.5	80.4
typhoon-ocr2-2b-Q4_K_M.gguf	4-bit (k-quant, medium)	1.27 GB	78.4	79.7
mmproj-typhoon-ocr2-2b-f16.gguf	image encoder (f16), needed with every file above	668 MB

MDPBench has 320 Thai and English pages, both digital and photographed. We ran every file through llama-server (llama.cpp build a691b23, CUDA, NVIDIA H100) with the prompt below, greedy decoding, thinking off and an image budget of 16,384 tokens (16.7 MP), the same settings as the vLLM numbers in the first row. We recommend Q8_0: it scored the same as f16 (79.5) at about half the size. Q4_K_M is about 1 point lower and is the choice when memory is tight. Differences of a few tenths of a point between files are within run-to-run variation.

Usage with llama.cpp

-hf downloads the model file and the matching mmproj image encoder:

llama-server -hf typhoon-ai/typhoon-ocr2-2b-GGUF:Q8_0 \
  --jinja --reasoning off \
  -c 40960 --image-max-tokens 16384 \
  -ngl 999 --port 8080

With local files, pass them directly: -m typhoon-ocr2-2b-Q8_0.gguf --mmproj mmproj-typhoon-ocr2-2b-f16.gguf.

• --image-max-tokens 16384 lets a page use up to 16.7 MP (one image token covers 32×32 pixels). Without it, llama.cpp scales large pages down further, and small text gets harder to read.
• --reasoning off keeps the chat template from opening a thinking block. The model was trained without one.
• -c 40960 leaves room for a 16,384-token image and a 16,384-token answer. To serve several requests at once, add -np N and multiply -c by N.

Then call the OpenAI-compatible endpoint, with PROMPT and the helpers from the next section:

import base64
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="EMPTY")

with open("page.jpg", "rb") as f:
    image_url = "data:image/jpeg;base64," + base64.b64encode(f.read()).decode()

resp = client.chat.completions.create(
    model="typhoon-ocr2",
    messages=[{"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": image_url}},
        {"type": "text", "text": PROMPT},
    ]}],
    temperature=0.0,
    max_tokens=16384,
)
elements = parse_elements(resp.choices[0].message.content)
print(to_markdown(elements))

Prompt and output

The model was trained on a fixed set of prompts. For full-page parsing, use this prompt exactly as written:

PROMPT = """- Extract layout information from the provided PDF image.
- For each layout element, output its bbox, category, and the text content within the bbox.
- Bbox format: [x1, y1, x2, y2].
- Allowed layout categories: ['header', 'title', 'text', 'figure', 'table', 'formula', 'figure_caption', 'table_caption', 'formula_caption', 'figure_footnote', 'table_footnote', 'page_footnote', 'footer'].
- Text extraction and formatting:
  1) For 'figure', the text field must be an empty string.
  2) For 'formula', format text as LaTeX.
  3) For 'table', format text as HTML.
  4) For all other categories (e.g., text, title), format text as Markdown.
- The output text must be exactly the original text from the image, with no translation or rewriting.
- Sort all layout elements in human reading order.
- Final output must be a single JSON object."""

The model returns a JSON list of elements inside a json code block. Bounding boxes use a 0–1000 grid relative to the page width and height. These helpers parse the output and turn it into Markdown:

import json, re

def parse_elements(raw: str) -> list[dict]:
    """Parse the model output: a JSON list of elements inside a ```json fence."""
    m = re.search(r"```json\n(.*?)(?:\n```|$)", raw, re.DOTALL)
    text = (m.group(1) if m else raw).strip()
    try:
        return json.loads(text)
    except json.JSONDecodeError:
        # Output stopped at the token limit: drop the last, incomplete element.
        cut = text.rfind('{"bbox":')
        return json.loads(text[:cut].rstrip().rstrip(",") + "]")

def to_markdown(elements: list[dict], keep_header_footer: bool = False) -> str:
    skip = set() if keep_header_footer else {"header", "footer", "page_footnote"}
    return "\n\n".join(e["text"] for e in elements if e.get("text") and e["category"] not in skip)

For questions about a document, send the question as the text prompt instead. For key-information extraction and the other tasks, see the main model card.

Limitations

• Quantization costs some accuracy; compare the scores above before picking a smaller file.
• We tested llama.cpp on Linux with CUDA only. CPU, Metal and Vulkan builds were not tested, and their outputs can differ slightly.
• The model can fall into repetition loops on dense multi-column pages. parse_elements above keeps the complete elements when that happens.
• All limitations of the original model apply.

Support

• Discord: https://discord.gg/us5gAYmrxw
• X (Twitter): https://twitter.com/opentyphoon