typhoon-ai

/

typhoon-ocr2-2b

Like 6

Follow

Typhoon 421

Hi Typhoon team, thanks for releasing V2. On real Thai government memos I hit a consistent misreading, and it reproduces on clean synthetic images.

Problem

In a memo, the document-number prefix "ที่ อก ๐๓๑๔/" (อก is the Ministry of Industry code) comes out as "ที่ ออก ๐๓๑๔/". Every occurrence on the page is affected. Digits, names and the rest of the page are correct.

Minimal repro (clean synthetic images)

I rendered short strings in Tahoma at 64 px, white background, no noise, and used a plain transcription prompt (Read the text in the image exactly, character by character. Output plain text only.) with temperature 0. The script is at the end of this post.

The model reads "อก" correctly on its own, so it does see the glyphs. Once other Thai text precedes it, the frequent word "ออก" wins. I also saw "ที่ อด" → "ที่ อต" and "ที่ อว" → "ที อว", so other short, rare codes may be affected as well.

Token probabilities

These come from the real memo page with the standard layout-parsing prompt and logprobs enabled. They show the token generated right after "ที่":

• first occurrence: ' ออก' 0.79, ' อ' 0.13, 'ออก' 0.01, 'อก' <0.01
• second occurrence on the same page: ' ออก' 0.99 (the model seems to follow its own earlier output)

A possible contributing factor is tokenization. In the V2 tokenizer, ' ออก' is a single token, but ' อก' has to be produced as ' ' + 'อก'. The no-space case also fails, though, so the main cause looks like a language prior and not just tokenization.

Not caused by quantization or resolution

• Real memo, 200 dpi, layout prompt:
  • vLLM 0.31 (fp16, enforce-eager, max_pixels 4 MP): same error
  • Ollama with a Q8_0 GGUF conversion: same error
• Synthetic images above: tested on the Q8_0 GGUF only. I couldn't run the official bf16 weights (my GPU, a Turing card, doesn't support bf16), so a check on your side with bf16 would help.
• For comparison, Typhoon OCR 1.5 (3B, Ollama) reads "ที่ อก ๐๓๑๔/" correctly on the same page.

Why it matters

Thai official correspondence puts a short agency code right after "ที่" in every document number (อก, มท, กค, ศธ, อว ...). A silent change from อก to ออก produces fluent text that looks plausible, so reviewers can easily miss it. Including such codes in the training or evaluation data might help.

Environment: Windows 11, RTX 2080 8 GB, Ollama 0.40.1 / vLLM 0.31.0.

Repro script

"""Minimal repro: Typhoon OCR V2 reads the abbreviation "อก" as the common word "ออก"
when it follows other Thai text.

Renders short Thai strings in Tahoma (clean, large, no noise), asks the model to
transcribe them, and prints whether the output matches.

    python repro_og.py                      # Ollama (default model name below)
    python repro_og.py --vllm http://localhost:8000/v1 --model typhoon-ocr2
"""

import argparse
import base64
import io
import sys

import requests
from PIL import Image, ImageDraw, ImageFont

FONT = r"C:\Windows\Fonts\tahoma.ttf"
PROMPT = "Read the text in the image exactly, character by character. Output plain text only."
CASES = [
    "อก",
    "อก ๐๓๑๔",
    "ที่ อก ๐๓๑๔/",
    "ที่อก ๐๓๑๔/",
    "เลขที่ อก ๐๓๑๔/",
    "คน อก หัก",
    "ดู อก ไก่",
    "ที่ มท ๐๒๑๑/",
    "ที่ กค ๐๔๒๒/",
    "ที่ ศธ ๐๕๑๓/",
]


def render(text: str, size: int = 64) -> Image.Image:
    font = ImageFont.truetype(FONT, size)
    img = Image.new("RGB", (int(size * len(text) * 0.6) + 80, size * 2), "white")
    ImageDraw.Draw(img).text((40, size // 3), text, font=font, fill="black")
    return img


def b64(img: Image.Image) -> str:
    buf = io.BytesIO()
    img.save(buf, "PNG")
    return base64.b64encode(buf.getvalue()).decode()


def ask_ollama(img: Image.Image, args) -> str:
    r = requests.post(f"{args.ollama}/api/chat", json={
        "model": args.model, "stream": False, "think": False,
        "messages": [{"role": "user", "content": PROMPT, "images": [b64(img)]}],
        "options": {"temperature": 0},
    }, timeout=600)
    r.raise_for_status()
    return r.json()["message"]["content"].strip()


def ask_vllm(img: Image.Image, args) -> str:
    r = requests.post(f"{args.vllm}/chat/completions", json={
        "model": args.model, "temperature": 0, "max_tokens": 200,
        "chat_template_kwargs": {"enable_thinking": False},
        "messages": [{"role": "user", "content": [
            {"type": "image_url", "image_url": {"url": "data:image/png;base64," + b64(img)}},
            {"type": "text", "text": PROMPT},
        ]}],
    }, timeout=600)
    r.raise_for_status()
    return r.json()["choices"][0]["message"]["content"].strip()


def main() -> None:
    sys.stdout.reconfigure(encoding="utf-8")
    p = argparse.ArgumentParser()
    p.add_argument("--ollama", default="http://localhost:11434")
    p.add_argument("--vllm", help="OpenAI-compatible base URL, e.g. http://localhost:8000/v1")
    p.add_argument("--model", default="typhoon-ocr2-2b-q8")
    p.add_argument("--save", action="store_true", help="save the rendered images as case_N.png")
    args = p.parse_args()
    ask = ask_vllm if args.vllm else ask_ollama

    for i, text in enumerate(CASES, 1):
        img = render(text)
        if args.save:
            img.save(f"case_{i}.png")
        out = ask(img, args)
        print(f"{'OK  ' if out == text else 'FAIL'} {text!r:22} -> {out!r}")


if __name__ == "__main__":
    main()