MoganOCR-TR-MTP
An MTP (multi-token prediction) head for moganai/MoganOCR-TR. It lets vLLM run the model with speculative decoding: 1.57× faster generation with the same output quality. The main model weights are not changed.
MoganOCR-TR ships without the MTP head of its base model, Qwen3.5-2B. This repo takes that head and retrains it on MoganOCR-TR's own outputs.
Files
| File | What it is |
|---|---|
mtp.safetensors |
The MTP head (15 tensors, 122 MB) |
model.safetensors.index.json |
Index that adds the head to the original model.safetensors |
Usage
Download the original model, then add the two files to the same folder:
hf download moganai/MoganOCR-TR --local-dir MoganOCR-TR
hf download burkap/MoganOCR-TR-MTP mtp.safetensors model.safetensors.index.json --local-dir MoganOCR-TR
Run with vLLM (tested on 0.33.0):
import base64
from vllm import LLM, SamplingParams
llm = LLM(
model="MoganOCR-TR",
max_model_len=32768,
limit_mm_per_prompt={"image": 1},
speculative_config={"method": "mtp", "num_speculative_tokens": 3},
)
image = base64.b64encode(open("page.png", "rb").read()).decode()
messages = [{"role": "user", "content": [
{"type": "image_url", "image_url": {"url": f"data:image/png;base64,{image}"}},
{"type": "text", "text": "Sayfayı markdown olarak yaz."},
]}]
out = llm.chat(messages, SamplingParams(temperature=0, max_tokens=8000),
chat_template_kwargs={"enable_thinking": False})
print(out[0].outputs[0].text)
Without speculative_config the model behaves exactly like the original. The measurements below were run with the environment variable VLLM_USE_FLASHINFER_SAMPLER=0.
Results
All 488 pages of TurkishOCR-Bench, one request at a time, RTX 3090 Ti, vLLM 0.33.0, greedy decoding:
| Setting | Tokens/s | Speedup |
|---|---|---|
| No speculation | 156 | 1.00× |
| Original Qwen3.5-2B head, 2 draft tokens | 191 | 1.22× |
| This head, 2 draft tokens | 230 | 1.47× |
| This head, 3 draft tokens | 245 | 1.57× |
The TurkishOCR-Bench score is 89.2 in all four settings.
Speculative decoding only keeps draft tokens that the main model agrees with, so the output quality is that of the original model. Speed under batched load and on other GPUs was not measured.
Training
The head was initialised from the Qwen3.5-2B MTP head and trained by self-distillation: MoganOCR-TR transcribed 7,821 pages, and the head learned to predict the model's next tokens on those outputs. No TurkishOCR-Bench page was used for training.
The table counts all 8,000 cached pages; 179 of them were held out for validation.