Instructions to use XingChen-AGI/TeleOCR with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use XingChen-AGI/TeleOCR with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="XingChen-AGI/TeleOCR", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("XingChen-AGI/TeleOCR", trust_remote_code=True) model = AutoModelForMultimodalLM.from_pretrained("XingChen-AGI/TeleOCR", trust_remote_code=True, device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use XingChen-AGI/TeleOCR with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "XingChen-AGI/TeleOCR" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "XingChen-AGI/TeleOCR", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/XingChen-AGI/TeleOCR
- SGLang
How to use XingChen-AGI/TeleOCR with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "XingChen-AGI/TeleOCR" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "XingChen-AGI/TeleOCR", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "XingChen-AGI/TeleOCR" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "XingChen-AGI/TeleOCR", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use XingChen-AGI/TeleOCR with Docker Model Runner:
docker model run hf.co/XingChen-AGI/TeleOCR
Incompatible with current transformers (5.x) and vLLM: ROPE "default", tied-weights format, mask signature, cache_position, and non-standard head_dim=128 — model loads but generates garbage
Model: XingChen-AGI/TeleOCR (revision a32e21d4f766317a6f42f341be57d374362bf843) Use case: full-book OCR of a 335-page Russian chemistry monograph (1956), document parsing → Markdown/LaTeX.
Environment: Python 3.12.16, Windows 11, RTX 5070 12GB, torch 2.14.1+cu132, transformers 5.17.0 (also tested 5.10.0 and 4.51.3), vLLM official Docker image (Nov 2026).
The model card says only pip install -U transformers — no version pin, so I assumed latest 5.x is supported. It is not, currently.
Summary. The custom modeling_naviocr.py targets an older transformers API. With transformers 5.17 the model cannot be loaded without 6 separate patches, and even after patching it loads but produces garbage output. Native vLLM loading also fails due to a non-standard attention shape (head_dim=128 with hidden_size=1024), which neither the stock Qwen2_5_VLForConditionalGeneration nor vLLM can build.
Issue 1 — AutoModelForMultimodalLM is missing from auto_map → falls back to stock class → weight mismatch. auto_map only contains AutoModel, AutoModelForCausalLM, AutoModelForImageTextToText, AutoModelForVision2Seq, so the card-recommended AutoModelForMultimodalLM resolves to the stock class, which builds the text model with head_dim = hidden_size // num_heads = 64 and crashes:
text
model.language_model.layers.{0...27}.self_attn.q_proj.weight | MISMATCH - ckpt: [2048, 1024] vs model: [1024, 1024]
model.language_model.layers.{0...27}.self_attn.k_proj.weight | MISMATCH - ckpt: [1024, 1024] vs model: [512, 1024]
RuntimeError: You set ignore_mismatched_sizes to False ...
The checkpoint was clearly trained with head_dim=128 (16 heads × 128 = 2048, GQA 8×128 = 1024), which config.json does declare — but this loader path ignores it. Workaround: load via AutoModelForImageTextToText (it is mapped to the custom class). Request: add "AutoModelForMultimodalLM" to auto_map or fix the card.
Issue 2 — ROPE: KeyError: 'default'. rope_scaling = {"mrope_section": [16, 24, 24], "rope_type": "default", "type": "default"} → custom code does ROPE_INIT_FUNCTIONS[self.rope_type] (modeling_naviocr.py:508), but transformers 5.x removed "default" from that registry. Additionally, 5.x weight-init calls module.compute_default_rope_parameters(...) on any *RotaryEmbedding with rope_type "default" — the class doesn't implement it → 'NoneType' object is not callable. Note: "default" + mrope_section is itself unusual; stock Qwen2.5-VL does not declare rope this way — I suspect this is why generation stays broken (Issue 6).
Issue 3 — _tied_weights_keys must be a dict in 5.x. _tied_weights_keys = ["lm_head.weight"] → AttributeError: 'list' object has no attribute 'keys' in get_expanded_tied_weights_keys. Embed path here is model.language_model.embed_tokens, not model.embed_tokens.
Issue 4 — cache_position arrives None in prepare_inputs_for_generation (modeling_naviocr.py:1626 → TypeError: 'NoneType' object is not subscriptable). 5.x API change; also past_key_values.get_seq_length() was renamed. Getting this wrong silently re-injects pixel_values at every decode step.
Issue 5 — mask utils signature: create_causal_mask() got an unexpected keyword argument 'input_embeds' (5.17 expects inputs_embeds, no cache_position).
Issue 6 — after all patches: loads, but generation is garbage (fp16 and bf16, greedy, real document pages): iiiiiiii…, tabletabletable…, >{>{>{…. My hypothesis is the mrope handling (Issue 2): the model needs Qwen2.5-VL-style 3D rope with mrope_section [16,24,24] over head_dim=128, but the patched "default" path applies plain 1D rope — vision-token positions end up wrong.
Issue 7 — vLLM cannot load the model natively. Since model_type=qwen2_5_vl, vLLM uses its native implementation and never looks at auto_map:
text
assert sum(self.mrope_section) == rotary_dim // 2 # mrope.py:296 → 64 != 32 (AssertionError)
vLLM derives head_dim = hidden_size // num_heads = 64 and ignores config.head_dim = 128. Patching that line then crashes the vision tower (shared line in qwen2_5_vl.py:671), so a proper fix must be scoped to the text stack.
Requests:
Pin exact supported versions in the model card (transformers/torch/vllm), instead of pip install -U transformers.
Add AutoModelForMultimodalLM to auto_map, or correct the card.
Publish a transformers-5.x-compatible modeling_naviocr.py (or state clearly that transformers <5.0 is required — currently neither 4.51 nor 5.x works out of the box).
Clarify the intended rope config: is rope_type: "default" + mrope_section really what was trained with? A corrected config.json might make the model loadable by stock transformers and vLLM without custom code.
Ideally a vLLM-compatible revision or a GGUF (the head_dim=128 "wide attention" is invisible to both stock transformers and vLLM config parsers).
Patches I applied locally (for reference): (1) rope fallback + compute_default_rope_parameters(); (2) _tied_weights_keys = {"lm_head.weight": "model.language_model.embed_tokens.weight"}; (3) cache_position fallback; (4) mask_kwargs "input_embeds"→"inputs_embeds", dropped cache_position; (5) load via AutoModelForImageTextToText, popped unsupported mm_token_type_ids. Happy to share the full diff or test any fixed revision — the model is #1 on OmniDocBench and I'd really like to use it.