Transformers documentation

UnlimitedOcr

Transformers

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v5.20.1).

Join the Hugging Face community

and get access to the augmented documentation experience

Collaborate on models, datasets and Spaces

Faster examples with accelerated inference

Switch between documentation themes

to get started

This model was published in HF papers on 2026-06-23 and contributed to Hugging Face Transformers on 2026-10-14.

UnlimitedOcr

Overview

The UnlimitedOcr model was proposed in Unlimited OCR Works by Youyang Yin, Huanhuan Liu, Qunyi Xie, Chaorun Liu, Shiqi Yang, Shaohua Wang, Zhanlong Liu, Hao Zou, Jinyue Chen, Shu Wei, Jingjing Wu, Mingxin Huang, Zhen Wu, Guibin Wang, Tengyu Du, and Lei Jia from Baidu Inc.

It is a single 3B parameter model with a standard context length of 32,768 tokens.

The abstract from the paper is the following:

Recently, end-to-end OCR models, exemplified by DeepSeek OCR, have once again thrust OCR into the spotlight. A widely held view is that employing a large language model (LLM) as the decoder allows the model to leverage the prior distribution of language, leading to improved OCR performance. However, the downside is equally evident: as the output sequence lengthens, the accumulated KV cache drives up memory consumption and progressively slows down generation. This stands in stark contrast to humans, who exhibit no such decline in efficiency during long-horizon copying tasks. In this technical report, we propose Unlimited OCR, a model designed to emulate human parsing working memory. Taking DeepSeek OCR as the baseline, we replace all attention layers in the decoder with our proposed Reference Sliding Window Attention (R-SWA), which reduces attention computation costs while maintaining a constant KV cache throughout the entire decoding process. By combining the high compression rate of DeepSeek OCR’s encoder with our constant KV cache design, Unlimited OCR can transcribe dozens of pages of documents in a single forward pass under a standard maximum length of 32K. More importantly, R-SWA is a general-purpose parsing attention mechanism — beyond OCR, it is equally applicable to tasks such as ASR, translation, etc. Codes and model weights are publicly available at http://github.com/baidu/Unlimited-OCR

Unlimited-OCR supports two inference configurations: the default “gundam” mode uses 640x640 tiles with dynamic cropping for high-resolution documents, and “base” mode uses a single 1024x1024 global view for standard-resolution inputs. To enable base mode set crop_to_patches=False in the processor, or pass it via processor_kwargs when using apply_chat_template().

The vision tower follows the two-stage approach from DeepSeek-OCR-2: a SAM ViT-B encoder feeds into a CLIP ViT encoder. Unlike DeepSeek-OCR-2, the CLIP features are additionally concatenated with the SAM features to yield the final image tokens. Unlimited-OCR also omits the learnable patch queries from DeepSeek-OCR-2.

The text model is identical to DeepSeek-OCR-2 with the additional Reference Sliding Window Attention (R-SWA). R-SWA applies only to generated tokens. All image and prompt tokens remain fully visible throughout decoding, so long documents do not lose context from earlier pages.

This model was contributed by guarin. The original code can be found here.

The original implementation runs the model with torch.autocast in bfloat16. Wrap the forward and generate calls in an autocast context to reproduce its outputs.
with torch.autocast(device_type=model.device.type, dtype=torch.bfloat16):
    output = model.generate(**inputs)

Single-page OCR

Multi-page OCR

from transformers import AutoProcessor, AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained("baidu/Unlimited-OCR", device_map="auto")
processor = AutoProcessor.from_pretrained("baidu/Unlimited-OCR")

image = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/ocr_suggestion_form.jpg"
messages = [
    {
        "role": "user",
        "content": [{"type": "image", "url": image}, {"type": "text", "text": "document parsing."}],
    }
]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt"
).to(model.device, model.dtype)

output = model.generate(**inputs, max_new_tokens=512)
processor.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True)
# image [383, 87, 497, 171]\ntext [333, 201, 558, 230]R&D QUALITY IMPROVEMENT\nSUGGESTION/SOLUTION FORM...

# All bounding boxes are in (x1, y1, x2, y2) format with coordinates normalized to [0, 999]

Batch processing

For batch processing, pass a list of messages. Set padding=True for the processor if images have different sizes.

from transformers import AutoProcessor, AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained("baidu/Unlimited-OCR", device_map="auto")
processor = AutoProcessor.from_pretrained("baidu/Unlimited-OCR")

image1 = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/ocr_suggestion_form.jpg"
image2 = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/ocr_receipt.jpeg"
messages = [
    [
        {
            "role": "user",
            "content": [{"type": "image", "url": image}, {"type": "text", "text": "document parsing."}],
        }
    ]
    for image in [image1, image2]
]
inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
    processor_kwargs={"padding": True},
).to(model.device, model.dtype)

output = model.generate(**inputs)
processor.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True)
# image [383, 87, 497, 171]\ntext [333, 201, 558, 230]R&D QUALITY IMPROVEMENT\nSUGGESTION/SOLUTION FORM...

# All bounding boxes are in (x1, y1, x2, y2) format with coordinates normalized to [0, 999]

Region detections

Set skip_special_tokens=False to wrap all detections and region types in <|det|>...<|/det|> markers. This is useful for further post-processing of the output, for example to plot the detected bounding boxes on the image. Each detection is wrapped as <|det|>region_type [x1, y1, x2, y2]<|/det|>text... with coordinates normalized to a [0, 999] range. Set return_detections=True to get an additional list of dictionaries with all detections parsed as {"region_type": region_type, "box": [x1, y1, x2, y2], "text": "..."}.

from transformers import AutoProcessor, AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained("baidu/Unlimited-OCR", device_map="auto")
processor = AutoProcessor.from_pretrained("baidu/Unlimited-OCR")

image = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/ocr_suggestion_form.jpg"
messages = [
    {
        "role": "user",
        "content": [{"type": "image", "url": image}, {"type": "text", "text": "document parsing."}],
    }
]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt"
).to(model.device, model.dtype)

output = model.generate(**inputs)
decoded, detections = processor.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=False, return_detections=True)
# <|det|>image [383, 87, 497, 171]<|/det|>\n<|det|>text [333, 201, 558, 230]<|/det|>R&D QUALITY IMPROVEMENT\nSUGGESTION/SOLUTION FORM...

# Visualization
import random
import matplotlib.pyplot as plt
import matplotlib.patches as patches
from transformers.image_utils import load_image

image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/ocr_suggestion_form.jpg")
width, height = image.size

figure, axis = plt.subplots(figsize=(10, 12))
axis.imshow(image)
for detection in detections:
    region_type = detection["region_type"]
    x1, y1, x2, y2 = detection["box"]
    x1, y1, x2, y2 = int(x1) / 999 * width, int(y1) / 999 * height, int(x2) / 999 * width, int(y2) / 999 * height
    color = (random.random(), random.random(), random.random())
    rectangle = patches.Rectangle((x1, y1), x2 - x1, y2 - y1, linewidth=1.5, edgecolor=color, facecolor="none")
    axis.add_patch(rectangle)
    axis.text(x1, y1, region_type, color="white", fontsize=8, backgroundcolor=color, verticalalignment="top")
axis.axis("off")
plt.show()

Transformers

UnlimitedOcr

Overview

Batch processing

Region detections

UnlimitedOcrConfig

class transformers.UnlimitedOcrConfig

UnlimitedOcrTextConfig

class transformers.UnlimitedOcrTextConfig

UnlimitedOcrVisionConfig

class transformers.UnlimitedOcrVisionConfig

UnlimitedOcrVisionEncoderConfig

class transformers.UnlimitedOcrVisionEncoderConfig

UnlimitedOcrSamVisionConfig

class transformers.UnlimitedOcrSamVisionConfig

UnlimitedOcrImageProcessor

class transformers.UnlimitedOcrImageProcessor

crop_image_to_patches

get_number_of_image_patches

pad_to_square

UnlimitedOcrProcessor

class transformers.UnlimitedOcrProcessor

batch_decode

decode

UnlimitedOcrPreTrainedModel

class transformers.UnlimitedOcrPreTrainedModel

UnlimitedOcrTextPreTrainedModel

class transformers.UnlimitedOcrTextPreTrainedModel

UnlimitedOcrTextModel

class transformers.UnlimitedOcrTextModel

forward

UnlimitedOcrVisionModel

class transformers.UnlimitedOcrVisionModel

forward

UnlimitedOcrModel

class transformers.UnlimitedOcrModel

forward

UnlimitedOcrForConditionalGeneration

class transformers.UnlimitedOcrForConditionalGeneration

forward