Instructions to use hitxhslhy/realocr with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use hitxhslhy/realocr with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="hitxhslhy/realocr")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("hitxhslhy/realocr")
model = AutoModelForMultimodalLM.from_pretrained("hitxhslhy/realocr", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use hitxhslhy/realocr with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "hitxhslhy/realocr"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hitxhslhy/realocr",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/hitxhslhy/realocr

SGLang

How to use hitxhslhy/realocr with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "hitxhslhy/realocr" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hitxhslhy/realocr",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "hitxhslhy/realocr" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hitxhslhy/realocr",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use hitxhslhy/realocr with Docker Model Runner:
```
docker model run hf.co/hitxhslhy/realocr
```

realocr / VALIDATION.md

hitxhslhy

Add Qwen3.5-2B-MDPBench-R7 merged weights + inference code

c395244 verified 6 days ago

preview code

Raw History Blame Contribute Delete

3.01 kB

Validation record — merged weights

Everything below was actually executed on this machine, not assumed.

1. Weight integrity

Check	Result
`model.safetensors`	4,426,558,864 bytes (4.43 GB), bf16
Origin	`outputs/exp_R7_grpo_remine/best_merged` (merged from `best/` adapter + `Qwen/Qwen3.5-2B`)
`config.json`	`architectures: ["Qwen3_5ForConditionalGeneration"]`, `model_type: qwen3_5`, `dtype: bfloat16`
Tokenizer / processor files	`tokenizer.json` (20.0 MB), `tokenizer_config.json`, `chat_template.jinja`, `processor_config.json` — all present
`adapter_config.json`	absent, as required for a merged (non-LoRA) model

2. End-to-end smoke test on the merged weights

Environment: torch 2.5.1+cu124, transformers 5.3.0, 1 GPU.

python inference/infer_mdpbench.py --model . --local-files-only \
    --image-dir .../MDPBench_img_sub408 --out-dir /tmp/merged_out --limit 2

Result: 2/2 pages, 0 empty, 21.1 s/page. The merged model loaded through plain Qwen3_5ForConditionalGeneration.from_pretrained(path) — no adapter, no peft needed at load time. Output is correct Arabic Markdown.

3. Merged vs adapter vs stored predictions (same 2 pages)

Character-level similarity (difflib.SequenceMatcher.ratio):

Pair	page 1	page 2
merged vs unmerged adapter run	99.951 %	99.610 %
merged vs Sep-18 stored predictions	99.967 %	99.629 %
merged vs repo script re-run	99.935 %	99.590 %
(for reference) unmerged adapter vs Sep-18 stored	99.951 %	99.863 %

Absolute differences are a handful of characters (e.g. insert/delete of one Arabic letter, a missing . in et al.). The merged weights are in fact as close to the stored predictions as the unmerged adapter is.

Why any difference at all: merging computes W + (alpha/r)·B·A in float32 and rounds the result back to bf16, whereas the adapter path computes the LoRA delta at runtime and adds it in bf16. Different rounding order → a token flips occasionally under greedy decoding. Impact on the aggregate score is negligible: MDPBench's clean_string drops non-word characters before scoring, and the affected characters are < 0.4 % of the output.

4. Notes / known warnings

Keyword argument 'enable_thinking' is not a valid argument for this processor and will be ignored. — harmless; the original reference script passes the same flag, so behaviour is identical to how the reported numbers were produced.
The fast path is not available ... Falling back to torch implementation. — refers to optional flash-linear-attention / causal-conv1d; installing them only speeds things up, it does not change results.
Requires transformers>=5.3.0. torch>=2.4 recommended.
Output is deterministic for a fixed environment (two runs of the same weights produced byte-identical files during the adapter-package validation), but not bit-identical across different torch/CUDA versions.