Instructions to use PaddlePaddle/PaddleOCR-VL-1.6 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PaddleOCR
How to use PaddlePaddle/PaddleOCR-VL-1.6 with PaddleOCR:
# See https://www.paddleocr.ai/latest/version3.x/pipeline_usage/PaddleOCR-VL.html to installation from paddleocr import PaddleOCRVL pipeline = PaddleOCRVL(pipeline_version="v1.6") output = pipeline.predict("path/to/document_image.png") for res in output: res.print() res.save_to_json(save_path="output") res.save_to_markdown(save_path="output") - Notebooks
- Google Colab
- Kaggle
Fix remote-code compatibility across Transformers 4.x and 5.x
trust_remote_code=True selects this repository's custom model implementation, including when Transformers already has native PaddleOCR-VL support. The current remote implementation fails at different stages across releases:
- Transformers 4.57.6 loads the checkpoint, then fails on the first forward with
create_causal_mask() got an unexpected keyword argument 'inputs_embeds'. - Transformers 5.0.0 and 5.21.1 fail during construction with
KeyError: 'default'from the RoPE initializer registry. - Fixing those exceptions alone exposes generation-position and initialization problems, including incorrect visual non-persistent buffers and, on 5.0.0, reinitialization of already-loaded visual parameters.
This draft keeps the existing AutoModelForCausalLM / AutoProcessor loading API and trust_remote_code=True. It:
- initializes default RoPE locally using the original frequency formula;
- builds causal/padding masks locally, avoiding the changing
create_causal_maskkeyword interface; - obtains generation positions from prepared inputs or cache length, and retains images when generating without a KV cache;
- explicitly initializes derived RoPE/position buffers and guards custom initializers against modifying loaded parameters;
- isolates packed image attention by image in the non-FlashAttention path, fixing cross-image contamination in batched OCR;
- adds a reproducible full-checkpoint regression script, three fixtures, and a machine-readable validation summary.
Validation
Tested the same source with the full PaddlePaddle/PaddleOCR-VL-1.6 checkpoint at c5630abae1d940eafe0697512a0325494b02ab42, Python 3.12, PyTorch 2.14.1, CPU float32:
| Transformers | Full checkpoint and regression suite |
|---|---|
| 4.54.1 | Passed |
| 4.57.0 | Passed |
| 4.57.6 | Passed |
| 5.0.0 | Passed |
| 5.13.0 | Passed |
| 5.13.1 | Passed |
| 5.20.2 | Passed |
| 5.21.1 | Passed |
For every version, all loaded parameters exactly match the checkpoint. Checks cover causal/padding/4-D boolean masks, English generation and EOS, cached/uncached token equality for a complete English example, Chinese OCR, table recognition, and equality of batched OCR text with individual outputs. Batched versus individual prefill logits have a maximum absolute difference of 4.3392181396484375e-05.
From a checkout containing the model weights:
python -m pip install einops sentencepiece protobuf
python tests/test_remote_code_compat.py --full --model . --inputs tests/data --output /tmp/paddleocr-vl-compat
Draft scope
The eight listed versions were executed; this is not a claim that every historical, intermediate, or future release is verified. CUDA/MPS, low precision, FlashAttention, StaticCache, training, and dataset-level accuracy/performance benchmarks still need validation. The current patch is for the 1.6 repository; other model repositories need their own port and checkpoint regression.
Related model repositories
A source audit of the current PaddlePaddle/PaddleOCR-VL (1.0) and PaddlePaddle/PaddleOCR-VL-1.5 repositories found the same default-RoPE registry lookup, create_causal_mask call using inputs_embeds, and unguarded cache_position[0] access. Their configs also select default RoPE. These shared loading compatibility defects are therefore not specific to 1.6. This draft's full-checkpoint validation is limited to 1.6; it does not claim that all initialization and batching symptoms were reproduced with the 1.0/1.5 weights.
Superseded by the explicitly created draft PR #15: https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.6/discussions/15