Fix remote-code compatibility across Transformers 4.x and 5.x

#14
by Bvicii - opened

trust_remote_code=True selects this repository's custom model implementation, including when Transformers already has native PaddleOCR-VL support. The current remote implementation fails at different stages across releases:

  • Transformers 4.57.6 loads the checkpoint, then fails on the first forward with create_causal_mask() got an unexpected keyword argument 'inputs_embeds'.
  • Transformers 5.0.0 and 5.21.1 fail during construction with KeyError: 'default' from the RoPE initializer registry.
  • Fixing those exceptions alone exposes generation-position and initialization problems, including incorrect visual non-persistent buffers and, on 5.0.0, reinitialization of already-loaded visual parameters.

This draft keeps the existing AutoModelForCausalLM / AutoProcessor loading API and trust_remote_code=True. It:

  • initializes default RoPE locally using the original frequency formula;
  • builds causal/padding masks locally, avoiding the changing create_causal_mask keyword interface;
  • obtains generation positions from prepared inputs or cache length, and retains images when generating without a KV cache;
  • explicitly initializes derived RoPE/position buffers and guards custom initializers against modifying loaded parameters;
  • isolates packed image attention by image in the non-FlashAttention path, fixing cross-image contamination in batched OCR;
  • adds a reproducible full-checkpoint regression script, three fixtures, and a machine-readable validation summary.

Validation

Tested the same source with the full PaddlePaddle/PaddleOCR-VL-1.6 checkpoint at c5630abae1d940eafe0697512a0325494b02ab42, Python 3.12, PyTorch 2.14.1, CPU float32:

Transformers Full checkpoint and regression suite
4.54.1 Passed
4.57.0 Passed
4.57.6 Passed
5.0.0 Passed
5.13.0 Passed
5.13.1 Passed
5.20.2 Passed
5.21.1 Passed

For every version, all loaded parameters exactly match the checkpoint. Checks cover causal/padding/4-D boolean masks, English generation and EOS, cached/uncached token equality for a complete English example, Chinese OCR, table recognition, and equality of batched OCR text with individual outputs. Batched versus individual prefill logits have a maximum absolute difference of 4.3392181396484375e-05.

From a checkout containing the model weights:

python -m pip install einops sentencepiece protobuf
python tests/test_remote_code_compat.py --full --model . --inputs tests/data --output /tmp/paddleocr-vl-compat

Draft scope

The eight listed versions were executed; this is not a claim that every historical, intermediate, or future release is verified. CUDA/MPS, low precision, FlashAttention, StaticCache, training, and dataset-level accuracy/performance benchmarks still need validation. The current patch is for the 1.6 repository; other model repositories need their own port and checkpoint regression.

Related model repositories

A source audit of the current PaddlePaddle/PaddleOCR-VL (1.0) and PaddlePaddle/PaddleOCR-VL-1.5 repositories found the same default-RoPE registry lookup, create_causal_mask call using inputs_embeds, and unguarded cache_position[0] access. Their configs also select default RoPE. These shared loading compatibility defects are therefore not specific to 1.6. This draft's full-checkpoint validation is limited to 1.6; it does not claim that all initialization and batching symptoms were reproduced with the 1.0/1.5 weights.

Superseded by the explicitly created draft PR #15: https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.6/discussions/15

Bvicii changed pull request status to closed

Sign up or log in to comment