dean2022/OXR-1.0/plugins/vllm - Storage Bucket

OXR vLLM plugin

Registers the OXR architecture through vllm.general_plugins. Requires Python 3.12+, vLLM 0.25.1 and Transformers 5.13.1. It does not modify the vLLM installation. The supported Docker runtime uses PyTorch 2.11.0+cu130.

Install from the model repository root in a Linux CUDA environment:

python -m pip install ./plugins/vllm

Start with the model files and weights in the repository root:

VLLM_PLUGINS=oxr vllm serve . \
  --trust-remote-code \
  --served-model-name oxr \
  --chat-template ./chat_template.jinja \
  --dtype bfloat16 \
  --attention-config '{"backend":"TRITON_ATTN"}' \
  --max-model-len 16384

The LA model in layout_analysis/ is served separately by the OXR layout analysis service, not by this plugin.

To roll back to vLLM 0.19.1, use the previous model plugin together with the matching previous OXR launcher and Docker image. Do not install this plugin into the old runtime.