Instructions to use vllm-sr/vela3-lite with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use vllm-sr/vela3-lite with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("zero-shot-classification", model="vllm-sr/vela3-lite", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModel processor = AutoProcessor.from_pretrained("vllm-sr/vela3-lite", trust_remote_code=True) model = AutoModel.from_pretrained("vllm-sr/vela3-lite", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Keep adapter passes out of d3's HIP graphs
With D3_GRAPH_TOKENS > 0, runtime.warmup() captures d3's HIP graphs while every Vela adapter is off. A later pass with an adapter on replays one of those graphs whenever its shape fits, and a graph never reads the adapter gate (vela3_adapters.GATE), so router questions get plain d3-lite answers. On one MI300X with D3_GRAPH_TOKENS=8192, the four router questions moved by up to 0.828 after warm-up against the same requests before it.
With this change, d3's fast path reports no graph shape while an adapter is on, so those passes run eagerly. Passes with every adapter off still replay d3's graphs. Nothing changes at the default D3_GRAPH_TOKENS=0, or off ROCm, where d3's fast path is not installed.
Tested on one MI300X with D3_GRAPH_TOKENS=8192 on 200-token requests: four router questions, router questions mixed with ordinary ones, four ordinary questions, and router questions with a PII span question, each answered before and after runtime.warmup(). Every question with an adapter on now gets the same answer after warm-up as before it, bit for bit. Ordinary questions in a pass of their own still replay d3's graphs and differ from eager by up to 0.0033, as d3-lite does on its own with graphs.