Instructions to use vllm-sr/d3.1-nano with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use vllm-sr/d3.1-nano with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("zero-shot-classification", model="vllm-sr/d3.1-nano", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModel processor = AutoProcessor.from_pretrained("vllm-sr/d3.1-nano", trust_remote_code=True) model = AutoModel.from_pretrained("vllm-sr/d3.1-nano", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
d3.1-nano
d3.1-nano is a 2B multimodal foundation decision model of Decision 3.0, the decision models of vLLM Semantic Router. Give it an input (text or JSON, optionally with images and videos) and the questions you need answered: pick one of several options, say yes or no, or rate on a scale. It answers them all in one call and returns a probability for every answer, without generating text.
| Parameters | 2.21B, including the 0.33B vision encoder |
| Inputs | Text or JSON, plus images and videos (several per request) |
| Decision types | Choice · Yes / No · Score |
| License | Apache-2.0 (third-party notices in NOTICE) |
Highlights
- Jev Decision Index 0.3, public suite: 62.96, measured with the official kit on the released weights: all 140,178 public requests answered, none unsupported (results). The official board score is pending.
- +20.8 over d3-nano on the public suite (42.2), ahead in all five of its areas, from +14.0 in retrieval to +32.0 in arts.
- Reads images: multiple images per request (PNG, JPEG or WebP), given as paths, URLs, PIL images or base64 data URLs; every question of the request sees all of them.
- Reads videos: multiple videos per request (MP4, WebM, MOV or MKV), given as paths, URLs, base64 data URLs or frame arrays, read at 2 frames per second.
- Many questions, one call: Choice, Yes / No and Score questions about the same input are answered together, each from its own forward pass over the input, with a probability for every option.
- Same API as d3-nano:
system_one, the pipeline and the runtime options are unchanged; switch by changing the model name.
Quickstart
pip install "transformers==5.17.0" torch torchvision pillow opencv-python-headless safetensors accelerate
pip install flash-linear-attention # optional: fast GPU kernels for the linear-attention layers
import json
from huggingface_hub import hf_hub_download
from transformers import AutoModel
model = AutoModel.from_pretrained("vllm-sr/d3.1-nano", trust_remote_code=True)
# Text
result = model.system_one(
state="The order arrived damaged yesterday. The customer has a receipt and asks for a replacement today.",
questions={
"route": {
"type": "choice",
"instructions": "Which team should handle this request?",
"criteria": {
"returns": "Refunds, replacements and damaged deliveries",
"billing": "Payments, invoices and charges",
"technical": "Product setup and faults"
}
},
"receipt": {
"type": "noul",
"instructions": "Does the customer have a receipt?"
},
"urgency": {
"type": "score",
"instructions": "How urgent is this request?",
"criteria": [
"Routine",
"Soon",
"Today"
]
}
},
)
print(json.dumps(result["answers"], indent=2))
# Text and an image (or several)
receipt = hf_hub_download("vllm-sr/d3.1-nano", "assets/example-receipt.png")
result = model.system_one(
state="The customer says the blender arrived cracked and attached the receipt.",
images=[receipt], # local paths, http(s) URLs, PIL images or base64 data URLs
questions={
"route": {
"type": "choice",
"instructions": "Which team should handle this request?",
"criteria": {
"returns": "Refunds, replacements and damaged deliveries",
"billing": "Payments, invoices and charges",
"technical": "Product setup and faults"
}
},
"on_receipt": {
"type": "noul",
"instructions": "Does the receipt list the blender?"
},
"payment": {
"type": "choice",
"instructions": "How was the order paid?",
"criteria": {
"card": None,
"cash": None,
"gift card": None
}
}
},
)
print(json.dumps(result["answers"], indent=2))
# Text and a video (or several)
clip = hf_hub_download("vllm-sr/d3.1-nano", "assets/example-video.mp4")
result = model.system_one(
state="A short clip from a test camera.",
videos=[clip], # local paths, http(s) URLs, base64 data URLs or frame arrays
questions={
"direction": {
"type": "choice",
"instructions": "Which way does the square move?",
"criteria": {
"right": "From left to right",
"left": "From right to left",
"still": "It does not move"
}
},
"color_change": {
"type": "noul",
"instructions": "Does the square change color?"
}
},
)
print(json.dumps(result["answers"], indent=2))
# Or as a pipeline:
# transformers.pipeline("decision", model="vllm-sr/d3.1-nano", trust_remote_code=True)(state=..., questions=..., images=..., videos=...)
Images go before the text of the request, each read at up to 1.6 megapixels; videos follow the images, read at 2 frames per second (at most 32 frames spread over the whole video, each at up to 0.2 megapixels). Every question of the request sees all of them.
Run with vLLM Semantic Router
vLLM Semantic Router serves Decision 3.0 natively in Engine mode, which runs one model behind the System One API on your own machine. d3.1-nano needs a build that includes commit e0008e2 of its main branch or a later one.
curl -fsSL https://vllm-sr.ai/install.sh | bash -s -- --channel dev --no-launch
export PATH="$HOME/.local/bin:$PATH"
vllm-sr serve -e vllm-sr/d3.1-nano
The platform is detected automatically; choose one with --platform auto|cpu|cuda|rocm (default auto). Then ask questions about a request:
curl -s localhost:8899/v1/systemone -H 'content-type: application/json' -d '{
"model": "vllm-sr/d3.1-nano",
"state": "Write a Python function that merges two sorted lists.",
"questions": {
"kind": {"type": "choice", "instructions": "What kind of work is this?",
"criteria": {"code": "Writing or fixing code", "math": "Mathematics", "chat": "Anything else"}},
"reasoning": {"type": "noul", "instructions": "Does answering this need multi-step reasoning?"}
}
}'
Send images as base64 data URLs (PNG, JPEG or WebP) in images and videos (MP4, WebM, QuickTime or Matroska) in videos; every question sees them. vllm-sr stop stops it. See Router and Engine modes for limits and options.
Serving performance
On one AMD Instinct MI300X, with the package's Decision Index engine (two option orders, see below):
| Latency, one request at a time | p50 462 ms, p90 776 ms (Decision Index requests, which often carry many questions and long inputs) |
| The whole public suite | 140,620 requests in 4 h 13 min with the default batch size of 8 |
Measured in November 2026 with ROCm 7.2, PyTorch 2.12 and Transformers 5.17.0.
Evaluation
Text: Jev Decision Index 0.3
| Model | Public ↑ | Knowledge ↑ | Language ↑ | Retrieval ↑ | Tools ↑ | Arts ↑ |
|---|---|---|---|---|---|---|
| d3.1-nano | 62.96 | 42.65 | 62.74 | 72.79 | 85.88 | 54.43 |
| d3-nano | 42.22 | 25.3 | 42.2 | 58.8 | 58.9 | 22.4 |
Reproduce the public suite with the official kit (apolinario/decision-index at 25c6ec7):
hf download vllm-sr/d3.1-nano --local-dir d3.1-nano
PYTHONPATH=d3.1-nano python -m decision_index run --engine d3_engine:D3Engine \
--option model=d3.1-nano --option device=cuda:0 --out runs/d3.1-nano --compact
python -m decision_index score --results runs/d3.1-nano/results.jsonl --engine d3_engine:D3Engine \
--out runs/d3.1-nano/score
The Decision Index engine reads each choice question in two option orders, the given one and its reverse, and averages the two probabilities of each option; --option permutation_average=0 reads one order. AutoModel, the pipeline and vLLM Semantic Router read one order.
No Decision Index evaluation items were used in training. Images and videos have not been evaluated for this version yet.
d3.1-nano: public suite of the official 0.3 kit, run on 2026-11-13 on one AMD Instinct MI300X; results and scores in vllm-sr/d3.1-nano-decision-index. d3-nano: official leaderboard, text board 0.3.2, 2026-10-12 (public index 42.22 with the official 0.3 kit, as on its card).
License
Apache-2.0 (LICENSE). Built on Qwen/Qwen3.5-2B (Apache-2.0). Third-party notices: NOTICE.
Citation
@misc{d3_1_nano_2026,
title = {{d3.1-nano}: A Multimodal Foundation Decision Model},
author = {{vLLM Semantic Router Team}},
year = {2026},
howpublished = {\url{https://huggingface.co/vllm-sr/d3.1-nano}}
}
Trained on AMD Instinct MI300X GPUs.
- Downloads last month
- 140

