Zero-Shot Classification
Transformers
Safetensors
qwen3_5
feature-extraction
decision-model
classification
system-one
multimodal
vision
video
custom_code
Eval Results
Instructions to use vllm-sr/d3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use vllm-sr/d3 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("zero-shot-classification", model="vllm-sr/d3", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModel processor = AutoProcessor.from_pretrained("vllm-sr/d3", trust_remote_code=True) model = AutoModel.from_pretrained("vllm-sr/d3", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Card: run with vLLM Semantic Router (Engine mode)
Browse filesAdds the "Run with vLLM Semantic Router" section: install the vllm-sr CLI from the development channel, `vllm-sr serve vllm-sr/d3 --engine --platform rocm`, and text, image and video requests to the System One API. MODEL_MANIFEST.json records the new README.md (SHA-256 and size) in the same commit. Weights, runtime files and the v3.1.0 tag are unchanged.
- MODEL_MANIFEST.json +2 -2
- README.md +55 -0
MODEL_MANIFEST.json
CHANGED
|
@@ -592,7 +592,7 @@
|
|
| 592 |
"files_sha256": {
|
| 593 |
"LICENSE": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4",
|
| 594 |
"NOTICE": "61f4ea488e1fed0c3fce97171506a23152c2591116442f78f15827fb9e6af469",
|
| 595 |
-
"README.md": "
|
| 596 |
"assets/banner.png": "eb7ecc949232e1de544247899dd55ee629a36cfe276042d5795f65178b9c64c5",
|
| 597 |
"assets/example-receipt.png": "b9dbb8b103bacd45f1de824d844a3c07b81d57cb1a7466712ef060ad7dcb3926",
|
| 598 |
"assets/example-video.mp4": "f4968ef89d07e97f20df4fd8047510a7a8e9df5c2d07c35e363a97927f164ab9",
|
|
@@ -633,7 +633,7 @@
|
|
| 633 |
"files_bytes": {
|
| 634 |
"LICENSE": 11357,
|
| 635 |
"NOTICE": 100,
|
| 636 |
-
"README.md":
|
| 637 |
"assets/banner.png": 2233137,
|
| 638 |
"assets/example-receipt.png": 65918,
|
| 639 |
"assets/example-video.mp4": 26300,
|
|
|
|
| 592 |
"files_sha256": {
|
| 593 |
"LICENSE": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4",
|
| 594 |
"NOTICE": "61f4ea488e1fed0c3fce97171506a23152c2591116442f78f15827fb9e6af469",
|
| 595 |
+
"README.md": "fec0348ded4425d20f15b87cbe782eb6c76646b871806e0d444ccad3ed77de4f",
|
| 596 |
"assets/banner.png": "eb7ecc949232e1de544247899dd55ee629a36cfe276042d5795f65178b9c64c5",
|
| 597 |
"assets/example-receipt.png": "b9dbb8b103bacd45f1de824d844a3c07b81d57cb1a7466712ef060ad7dcb3926",
|
| 598 |
"assets/example-video.mp4": "f4968ef89d07e97f20df4fd8047510a7a8e9df5c2d07c35e363a97927f164ab9",
|
|
|
|
| 633 |
"files_bytes": {
|
| 634 |
"LICENSE": 11357,
|
| 635 |
"NOTICE": 100,
|
| 636 |
+
"README.md": 10307,
|
| 637 |
"assets/banner.png": 2233137,
|
| 638 |
"assets/example-receipt.png": 65918,
|
| 639 |
"assets/example-video.mp4": 26300,
|
README.md
CHANGED
|
@@ -142,6 +142,61 @@ print(json.dumps(result["answers"], indent=2))
|
|
| 142 |
|
| 143 |
Images go before the text of the request, each read at up to 1.6 megapixels; videos follow the images, read at 2 frames per second (at most 32 frames spread over the whole video, each at up to 0.2 megapixels). Every question of the request sees all of them.
|
| 144 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 145 |
## Evaluation
|
| 146 |
|
| 147 |
### Text: Jev Decision Index 0.3.1
|
|
|
|
| 142 |
|
| 143 |
Images go before the text of the request, each read at up to 1.6 megapixels; videos follow the images, read at 2 frames per second (at most 32 frames spread over the whole video, each at up to 0.2 megapixels). Every question of the request sees all of them.
|
| 144 |
|
| 145 |
+
## Run with vLLM Semantic Router
|
| 146 |
+
|
| 147 |
+
[vLLM Semantic Router](https://github.com/vllm-project/semantic-router) serves Decision 3.0 natively in Engine mode, which runs one model behind the System One API on your own machine. Its answers match this package's own `d3_server.py` exactly, for text, images and videos. On an AMD Instinct MI325X:
|
| 148 |
+
|
| 149 |
+
```bash
|
| 150 |
+
curl -fsSL https://vllm-sr.ai/install.sh | bash -s -- --channel dev --no-launch
|
| 151 |
+
export PATH="$HOME/.local/bin:$PATH"
|
| 152 |
+
vllm-sr serve vllm-sr/d3 --engine --platform rocm
|
| 153 |
+
```
|
| 154 |
+
|
| 155 |
+
Engine mode and Decision 3.0 are in the development channel (`--channel dev`) until a stable vLLM Semantic Router release includes them. The first start pulls the `vllm-sr-rocm` image and downloads the model. Then ask questions about a request:
|
| 156 |
+
|
| 157 |
+
```bash
|
| 158 |
+
curl -s localhost:8899/v1/systemone -H 'content-type: application/json' -d '{
|
| 159 |
+
"model": "vllm-sr/d3",
|
| 160 |
+
"state": "Write a Python function that merges two sorted lists.",
|
| 161 |
+
"questions": {
|
| 162 |
+
"kind": {"type": "choice", "instructions": "What kind of work is this?",
|
| 163 |
+
"criteria": {"code": "Writing or fixing code", "math": "Mathematics", "chat": "Anything else"}},
|
| 164 |
+
"reasoning": {"type": "noul", "instructions": "Does answering this need multi-step reasoning?"}
|
| 165 |
+
}
|
| 166 |
+
}'
|
| 167 |
+
```
|
| 168 |
+
|
| 169 |
+
Send images as base64 data URLs (PNG, JPEG or WebP) in `images` and videos (MP4, WebM, QuickTime or Matroska) in `videos`; every question sees them:
|
| 170 |
+
|
| 171 |
+
```bash
|
| 172 |
+
curl -s localhost:8899/v1/systemone -H 'content-type: application/json' --data-binary @- <<EOF
|
| 173 |
+
{
|
| 174 |
+
"model": "vllm-sr/d3",
|
| 175 |
+
"state": "What does the attached image show?",
|
| 176 |
+
"images": ["data:image/png;base64,$(base64 -w0 chart.png)"],
|
| 177 |
+
"questions": {
|
| 178 |
+
"kind": {"type": "choice", "instructions": "What kind of image is this?",
|
| 179 |
+
"criteria": {"chart": "A chart or plot", "photo": "A photograph", "document": "A document"}}
|
| 180 |
+
}
|
| 181 |
+
}
|
| 182 |
+
EOF
|
| 183 |
+
```
|
| 184 |
+
|
| 185 |
+
```bash
|
| 186 |
+
curl -s localhost:8899/v1/systemone -H 'content-type: application/json' --data-binary @- <<EOF
|
| 187 |
+
{
|
| 188 |
+
"model": "vllm-sr/d3",
|
| 189 |
+
"state": "What happens in the clip?",
|
| 190 |
+
"videos": ["data:video/mp4;base64,$(base64 -w0 clip.mp4)"],
|
| 191 |
+
"questions": {
|
| 192 |
+
"moving": {"type": "noul", "instructions": "Does something move across the scene?"}
|
| 193 |
+
}
|
| 194 |
+
}
|
| 195 |
+
EOF
|
| 196 |
+
```
|
| 197 |
+
|
| 198 |
+
`vllm-sr stop` stops it. See [Router and Engine modes](https://github.com/vllm-project/semantic-router/blob/main/src/vllm-sr/INSTANCE_MODES.md#ask-about-images-and-videos) for limits and options.
|
| 199 |
+
|
| 200 |
## Evaluation
|
| 201 |
|
| 202 |
### Text: Jev Decision Index 0.3.1
|