Xunzhuo commited on
Commit
8c4ead9
·
verified ·
1 Parent(s): c91d55d

Card: shorter vLLM Semantic Router command

Browse files

The "Run with vLLM Semantic Router" section now uses `vllm-sr serve -e vllm-sr/d3-lite`: the platform is detected automatically, and one line lists the `--platform` values (auto, cpu, cuda, rocm; default auto). The parity sentence keeps its measurement context (AMD Instinct MI325X). MODEL_MANIFEST.json records the new README.md (SHA-256 and size) in the same commit. Weights, runtime files and the v3.1.0 tag are unchanged.

Files changed (2) hide show
  1. MODEL_MANIFEST.json +2 -2
  2. README.md +5 -3
MODEL_MANIFEST.json CHANGED
@@ -582,7 +582,7 @@
582
  "files_sha256": {
583
  "LICENSE": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4",
584
  "NOTICE": "e7fecd68cfb3c894924d48c642230cd2b96d3cf03ecc13907f5bcc55d52f3a9f",
585
- "README.md": "640c6313f83abed0f876c745ee862c1a657861549e3691485b06728bb3897fb6",
586
  "assets/banner.png": "5aef9308fc700d012c660d75a9f97976e4f1287afd15b46eb69fbd11bfc24949",
587
  "assets/example-receipt.png": "b9dbb8b103bacd45f1de824d844a3c07b81d57cb1a7466712ef060ad7dcb3926",
588
  "assets/example-video.mp4": "f4968ef89d07e97f20df4fd8047510a7a8e9df5c2d07c35e363a97927f164ab9",
@@ -613,7 +613,7 @@
613
  "files_bytes": {
614
  "LICENSE": 11357,
615
  "NOTICE": 106,
616
- "README.md": 10379,
617
  "assets/banner.png": 2233930,
618
  "assets/example-receipt.png": 65918,
619
  "assets/example-video.mp4": 26300,
 
582
  "files_sha256": {
583
  "LICENSE": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4",
584
  "NOTICE": "e7fecd68cfb3c894924d48c642230cd2b96d3cf03ecc13907f5bcc55d52f3a9f",
585
+ "README.md": "c7eebb86c4ca4c098707da6a864c38e4df4797b33025153244d6146d41f2832a",
586
  "assets/banner.png": "5aef9308fc700d012c660d75a9f97976e4f1287afd15b46eb69fbd11bfc24949",
587
  "assets/example-receipt.png": "b9dbb8b103bacd45f1de824d844a3c07b81d57cb1a7466712ef060ad7dcb3926",
588
  "assets/example-video.mp4": "f4968ef89d07e97f20df4fd8047510a7a8e9df5c2d07c35e363a97927f164ab9",
 
613
  "files_bytes": {
614
  "LICENSE": 11357,
615
  "NOTICE": 106,
616
+ "README.md": 10473,
617
  "assets/banner.png": 2233930,
618
  "assets/example-receipt.png": 65918,
619
  "assets/example-video.mp4": 26300,
README.md CHANGED
@@ -144,15 +144,17 @@ Images go before the text of the request, each read at up to 1.6 megapixels; vid
144
 
145
  ## Run with vLLM Semantic Router
146
 
147
- [vLLM Semantic Router](https://github.com/vllm-project/semantic-router) serves Decision 3.0 natively in Engine mode, which runs one model behind the System One API on your own machine. Its answers match this package's own `d3_server.py` exactly, for text, images and videos. On an AMD Instinct MI325X:
148
 
149
  ```bash
150
  curl -fsSL https://vllm-sr.ai/install.sh | bash -s -- --channel dev --no-launch
151
  export PATH="$HOME/.local/bin:$PATH"
152
- vllm-sr serve vllm-sr/d3-lite --engine --platform rocm
153
  ```
154
 
155
- Engine mode and Decision 3.0 are in the development channel (`--channel dev`) until a stable vLLM Semantic Router release includes them. The first start pulls the `vllm-sr-rocm` image and downloads the model. Then ask questions about a request:
 
 
156
 
157
  ```bash
158
  curl -s localhost:8899/v1/systemone -H 'content-type: application/json' -d '{
 
144
 
145
  ## Run with vLLM Semantic Router
146
 
147
+ [vLLM Semantic Router](https://github.com/vllm-project/semantic-router) serves Decision 3.0 natively in Engine mode, which runs one model behind the System One API on your own machine. Its answers match this package's own `d3_server.py` exactly, for text, images and videos (measured on AMD Instinct MI325X).
148
 
149
  ```bash
150
  curl -fsSL https://vllm-sr.ai/install.sh | bash -s -- --channel dev --no-launch
151
  export PATH="$HOME/.local/bin:$PATH"
152
+ vllm-sr serve -e vllm-sr/d3-lite
153
  ```
154
 
155
+ The platform is detected automatically; choose one with `--platform auto|cpu|cuda|rocm` (default `auto`).
156
+
157
+ Engine mode and Decision 3.0 are in the development channel (`--channel dev`) until a stable vLLM Semantic Router release includes them. The first start pulls the image for your machine and downloads the model. Then ask questions about a request:
158
 
159
  ```bash
160
  curl -s localhost:8899/v1/systemone -H 'content-type: application/json' -d '{