Xunzhuo commited on
Commit
5ce995d
·
verified ·
1 Parent(s): 7f257ad

Card: shorter vLLM Semantic Router command

Browse files

The "Run with vLLM Semantic Router" section now uses `vllm-sr serve -e vllm-sr/d3`: the platform is detected automatically, and one line lists the `--platform` values (auto, cpu, cuda, rocm; default auto). The parity sentence keeps its measurement context (AMD Instinct MI325X). MODEL_MANIFEST.json records the new README.md (SHA-256 and size) in the same commit. Weights, runtime files and the v3.1.0 tag are unchanged.

Files changed (2) hide show
  1. MODEL_MANIFEST.json +2 -2
  2. README.md +5 -3
MODEL_MANIFEST.json CHANGED
@@ -592,7 +592,7 @@
592
  "files_sha256": {
593
  "LICENSE": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4",
594
  "NOTICE": "61f4ea488e1fed0c3fce97171506a23152c2591116442f78f15827fb9e6af469",
595
- "README.md": "fec0348ded4425d20f15b87cbe782eb6c76646b871806e0d444ccad3ed77de4f",
596
  "assets/banner.png": "eb7ecc949232e1de544247899dd55ee629a36cfe276042d5795f65178b9c64c5",
597
  "assets/example-receipt.png": "b9dbb8b103bacd45f1de824d844a3c07b81d57cb1a7466712ef060ad7dcb3926",
598
  "assets/example-video.mp4": "f4968ef89d07e97f20df4fd8047510a7a8e9df5c2d07c35e363a97927f164ab9",
@@ -633,7 +633,7 @@
633
  "files_bytes": {
634
  "LICENSE": 11357,
635
  "NOTICE": 100,
636
- "README.md": 10307,
637
  "assets/banner.png": 2233137,
638
  "assets/example-receipt.png": 65918,
639
  "assets/example-video.mp4": 26300,
 
592
  "files_sha256": {
593
  "LICENSE": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4",
594
  "NOTICE": "61f4ea488e1fed0c3fce97171506a23152c2591116442f78f15827fb9e6af469",
595
+ "README.md": "f9a814de11d297c0080241e408f067b62f7d245792ad7e2099a31e4e5090797a",
596
  "assets/banner.png": "eb7ecc949232e1de544247899dd55ee629a36cfe276042d5795f65178b9c64c5",
597
  "assets/example-receipt.png": "b9dbb8b103bacd45f1de824d844a3c07b81d57cb1a7466712ef060ad7dcb3926",
598
  "assets/example-video.mp4": "f4968ef89d07e97f20df4fd8047510a7a8e9df5c2d07c35e363a97927f164ab9",
 
633
  "files_bytes": {
634
  "LICENSE": 11357,
635
  "NOTICE": 100,
636
+ "README.md": 10401,
637
  "assets/banner.png": 2233137,
638
  "assets/example-receipt.png": 65918,
639
  "assets/example-video.mp4": 26300,
README.md CHANGED
@@ -144,15 +144,17 @@ Images go before the text of the request, each read at up to 1.6 megapixels; vid
144
 
145
  ## Run with vLLM Semantic Router
146
 
147
- [vLLM Semantic Router](https://github.com/vllm-project/semantic-router) serves Decision 3.0 natively in Engine mode, which runs one model behind the System One API on your own machine. Its answers match this package's own `d3_server.py` exactly, for text, images and videos. On an AMD Instinct MI325X:
148
 
149
  ```bash
150
  curl -fsSL https://vllm-sr.ai/install.sh | bash -s -- --channel dev --no-launch
151
  export PATH="$HOME/.local/bin:$PATH"
152
- vllm-sr serve vllm-sr/d3 --engine --platform rocm
153
  ```
154
 
155
- Engine mode and Decision 3.0 are in the development channel (`--channel dev`) until a stable vLLM Semantic Router release includes them. The first start pulls the `vllm-sr-rocm` image and downloads the model. Then ask questions about a request:
 
 
156
 
157
  ```bash
158
  curl -s localhost:8899/v1/systemone -H 'content-type: application/json' -d '{
 
144
 
145
  ## Run with vLLM Semantic Router
146
 
147
+ [vLLM Semantic Router](https://github.com/vllm-project/semantic-router) serves Decision 3.0 natively in Engine mode, which runs one model behind the System One API on your own machine. Its answers match this package's own `d3_server.py` exactly, for text, images and videos (measured on AMD Instinct MI325X).
148
 
149
  ```bash
150
  curl -fsSL https://vllm-sr.ai/install.sh | bash -s -- --channel dev --no-launch
151
  export PATH="$HOME/.local/bin:$PATH"
152
+ vllm-sr serve -e vllm-sr/d3
153
  ```
154
 
155
+ The platform is detected automatically; choose one with `--platform auto|cpu|cuda|rocm` (default `auto`).
156
+
157
+ Engine mode and Decision 3.0 are in the development channel (`--channel dev`) until a stable vLLM Semantic Router release includes them. The first start pulls the image for your machine and downloads the model. Then ask questions about a request:
158
 
159
  ```bash
160
  curl -s localhost:8899/v1/systemone -H 'content-type: application/json' -d '{