Update vLLM-Omni serving usage in README

#3
Files changed (1) hide show
  1. README.md +29 -0
README.md CHANGED
@@ -140,6 +140,35 @@ See the [Tiny examples](https://github.com/inclusionAI/Ming-3.0-tts#ming-30-tts-
140
  python tools/preflight_tiny.py --model_path /path/to/Ming-3.0-tts-tiny
141
  ```
142
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
143
  ## Serving with SGLang-Omni
144
 
145
  [SGLang-Omni](https://sgl-project.github.io/sglang-omni/) serves this checkpoint on one GPU:
 
140
  python tools/preflight_tiny.py --model_path /path/to/Ming-3.0-tts-tiny
141
  ```
142
 
143
+ ## Serving with vLLM-Omni
144
+
145
+ For installation, follow the [vLLM-Omni official installation guide](https://docs.vllm.ai/projects/vllm-omni/en/latest/getting_started/installation/); [vLLM-Omni](https://github.com/vllm-project/vllm-omni) serves this checkpoint through the OpenAI-compatible `/v1/audio/speech` endpoint. The current recipe targets vLLM 0.32.0. Use a separate environment from the reference inference setup above.
146
+
147
+ Start the server in one terminal:
148
+
149
+ ```bash
150
+ vllm serve inclusionAI/Ming-3.0-tts-tiny --omni --port 8000
151
+ ```
152
+
153
+ The HF model ID selects the two-stage Ming3-TTS pipeline automatically and downloads the weights and required assets on first launch. Both stages share GPU 0. A complete local model directory may also be used; pass `--revision <HF-commit>` to pin a remote checkpoint.
154
+
155
+ After the server is ready, run this reference-free voice design example in another terminal:
156
+
157
+ ```bash
158
+ curl --fail-with-body --silent --show-error http://127.0.0.1:8000/v1/audio/speech \
159
+ -H 'Content-Type: application/json' \
160
+ -d '{
161
+ "model": "inclusionAI/Ming-3.0-tts-tiny",
162
+ "input": "傍晚的风从山口吹过来,远处的云慢慢染成了金色。",
163
+ "instructions": "{\"风格\":\"年轻女声,声音清晰自然。\"}",
164
+ "response_format": "wav",
165
+ "extra_params": {"temperature": 0.0, "ming3_cfg": 1.0},
166
+ "seed": 1234
167
+ }' --output ming3_tiny_design.wav
168
+ ```
169
+
170
+ The output is a mono 44.1 kHz WAV. Voice cloning uses `ref_audio` and its matching transcript in `ref_text`; speech controls use `instructions`. For voice cloning, dialect and emotion controls, built-in voices, multi-speaker Podcast, and PCM streaming, see the upstream [Ming-3.0-TTS recipe](https://github.com/vllm-project/vllm-omni/blob/main/recipes/inclusionAI/Ming-3.0-TTS.md).
171
+
172
  ## Serving with SGLang-Omni
173
 
174
  [SGLang-Omni](https://sgl-project.github.io/sglang-omni/) serves this checkpoint on one GPU: