Add SGLang-Omni serving instructions to README

#1
Files changed (1) hide show
  1. README.md +10 -0
README.md CHANGED
@@ -201,6 +201,16 @@ MING_MODEL_VARIANT=tiny CUDA_VISIBLE_DEVICES=0 \
201
 
202
  The validated Tiny SGLang profile uses one GPU and provides text output. See the [SGLang example](https://github.com/inclusionAI/Ming-3.0-audio-flash/blob/main/examples/sglang_audio_chat.py) for client usage.
203
 
 
 
 
 
 
 
 
 
 
 
204
  ## Evaluation
205
 
206
  The shared benchmark figures below include results for both Ming-3.0-audio-tiny and Ming-3.0-audio-flash.
 
201
 
202
  The validated Tiny SGLang profile uses one GPU and provides text output. See the [SGLang example](https://github.com/inclusionAI/Ming-3.0-audio-flash/blob/main/examples/sglang_audio_chat.py) for client usage.
203
 
204
+ ### SGLang-Omni
205
+
206
+ [SGLang-Omni](https://sgl-project.github.io/sglang-omni/) serves transcription, audio chat, and speech synthesis from one server. The Tiny checkpoint fits on a single GPU; `--talker.gpu 0` colocates speech synthesis with the other stages:
207
+
208
+ ```bash
209
+ sgl-omni serve --model-path inclusionAI/Ming-3.0-audio-tiny --talker.gpu 0
210
+ ```
211
+
212
+ The server exposes OpenAI-compatible `/v1/audio/transcriptions`, `/v1/audio/speech`, and `/v1/chat/completions` endpoints. The [SGLang-Omni cookbook](https://sgl-project.github.io/sglang-omni/cookbook/ming3_audio.html) shows client examples; the same requests work with the Tiny model name.
213
+
214
  ## Evaluation
215
 
216
  The shared benchmark figures below include results for both Ming-3.0-audio-tiny and Ming-3.0-audio-flash.