ryanlee-dev Isotr0py commited on
Commit
6534ea1
·
1 Parent(s): 622d674

Update vllm-omni deployment guide (#1)

Browse files

- Update vllm-omni deployment guide (ead029e740a7d1fcd89f7637511dcd0136fce70c)
- Merge remote-tracking branch 'origin/main' into pr1-resolve (935ffcc19bb10a0aacd0cc0cd4d07ab5878432c5)


Co-authored-by: Isotr0py <Isotr0py@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +21 -1
README.md CHANGED
@@ -154,10 +154,30 @@ hf download MiniMaxAI/MiniMax-H3-Image-PE
154
  We recommend the following inference frameworks to serve the model:
155
 
156
  - [SGLang](https://docs.sglang.io/) - see [cookbook](https://docs.sglang.io/cookbook/diffusion/MiniMax/MiniMax-H3-image)
157
- - [vLLM](https://github.com/vllm-project/vllm) - see [vllm recipes](https://recipes.vllm.ai/MiniMaxAI/MiniMax-H3-Image)
158
  - [diffusers](https://github.com/huggingface/diffusers) - see [diffusers docs](https://huggingface.co/docs/diffusers/main/en/api/pipelines/minimax_h3_image)
159
  - [ComfyUI](https://github.com/Comfy-Org/ComfyUI) - see [Comfy tutorial](https://docs.comfy.org/tutorials/image/minimax/minimax-h3-image); use [T2I template]() / [Edit template]()
160
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
161
  #### SGLang Deployment
162
 
163
  Here we use SGLang as a deployment example. See the [MiniMax-H3-Image deployment guide](https://docs.sglang.io/cookbook/diffusion/MiniMax/MiniMax-H3-image) for
 
154
  We recommend the following inference frameworks to serve the model:
155
 
156
  - [SGLang](https://docs.sglang.io/) - see [cookbook](https://docs.sglang.io/cookbook/diffusion/MiniMax/MiniMax-H3-image)
157
+ - [vLLM-Omni](https://github.com/vllm-project/vllm-omni) - see [vllm-omni recipe](https://recipes.vllm.ai/MiniMaxAI/MiniMax-H3-Image)
158
  - [diffusers](https://github.com/huggingface/diffusers) - see [diffusers docs](https://huggingface.co/docs/diffusers/main/en/api/pipelines/minimax_h3_image)
159
  - [ComfyUI](https://github.com/Comfy-Org/ComfyUI) - see [Comfy tutorial](https://docs.comfy.org/tutorials/image/minimax/minimax-h3-image); use [T2I template]() / [Edit template]()
160
 
161
+ #### vLLM-Omni Deployment
162
+
163
+ See the [vLLM-Omni recipe](https://recipes.vllm.ai/MiniMaxAI/MiniMax-H3-Image) for
164
+ per-hardware deployment configurations and measured latency. Both NVIDIA and AMD accelerators are
165
+ supported, including NVIDIA H200, RTX PRO 6000 Blackwell, AMD Instinct MI350X and more.
166
+
167
+ Start the server on one GPU:
168
+
169
+ ```bash
170
+ vllm serve MiniMaxAI/MiniMax-H3-Image --omni --host 0.0.0.0 --port 8000
171
+ ```
172
+
173
+ For lower latency, split each request over several GPUs with sequence parallelism
174
+ (Ulysses), for example on 4 GPUs:
175
+
176
+ ```bash
177
+ vllm serve MiniMaxAI/MiniMax-H3-Image --omni --host 0.0.0.0 --port 8000 \
178
+ --num-gpus 4 --usp 4 --text-encoder-tp-size 4
179
+ ```
180
+
181
  #### SGLang Deployment
182
 
183
  Here we use SGLang as a deployment example. See the [MiniMax-H3-Image deployment guide](https://docs.sglang.io/cookbook/diffusion/MiniMax/MiniMax-H3-image) for