hanwu311 commited on
Commit
364b3b0
·
verified ·
1 Parent(s): 2ed214c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -4
README.md CHANGED
@@ -111,6 +111,9 @@ Use a framework build that includes Step 5 Preview support, including the reason
111
  | vLLM | BF16 | Two H200 nodes, 16 GPUs | TP16, TEP16, or DEP16 | OpenAI-compatible API |
112
  | Transformers | FP8 | One H200 node, 8 GPUs | TP8 / EP8 | Offline generation |
113
  | Transformers | BF16 | Two H200 nodes, 16 GPUs | TP16 / EP16 | Offline generation (see validation note below) |
 
 
 
114
 
115
  DP, TP, and EP refer to data, tensor, and expert parallelism, respectively. TEP combines tensor and expert parallelism; DEP combines data and expert parallelism. These example configurations are sized for the deployments below. Actual memory requirements, usable context length, and concurrency vary with the checkpoint, inference framework, and KV cache allocation.
116
 
@@ -425,10 +428,6 @@ This launches 16 processes across the two nodes, matching the 16-shard checkpoin
425
 
426
  ### SGLang
427
 
428
- | Framework | Precision | Hardware | Parallelism | Interface |
429
- | --- | --- | --- | --- | --- |
430
- | SGLang | FP8 | One B300 node, 4 GPUs | TP4/EP4 | OpenAI-compatible API |
431
- | SGLang | FP8 | One H200 node, 8 GPUs | TP8/EP8 | OpenAI-compatible API |
432
 
433
  #### Environment Setup
434
 
 
111
  | vLLM | BF16 | Two H200 nodes, 16 GPUs | TP16, TEP16, or DEP16 | OpenAI-compatible API |
112
  | Transformers | FP8 | One H200 node, 8 GPUs | TP8 / EP8 | Offline generation |
113
  | Transformers | BF16 | Two H200 nodes, 16 GPUs | TP16 / EP16 | Offline generation (see validation note below) |
114
+ | SGLang | FP8 | One B300 node, 4 GPUs | TP4/EP4 | OpenAI-compatible API |
115
+ | SGLang | FP8 | One H200 node, 8 GPUs | TP8/EP8 | OpenAI-compatible API |
116
+
117
 
118
  DP, TP, and EP refer to data, tensor, and expert parallelism, respectively. TEP combines tensor and expert parallelism; DEP combines data and expert parallelism. These example configurations are sized for the deployments below. Actual memory requirements, usable context length, and concurrency vary with the checkpoint, inference framework, and KV cache allocation.
119
 
 
428
 
429
  ### SGLang
430
 
 
 
 
 
431
 
432
  #### Environment Setup
433