Update README.md

README.md

@@ -3,7 +3,6 @@
 license: other
 license_link: https://huggingface.co/stepfun-ai/Step-5-Preview/blob/main/LICENSE.txt
 ---
----
 <p align="center">
   <img src="https://cdn-uploads.huggingface.co/production/uploads/65dc9de42a6348b052a3286e/-R9DERgmQiBfO2cF0HXoD.png" alt="StepFun logo" width="64" />
 </p>
@@ -15,7 +14,14 @@ license_link: https://huggingface.co/stepfun-ai/Step-5-Preview/blob/main/LICENSE
 </p>
 
 <p align="center">
-  <strong>600B total parameters &nbsp;·&nbsp; 27B active parameters &nbsp;·&nbsp; Up to 1M context</strong>
+  600B total parameters &nbsp;·&nbsp; 27B active parameters &nbsp;·&nbsp; Up to 1M context
+</p>
+
+<p align="center">
+  <a href="https://huggingface.co/stepfun-ai/Step-5-Preview"><img src="https://img.shields.io/badge/Hugging_Face-FP8-FFD21E?style=flat-square" alt="Hugging Face FP8" /></a>
+  <a href="https://huggingface.co/stepfun-ai/Step-5-Preview-BF16"><img src="https://img.shields.io/badge/Hugging_Face-BF16-FFD21E?style=flat-square" alt="Hugging Face BF16" /></a>
+  <a href="https://modelscope.ai/models/stepfun-ai/Step-5-Preview"><img src="https://img.shields.io/badge/ModelScope-FP8-624AFF?style=flat-square" alt="ModelScope FP8" /></a>
+  <a href="https://modelscope.ai/models/stepfun-ai/Step-5-Preview-BF16"><img src="https://img.shields.io/badge/ModelScope-BF16-624AFF?style=flat-square" alt="ModelScope BF16" /></a>
 </p>
 
 <p align="center">
@@ -30,7 +36,8 @@ license_link: https://huggingface.co/stepfun-ai/Step-5-Preview/blob/main/LICENSE
   <a href="#model-specifications">Specifications</a> &nbsp;·&nbsp;
   <a href="#performance">Performance</a> &nbsp;·&nbsp;
   <a href="#deployment">Deployment</a> &nbsp;·&nbsp;
-  <a href="#resources">Resources</a>
+  <a href="#resources">Resources</a> &nbsp;·&nbsp;
+  <a href="#license">License</a>
 </p>
 
 ---
@@ -94,44 +101,99 @@ Step 5 Preview can be served through an OpenAI-compatible API with vLLM or SGLan
 
 Use a framework build that includes Step 5 Preview support, including the reasoning, tool-call, and speculative decoding features used in the serving commands.
 
-### Deployment at a Glance
+### Deployment Configurations
 
 | Framework | Precision | Hardware | Parallelism | Interface |
 | --- | --- | --- | --- | --- |
-| vLLM | FP8 | One H200 node, 8 GPUs | DP2 × TP4; EP8 | OpenAI-compatible API |
-| vLLM | BF16 | Two H200 nodes, 16 GPUs | DP4 × TP4; EP16 | OpenAI-compatible API |
+| vLLM | FP8 | One B300 node, 4 GPUs | TP4, TEP4, or DEP4 | OpenAI-compatible API |
+| vLLM | FP8 | One H200 node, 8 GPUs | TP8, TEP8, or DEP8 | OpenAI-compatible API |
+| vLLM | BF16 | One B300 node, 8 GPUs | TP8, TEP8, or DEP8 | OpenAI-compatible API |
+| vLLM | BF16 | Two H200 nodes, 16 GPUs | TP16, TEP16, or DEP16 | OpenAI-compatible API |
 | Transformers | FP8 | One H200 node, 8 GPUs | TP8 / EP8 | Offline generation |
 | Transformers | BF16 | Two H200 nodes, 16 GPUs | TP16 / EP16 | Offline generation (see validation note below) |
 
-DP, TP, and EP refer to data, tensor, and expert parallelism, respectively. These example configurations are sized for the deployments below. Actual memory requirements, usable context length, and concurrency vary with the checkpoint, inference framework, and KV cache allocation.
+DP, TP, and EP refer to data, tensor, and expert parallelism, respectively. TEP combines tensor and expert parallelism; DEP combines data and expert parallelism. These example configurations are sized for the deployments below. Actual memory requirements, usable context length, and concurrency vary with the checkpoint, inference framework, and KV cache allocation.
 
 **Deployment guides:** [vLLM](#vllm) · [Transformers](#transformers) · [SGLang](#sglang) · [Send a Request](#send-a-request)
 
 ### vLLM
 
+The examples below expose Step 5 Preview through an OpenAI-compatible API at port `18000`, with `step5` as the model name used in requests.
+
 #### Environment Setup
 
-We recommend using the official vLLM Docker image:
+Pull the Step 5 Preview image and start an interactive container with GPU access:
 
 ```bash
-docker pull vllm/vllm-openai:latest
+docker pull vllm/vllm-openai:step-5-preview
+
+docker run --rm -it \
+    --gpus all \
+    --ipc=host \
+    --network=host \
+    --volume "$HOME/.cache/huggingface:/root/.cache/huggingface" \
+    --entrypoint /bin/bash \
+    vllm/vllm-openai:step-5-preview
 ```
 
-After pulling the image, follow the [vLLM Docker guide](https://docs.vllm.ai/en/latest/deployment/docker/) to start a GPU-enabled container with access to your model weights. Run the commands below inside the container; the two-node setup also requires network connectivity between the containers.
+Run the serving commands below inside the container. If the checkpoint requires authentication, sign in with `hf auth login` before starting the server.
 
 <details>
-<summary><strong>FP8: One H200 Node, 8 GPUs</strong></summary>
+<summary><strong>FP8: Single Node on 4 B300 GPUs</strong></summary>
+
+#### FP8: Single Node on 4 B300 GPUs
+
+Choose one of the following parallel configurations for a single node with four B300 GPUs:
 
-To serve the FP8 checkpoint on a single 8-GPU H200 node, run the following command. It uses DP2 × TP4 with EP8 and serves the model as `step5`:
+| Mode | Tensor parallel | Data parallel | Expert parallel |
+| :---: | :---: | :---: | :---: |
+| TP4 | 4 | 1 | Disabled |
+| TEP4 | 4 | 1 | 4 |
+| DEP4 | 1 | 4 | 4 |
+
+Make all four GPUs visible, then run the command for your chosen configuration:
+
+```bash
+export CUDA_VISIBLE_DEVICES=0,1,2,3
+```
+
+On B300 (Blackwell), the TEP4 and DEP4 commands select MegaMoE with `--moe-backend deep_gemm_mega_moe`. The TP4 command uses the default MoE backend, since MegaMoE requires expert parallelism.
+
+##### TP4: Tensor Parallelism
 
 ```bash
 vllm serve stepfun-ai/Step-5-Preview \
+    --host 0.0.0.0 \
+    --port 18000 \
     --served-model-name "step5" \
     --tensor-parallel-size 4 \
-    --data-parallel-size 2 \
     --data-parallel-backend mp \
     --distributed-executor-backend mp \
+    --disable-custom-all-reduce \
+    --max-model-len 1048576 \
+    --max-num-batched-tokens 16384 \
+    --max-num-seqs 64 \
+    --gpu-memory-utilization 0.85 \
+    --enable-prefix-caching \
+    --spec-method mtp \
+    --spec-tokens 3 \
+    --reasoning-parser step5 \
+    --enable-auto-tool-choice \
+    --tool-call-parser step5
+```
+
+##### TEP4: Tensor and Expert Parallelism
+
+```bash
+vllm serve stepfun-ai/Step-5-Preview \
+    --host 0.0.0.0 \
+    --port 18000 \
+    --served-model-name "step5" \
+    --tensor-parallel-size 4 \
     --enable-expert-parallel \
+    --moe-backend deep_gemm_mega_moe \
+    --data-parallel-backend mp \
+    --distributed-executor-backend mp \
     --disable-custom-all-reduce \
     --max-model-len 1048576 \
     --max-num-batched-tokens 16384 \
@@ -145,77 +207,144 @@ vllm serve stepfun-ai/Step-5-Preview \
     --tool-call-parser step5
 ```
 
+##### DEP4: Data and Expert Parallelism
+
+```bash
+vllm serve stepfun-ai/Step-5-Preview \
+    --host 0.0.0.0 \
+    --port 18000 \
+    --served-model-name "step5" \
+    --tensor-parallel-size 1 \
+    --data-parallel-size 4 \
+    --enable-expert-parallel \
+    --moe-backend deep_gemm_mega_moe \
+    --data-parallel-backend mp \
+    --distributed-executor-backend mp \
+    --disable-custom-all-reduce \
+    --max-model-len 1048576 \
+    --max-num-batched-tokens 16384 \
+    --max-num-seqs 64 \
+    --gpu-memory-utilization 0.85 \
+    --enable-prefix-caching \
+    --spec-method mtp \
+    --spec-tokens 3 \
+    --reasoning-parser step5 \
+    --enable-auto-tool-choice \
+    --tool-call-parser step5
+```
+
+##### FP8 on H200
+
+For a single node with eight H200 GPUs, adapt one of the B300 commands above:
+
+1. Set `CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7`.
+2. For TP8 or TEP8, set `--tensor-parallel-size 8`. For DEP8, keep `--tensor-parallel-size 1` and set `--data-parallel-size 8`.
+3. For TEP8 or DEP8, retain `--enable-expert-parallel` and remove `--moe-backend deep_gemm_mega_moe`; H200 (Hopper) uses the default MoE backend.
+
+TP and TEP use DP=1; DEP uses TP=1. In all configurations, `--max-num-seqs` applies per DP rank. The 1M-token context limit covers input and output combined. If KV cache memory is insufficient, reduce `--max-model-len` or concurrency.
+
 </details>
 
 <details>
-<summary><strong>BF16: Two H200 Nodes, 16 GPUs</strong></summary>
+<summary><strong>BF16</strong></summary>
+
+#### BF16
 
-For the BF16 checkpoint, run one command on each of two 8-GPU H200 nodes. Both nodes must use the same container image and BF16 checkpoint path, accessible inside each container. Set `DP_ADDRESS` to node 0's reachable IP on the high-speed inter-node network, and use the same `DP_RPC_PORT` on both nodes.
+BF16 deployments require a BF16 checkpoint. Setting `--dtype bfloat16` on the FP8 checkpoint does not convert its quantized weights to BF16.
 
-On **node 0**, launch DP ranks 0–1 and the API server:
+Mount the BF16 checkpoint directory at the same path inside the container, then set `MODEL` to that path:
 
 ```bash
 export MODEL="<STEP5_BF16_MODEL_PATH>"
-export DP_ADDRESS="<NODE_0_HIGH_SPEED_NETWORK_IP>"
-export DP_RPC_PORT=29550
-export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
-vllm serve "$MODEL" \
-  --host 0.0.0.0 \
-  --port 18000 \
-  --served-model-name step5 \
-  --tensor-parallel-size 4 \
-  --data-parallel-size 4 \
-  --data-parallel-size-local 2 \
-  --data-parallel-address "$DP_ADDRESS" \
-  --data-parallel-rpc-port "$DP_RPC_PORT" \
-  --data-parallel-backend mp \
-  --distributed-executor-backend mp \
-  --enable-expert-parallel \
-  --disable-custom-all-reduce \
-  --max-model-len 1048576 \
-  --max-num-batched-tokens 16384 \
-  --max-num-seqs 64 \
-  --gpu-memory-utilization 0.85 \
-  --enable-prefix-caching \
-  --spec-method mtp \
-  --spec-tokens 3 \
-  --reasoning-parser step5 \
-  --enable-auto-tool-choice \
-  --tool-call-parser step5
 ```
 
-On **node 1**, launch DP ranks 2–3 in headless mode:
+Choose a configuration for your hardware:
 
-```bash
-export MODEL="<STEP5_BF16_MODEL_PATH>"
-export DP_ADDRESS="<NODE_0_HIGH_SPEED_NETWORK_IP>"
-export DP_RPC_PORT=29550
-export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
-vllm serve "$MODEL" \
-  --served-model-name step5 \
-  --tensor-parallel-size 4 \
-  --data-parallel-size 4 \
-  --data-parallel-size-local 2 \
-  --data-parallel-address "$DP_ADDRESS" \
-  --data-parallel-rpc-port "$DP_RPC_PORT" \
-  --data-parallel-backend mp \
-  --distributed-executor-backend mp \
-  --enable-expert-parallel \
-  --disable-custom-all-reduce \
-  --max-model-len 1048576 \
-  --max-num-batched-tokens 16384 \
-  --max-num-seqs 64 \
-  --gpu-memory-utilization 0.85 \
-  --enable-prefix-caching \
-  --spec-method mtp \
-  --spec-tokens 3 \
-  --reasoning-parser step5 \
-  --enable-auto-tool-choice \
-  --tool-call-parser step5 \
-  --headless
+| Hardware | BF16 layout | MoE backend |
+| :---: | :---: | :---: |
+| 8 B300 GPUs | TP8, TEP8, or DEP8 | MegaMoE for TEP/DEP; default for TP |
+| 2 × 8 H200 GPUs | TP16, TEP16, or DEP16 | Default |
+
+These configurations provide a starting point for loading the BF16 weights. They do not guarantee enough memory for a 1M-token request; KV cache and runtime buffers also require GPU memory.
+
+##### B300: One Node, 8 GPUs
+
+Set `CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7` inside the container. Start from the [TP4 command](#tp4-tensor-parallelism), replace `stepfun-ai/Step-5-Preview` with `"$MODEL"`, and replace `--tensor-parallel-size 4` with one of the argument sets below:
+
+| Mode | Parallel arguments |
+| :---: | :--- |
+| TP8 | `--tensor-parallel-size 8` |
+| TEP8 | `--tensor-parallel-size 8 --enable-expert-parallel --moe-backend deep_gemm_mega_moe` |
+| DEP8 | `--tensor-parallel-size 1 --data-parallel-size 8 --enable-expert-parallel --moe-backend deep_gemm_mega_moe` |
+
+Retain the remaining serving arguments, including `--served-model-name "step5"`, `--reasoning-parser step5`, and `--tool-call-parser step5`.
+
+##### H200: Two Nodes, 16 GPUs
+
+Use the same BF16 checkpoint path and container image on both nodes. Replace `<NODE_0_IP>` below with node 0's high-speed-network IP, reachable from node 1. H200 uses the default MoE backend; omit `--moe-backend deep_gemm_mega_moe`.
+
+On each node:
+
+1. Set `MODEL` to the mounted BF16 checkpoint path and `CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7`.
+2. Start from the [TP4 command](#tp4-tensor-parallelism) and replace `stepfun-ai/Step-5-Preview` with `"$MODEL"`.
+3. Remove `--host`, `--port`, and `--tensor-parallel-size` from that command, then insert the argument block for your configuration and node below. These blocks contain arguments to append to `vllm serve`, rather than standalone commands.
+
+Keep all other serving arguments, including `--data-parallel-backend mp`, `--distributed-executor-backend mp`, the model name, and both parsers. Run one completed command per node. Only node 0 exposes the API, on port `18000`.
+
+**TP16 / TEP16**
+
+For TP16, use the following arguments. For TEP16, also add `--enable-expert-parallel` on both nodes. Both configurations use DP=1.
+
+Node 0:
+
+```text
+--host 0.0.0.0 \
+--port 18000 \
+--tensor-parallel-size 16 \
+--nnodes 2 \
+--node-rank 0 \
+--master-addr <NODE_0_IP>
 ```
 
-Together, the two nodes run four TP4 groups, with EP16 spanning all 16 GPUs. `--max-num-seqs` applies per DP rank. Only node 0 exposes the API endpoint, on port 18000.
+Node 1:
+
+```text
+--tensor-parallel-size 16 \
+--nnodes 2 \
+--node-rank 1 \
+--master-addr <NODE_0_IP> \
+--headless
+```
+
+**DEP16**
+
+DEP16 uses TP=1 and distributes the experts across 16 data-parallel ranks, with eight ranks on each node. Use the same data-parallel address and RPC port on both nodes.
+
+Node 0:
+
+```text
+--host 0.0.0.0 \
+--port 18000 \
+--tensor-parallel-size 1 \
+--data-parallel-size 16 \
+--data-parallel-size-local 8 \
+--data-parallel-address <NODE_0_IP> \
+--data-parallel-rpc-port 29550 \
+--enable-expert-parallel
+```
+
+Node 1:
+
+```text
+--tensor-parallel-size 1 \
+--data-parallel-size 16 \
+--data-parallel-size-local 8 \
+--data-parallel-start-rank 8 \
+--data-parallel-address <NODE_0_IP> \
+--data-parallel-rpc-port 29550 \
+--enable-expert-parallel \
+--headless
+```
 
 </details>
 
@@ -232,18 +361,23 @@ Use separate checkpoints and output directories for FP8 and BF16. Conversion pre
 <details>
 <summary><strong>FP8: One H200 Node, 8 GPUs</strong></summary>
 
+#### FP8: One H200 Node, 8 GPUs
+
 Convert the FP8 checkpoint once, then launch eight local processes for TP8 / EP8 generation:
 
 ```bash
 export STEP5_FP8_CHECKPOINT="<FP8_CHECKPOINT_DIRECTORY>"
 export STEP5_FP8_OUT="<FP8_SHARD_OUTPUT_DIRECTORY>"
+
 python3 convert.py \
   --checkpoint "$STEP5_FP8_CHECKPOINT" \
   --tp-size 8 --ep-size 8 --check-only
+
 python3 convert.py \
   --checkpoint "$STEP5_FP8_CHECKPOINT" \
   --out-dir "$STEP5_FP8_OUT" \
   --tp-size 8 --ep-size 8
+
 torchrun --standalone --nproc-per-node=8 generate.py \
   --tp-dir "$STEP5_FP8_OUT/tp8" --ep-size 8 \
   --prompt "Please introduce tensor parallelism." \
@@ -255,14 +389,18 @@ torchrun --standalone --nproc-per-node=8 generate.py \
 <details>
 <summary><strong>BF16: Two H200 Nodes, 16 GPUs</strong></summary>
 
+#### BF16: Two H200 Nodes, 16 GPUs
+
 For TP16 / EP16 generation across two nodes, first convert the BF16 checkpoint into 16 shards. Make the resulting `tp16/` and `tokenizer_files/` directories accessible to both nodes:
 
 ```bash
 export STEP5_BF16_CHECKPOINT="<BF16_CHECKPOINT_DIRECTORY>"
 export STEP5_BF16_OUT="<BF16_SHARD_OUTPUT_DIRECTORY>"
+
 python3 convert.py \
   --checkpoint "$STEP5_BF16_CHECKPOINT" \
   --tp-size 16 --ep-size 16 --check-only
+
 python3 convert.py \
   --checkpoint "$STEP5_BF16_CHECKPOINT" \
   --out-dir "$STEP5_BF16_OUT" \
@@ -276,6 +414,7 @@ export STEP5_BF16_OUT="<BF16_SHARD_OUTPUT_DIRECTORY>"
 export MASTER_ADDR="<NODE_0_REACHABLE_IP_OR_HOSTNAME>"
 export MASTER_PORT=29500
 export NODE_RANK=0  # Set to 1 on node 1.
+
 torchrun --nnodes=2 --nproc-per-node=8 \
   --node-rank="$NODE_RANK" \
   --master-addr="$MASTER_ADDR" \
@@ -296,58 +435,192 @@ This launches 16 processes across the two nodes, matching the 16-shard checkpoin
 
 #### Environment Setup
 
-Pull the official SGLang Docker image:
+Pull the Step 5 Preview SGLang Docker image:
 
 ```bash
-docker pull lmsysorg/sglang:latest
+docker pull lmsysorg/sglang:step-5-preview
 ```
 
-After pulling the image, follow the [SGLang installation guide](https://docs.sglang.io/docs/get-started/install) to start a GPU-enabled container with access to your model weights, then run the command below inside it.
+Then, launch the server using the configuration for your hardware.
 
 <details>
-<summary><strong>FP8</strong></summary>
+<summary><strong>B300: Low latency (4 GPUs, TP4/EP1)</strong></summary>
+
+#### B300: Low latency (4 GPUs, TP4/EP1)
 
-The following command serves the FP8 checkpoint as `step5` with TP4 / EP1 and EAGLE speculative decoding. The selected attention and MoE backends require compatible GPU hardware:
+For single-user chat and latency-sensitive serving.
 
 ```bash
+export CUDA_VISIBLE_DEVICES=0,1,2,3
+
 python -m sglang.launch_server \
-  --model-path stepfun-ai/Step-5-Preview --served-model-name step5 \
-  --tp 4 --ep-size 1 \
-  --mem-fraction-static 0.9 --swa-full-tokens-ratio 0.1 \
-  --attention-backend fa4 --page-size 64 --disable-prefill-cuda-graph \
-  --reasoning-parser step5 --tool-call-parser step5 \
+  --trust-remote-code \
+  --model-path stepfun-ai/Step-5-Preview \
+  --served-model-name step5 \
+  --tp 4 \
+  --ep-size 1 \
   --moe-runner-backend flashinfer_trtllm \
-  --speculative-algorithm EAGLE --speculative-num-steps 3 \
-  --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 \
-  --enable-multi-layer-eagle --max-running-requests 64 \
+  --attention-backend fa4 \
+  --page-size 64 \
+  --kv-cache-dtype bfloat16 \
+  --mem-fraction-static 0.9 \
+  --swa-full-tokens-ratio 0.03 \
   --chunked-prefill-size 16384 \
-  --flashinfer-allreduce-fusion-backend trtllm
+  --max-running-requests 64 \
+  --cuda-graph-backend-prefill breakable \
+  --flashinfer-allreduce-fusion-backend trtllm \
+  --speculative-algorithm EAGLE \
+  --speculative-num-steps 3 \
+  --speculative-eagle-topk 1 \
+  --speculative-num-draft-tokens 4 \
+  --enable-multi-layer-eagle \
+  --reasoning-parser step3p5 \
+  --tool-call-parser step3p5 \
+  --host 0.0.0.0 \
+  --port 30000
+```
+
+</details>
+
+<details>
+<summary><strong>B300: Balanced (4 GPUs, TP4/EP1)</strong></summary>
+
+#### B300: Balanced (4 GPUs, TP4/EP1)
+
+For serving several users concurrently with MegaMoE.
+
+```bash
+export CUDA_VISIBLE_DEVICES=0,1,2,3
+
+python -m sglang.launch_server \
+  --trust-remote-code \
+  --model-path stepfun-ai/Step-5-Preview \
+  --served-model-name step5 \
+  --tp 4 \
+  --ep-size 1 \
+  --moe-a2a-backend megamoe \
+  --attention-backend fa4 \
+  --page-size 64 \
+  --kv-cache-dtype bfloat16 \
+  --mem-fraction-static 0.9 \
+  --swa-full-tokens-ratio 0.03 \
+  --chunked-prefill-size 16384 \
+  --max-running-requests 64 \
+  --cuda-graph-backend-prefill breakable \
+  --flashinfer-allreduce-fusion-backend trtllm \
+  --speculative-algorithm EAGLE \
+  --speculative-num-steps 3 \
+  --speculative-eagle-topk 1 \
+  --speculative-num-draft-tokens 4 \
+  --enable-multi-layer-eagle \
+  --reasoning-parser step3p5 \
+  --tool-call-parser step3p5 \
+  --host 0.0.0.0 \
+  --port 30000
+```
+
+</details>
+
+<details>
+<summary><strong>B300: High throughput (8 GPUs, TP8/Attention DP2/EP8)</strong></summary>
+
+#### B300: High throughput (8 GPUs, TP8/Attention DP2/EP8)
+
+For batch generation and high-concurrency serving.
+
+```bash
+export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
+
+python -m sglang.launch_server \
+  --trust-remote-code \
+  --model-path stepfun-ai/Step-5-Preview \
+  --served-model-name step5 \
+  --tp 8 \
+  --attn-dp-size 2 \
+  --ep-size 8 \
+  --moe-runner-backend flashinfer_trtllm \
+  --attention-backend fa4 \
+  --page-size 64 \
+  --kv-cache-dtype bfloat16 \
+  --mem-fraction-static 0.9 \
+  --swa-full-tokens-ratio 0.03 \
+  --chunked-prefill-size 16384 \
+  --max-running-requests 64 \
+  --cuda-graph-backend-prefill disabled \
+  --flashinfer-allreduce-fusion-backend trtllm \
+  --speculative-algorithm EAGLE \
+  --speculative-num-steps 3 \
+  --speculative-eagle-topk 1 \
+  --speculative-num-draft-tokens 4 \
+  --enable-multi-layer-eagle \
+  --reasoning-parser step3p5 \
+  --tool-call-parser step3p5 \
+  --host 0.0.0.0 \
+  --port 30000
+```
+
+</details>
+
+<details>
+<summary><strong>H200: Balanced (8 GPUs, TP8/EP8)</strong></summary>
+
+#### H200: Balanced (8 GPUs, TP8/EP8)
+
+For single-node H200 serving with Triton MoE and FA3 attention.
+
+```bash
+export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
+
+python -m sglang.launch_server \
+  --trust-remote-code \
+  --model-path stepfun-ai/Step-5-Preview \
+  --served-model-name step5 \
+  --tp 8 \
+  --ep-size 8 \
+  --moe-runner-backend triton \
+  --attention-backend fa3 \
+  --page-size 64 \
+  --kv-cache-dtype bfloat16 \
+  --mem-fraction-static 0.9 \
+  --swa-full-tokens-ratio 0.03 \
+  --chunked-prefill-size 16384 \
+  --max-running-requests 64 \
+  --cuda-graph-backend-prefill breakable \
+  --flashinfer-allreduce-fusion-backend trtllm \
+  --speculative-algorithm EAGLE \
+  --speculative-num-steps 3 \
+  --speculative-eagle-topk 1 \
+  --speculative-num-draft-tokens 4 \
+  --enable-multi-layer-eagle \
+  --reasoning-parser step3p5 \
+  --tool-call-parser step3p5 \
+  --host 0.0.0.0 \
+  --port 30000
 ```
 
 </details>
 
 ### Send a Request
 
-Once the server is ready, set `BASE_URL` for your deployment:
+Once the server is ready, use the base URL for your deployment:
 
 | Serving configuration | Base URL on the server host |
 | --- | --- |
-| vLLM FP8 | `http://localhost:8000` |
+| vLLM FP8 | `http://localhost:18000` |
 | vLLM BF16 | `http://localhost:18000` (node 0) |
 | SGLang FP8 | `http://localhost:30000` |
 
 The example below targets vLLM FP8:
 
 ```bash
-export BASE_URL="http://localhost:8000"
-curl "$BASE_URL/v1/chat/completions" \
-  -H "Content-Type: application/json" \
-  -d '{
-    "model": "step5",
-    "messages": [
-      {"role": "user", "content": "Please introduce tensor parallelism."}
-    ]
-  }'
+curl http://127.0.0.1:18000/v1/chat/completions \
+    --header 'Content-Type: application/json' \
+    --data '{
+      "model": "step5",
+      "messages": [{"role": "user", "content": "Hello!"}],
+      "max_tokens": 4096,
+      "stream": true
+    }'
 ```
 
 ## Resources
@@ -355,4 +628,10 @@ curl "$BASE_URL/v1/chat/completions" \
 - [Step 5 Preview Model Page](https://www.stepfun.com/step-5-preview) — Model overview, evaluation context, and long-horizon case studies.
 - [Documentation](https://platform.stepfun.ai/docs/en/guides/models/step-5-preview) — Official documentation and developer guides.
 - [API Platform](https://platform.stepfun.ai/) — Model access and platform services.
-- [StepFun Website](https://www.stepfun.com/) — Company information and model announcements.
+- [StepFun Website](https://www.stepfun.com/) — Company information and model announcements.
+
+## License
+
+This model, including its weights, parameters, configuration, tokenizer, model-specific code, and documentation, is licensed under the StepFun Model License 1.0. The full text of the License is available in the [LICENSE](LICENSE) file in this repository. The StepFun Model License 1.0 is an Apache-2.0-Style license, with additional attribution and commercial-license conditions specific to model use, modification and distribution.
+
+Where accompanying materials in this repository, including sample code, datasets, third-party components and other documentation, are separately marked as subject to a different license, such materials are licensed under that license.