neural-nova
/

GLM-5.2-Instruct-Optimized

inference-optimization

Mixture of Experts

Model card Files Files and versions

NeuralNovaAI commited on 28 days ago

Commit

f6589a4

·

verified ·

1 Parent(s): 187ab00

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -95,7 +95,7 @@ export VLLM_WORKER_MULTIPROC_METHOD=spawn
 vllm serve zai-org/GLM-5.2 \  --host 0.0.0.0 --port 8000 \  --tensor-parallel-size 8 \  --pipeline-parallel-size 1 \
 --distributed-executor-backend mp \  --max-model-len 65536 \  --gpu-memory-utilization 0.90 \  --kv-cache-dtype fp8_e4m3 \  --max-num-seqs 256 \  --max-num-batched-tokens 16384 \  --long-prefill-token-threshold 4096 \
 --enable-prefix-caching \  --async-scheduling \  --compilation-config ‘{“cudagraph_mode”: “FULL_AND_PIECEWISE”, “max_cudagraph_capture_size”: 256}’ \  --trust-remote-code
-```bash
 ### Validation

 vllm serve zai-org/GLM-5.2 \  --host 0.0.0.0 --port 8000 \  --tensor-parallel-size 8 \  --pipeline-parallel-size 1 \
 --distributed-executor-backend mp \  --max-model-len 65536 \  --gpu-memory-utilization 0.90 \  --kv-cache-dtype fp8_e4m3 \  --max-num-seqs 256 \  --max-num-batched-tokens 16384 \  --long-prefill-token-threshold 4096 \
 --enable-prefix-caching \  --async-scheduling \  --compilation-config ‘{“cudagraph_mode”: “FULL_AND_PIECEWISE”, “max_cudagraph_capture_size”: 256}’ \  --trust-remote-code
+```
 ### Validation