sysssyssy baishihao commited on
Commit
b226a97
·
1 Parent(s): 66810cf

docs: add Docker deployment examples for LightLLM, vLLM and SGLang (#1)

Browse files

- docs: add Docker deployment examples for LightLLM, vLLM and SGLang (e333506667744c4856916d2a38dfccdb75e2bb73)


Co-authored-by: baishihao <baishihao@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +43 -5
README.md CHANGED
@@ -42,12 +42,50 @@ We believe code can become a common language between ideas and finished work. Ou
42
 
43
  ## Deployment
44
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
  ```bash
46
- python -m lightllm.server.api_server \
47
- --model_dir /path/to/SenseNova-6.8-Code2Video \
48
- --tp 4 \
49
- --max_req_total_len 524288 \
50
- --host 0.0.0.0 --port 8000
 
 
 
 
 
 
51
  ```
52
 
53
  ## Recommended harness: Claude Code
 
42
 
43
  ## Deployment
44
 
45
+
46
+ ### [LightLLM](https://github.com/ModelTC/LightLLM)
47
+
48
+ ```bash
49
+ docker run --rm --gpus all --ipc=host -p 8000:8000 \
50
+ -v /path/to/SenseNova-6.8-Code2Video:/model \
51
+ ghcr.io/modeltc/lightllm:main \
52
+ python -m lightllm.server.api_server \
53
+ --model_dir /model \
54
+ --tp 4 \
55
+ --max_req_total_len 524288 \
56
+ --host 0.0.0.0 --port 8000
57
+ ```
58
+
59
+ ### [vLLM](https://github.com/vllm-project/vllm)
60
+
61
+ ```bash
62
+ docker run --rm --gpus all --ipc=host -p 8000:8000 \
63
+ -e VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \
64
+ -v /path/to/SenseNova-6.8-Code2Video:/model \
65
+ vllm/vllm-openai:latest \
66
+ --model /model \
67
+ --tensor-parallel-size 4 \
68
+ --max-model-len 524288 \
69
+ --reasoning-parser qwen3 \
70
+ --enable-auto-tool-choice \
71
+ --tool-call-parser qwen3_coder \
72
+ --host 0.0.0.0 --port 8000
73
+ ```
74
+
75
+ ### [SGLang](https://github.com/sgl-project/sglang)
76
+
77
  ```bash
78
+ docker run --rm --gpus all --ipc=host -p 8000:8000 \
79
+ -e SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 \
80
+ -v /path/to/SenseNova-6.8-Code2Video:/model \
81
+ lmsysorg/sglang:latest \
82
+ python -m sglang.launch_server \
83
+ --model-path /model \
84
+ --tp-size 4 \
85
+ --context-length 524288 \
86
+ --reasoning-parser qwen3 \
87
+ --tool-call-parser qwen3_coder \
88
+ --host 0.0.0.0 --port 8000
89
  ```
90
 
91
  ## Recommended harness: Claude Code