Update Readme.md

#3
by Bin2bin - opened
Files changed (1) hide show
  1. README.md +41 -0
README.md CHANGED
@@ -155,6 +155,47 @@ If you are in mainland China, we strongly recommend accessing our models via
155
 
156
  ---
157
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
158
  ## 🌐 License
159
 
160
  This project is licensed under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).
 
155
 
156
  ---
157
 
158
+ ## Deployment
159
+
160
+ ### SGLang
161
+
162
+ You can serve LLaDA2.2-mini through an OpenAI-compatible API using SGLang.
163
+
164
+ Install SGLang from the latest `main` branch:
165
+
166
+ ```bash
167
+ git clone https://github.com/sgl-project/sglang.git
168
+ cd sglang
169
+ pip install -e "python"
170
+ ```
171
+
172
+ For environment requirements, see the
173
+ [SGLang installation guide](https://docs.sglang.io/docs/get-started/install).
174
+
175
+ Launch the server with the following command. This example uses eight NVIDIA GPUs:
176
+
177
+ ```bash
178
+ python3 -m sglang.launch_server \
179
+ --model-path inclusionAI/LLaDA2.2-mini \
180
+ --trust-remote-code \
181
+ --dllm-algorithm JointThresholdInDel \
182
+ --moe-runner-backend triton \
183
+ --tp-size 8 \
184
+ --context-length 131072 \
185
+ --max-running-requests 8 \
186
+ --host 0.0.0.0 \
187
+ --port 30000
188
+ ```
189
+
190
+ Adjust `--tp-size` and `--max-running-requests` according to your hardware
191
+ and available GPU memory.
192
+
193
+ `JointThresholdInDel` currently uses greedy decoding, with default settings
194
+ of `block_size=32`, `threshold=0.5`, `edit_threshold=0.0`, and
195
+ `max_post_edit_steps=16`.
196
+
197
+ ---
198
+
199
  ## 🌐 License
200
 
201
  This project is licensed under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).