Add deployment guide and vLLM 0.33.0 patch; sync tokenizer and processor code
#1
by HuayuSha - opened
This PR adds deployment docs for Sage-S2.1-Flash and does not change the weights, configs or chat template.
deployment_guide.md(new): basic serving commands for LMDeploy, vLLM and SGLang.vllm_patch/(new): a one-file patch for vLLM 0.33.0 (vllm/renderers/hf.py). Without it, when tool results come back out of call order, vLLM pairs their images with the wrong results. Seevllm_patch/README.md(English) andvllm_patch/README_zh.md(Chinese).tokenization_glm5.py,processing_glm5.py: comments and license header only (model name Sage-S2.1-Flash). The code is unchanged.
Checks:
- Tokenizer and processor regression suite (426 tests) with this repository's chat template: all pass.
- vLLM patch, CPU check on vLLM 0.30.0 to 0.33.0 and nightly: tool-result images paired correctly in all 7 cases.
- GPU, 4ΓH200, these FP8 weights with this repository's files, 20 chat requests (text, control-token spellings in content, tool calls, images, images in tool results), prompt ids checked against the expected encoding:
- vLLM 0.33.0 with the patch,
--trust-remote-code: 19/20 equal, including tool results returned out of call order. The other request uses the aggregated{"id", "output"}tool-result format, which vLLM's request validation rejects (400). - SGLang 0.5.23,
--trust-remote-code --chat-template chat_template.jinja: text requests equal the expected encoding; image requests are encoded like the stock tokenizer. The two 400s are SGLang limitations (aggregated tool results; a literal<|image|>in an image request, sgl-project/sglang#50362).
- vLLM 0.33.0 with the patch,
haijunlv changed pull request status to merged