Add deployment guide and vLLM 0.33.0 patch; sync tokenizer and processor code

#1
by HuayuSha - opened
Intern Large Models org
β€’
edited 1 day ago Hugging Face CLI

This PR adds deployment docs for Sage-S2.1-Flash and does not change the weights, configs or chat template.

  • deployment_guide.md (new): basic serving commands for LMDeploy, vLLM and SGLang.
  • vllm_patch/ (new): a one-file patch for vLLM 0.33.0 (vllm/renderers/hf.py). Without it, when tool results come back out of call order, vLLM pairs their images with the wrong results. See vllm_patch/README.md (English) and vllm_patch/README_zh.md (Chinese).
  • tokenization_glm5.py, processing_glm5.py: comments and license header only (model name Sage-S2.1-Flash). The code is unchanged.

Checks:

  • Tokenizer and processor regression suite (426 tests) with this repository's chat template: all pass.
  • vLLM patch, CPU check on vLLM 0.30.0 to 0.33.0 and nightly: tool-result images paired correctly in all 7 cases.
  • GPU, 4Γ—H200, these FP8 weights with this repository's files, 20 chat requests (text, control-token spellings in content, tool calls, images, images in tool results), prompt ids checked against the expected encoding:
    • vLLM 0.33.0 with the patch, --trust-remote-code: 19/20 equal, including tool results returned out of call order. The other request uses the aggregated {"id", "output"} tool-result format, which vLLM's request validation rejects (400).
    • SGLang 0.5.23, --trust-remote-code --chat-template chat_template.jinja: text requests equal the expected encoding; image requests are encoded like the stock tokenizer. The two 400s are SGLang limitations (aggregated tool results; a literal <|image|> in an image request, sgl-project/sglang#50362).
haijunlv changed pull request status to merged

Sign up or log in to comment