harrypart commited on
Commit
ac12dbb
·
verified ·
1 Parent(s): 5b1270b

model card: sglang only

Browse files
Files changed (1) hide show
  1. README.md +3 -17
README.md CHANGED
@@ -7,7 +7,6 @@ tags:
7
  - vision-language
8
  - glm
9
  - sglang
10
- - vllm
11
  base_model:
12
  - zai-org/GLM-5.2
13
  ---
@@ -17,7 +16,7 @@ base_model:
17
  **GLM-5.2 with sight.** A vision-language model that bolts the MoonViT vision encoder from
18
  [Kimi-K2.6](https://huggingface.co/moonshotai/Kimi-K2.6) onto
19
  [GLM-5.2](https://huggingface.co/zai-org/GLM-5.2) through a trained PatchMerger projector —
20
- shipped as **one self-contained checkpoint** you can hand straight to SGLang or vLLM.
21
 
22
  GLM-5.2 is a strong open reasoning model with no vision input. This checkpoint adds it,
23
  without touching a single GLM weight: the text backbone and the vision tower are both frozen
@@ -45,8 +44,8 @@ At ~757 GB the weights do not fit on four GPUs. For a 4-GPU deployment use
45
 
46
  ## Quickstart
47
 
48
- Both engines need a small out-of-tree plugin, because `Glm5vForConditionalGeneration` is not
49
- yet an upstream architecture. It ships inside this repo, so there is nothing else to clone:
50
 
51
  ```bash
52
  hf download baseten/GLM-5.2-Vision-FP8 --include 'plugins/*' --local-dir ./glm5v
@@ -77,19 +76,6 @@ python -m sglang.launch_server \
77
  --port 30000
78
  ```
79
 
80
- Measured KV budget at these settings: ~1.16M tokens.
81
-
82
- ### vLLM
83
-
84
- The pip install registers a `vllm.general_plugins` entry point, so no flags and no env vars:
85
-
86
- ```bash
87
- vllm serve baseten/GLM-5.2-Vision-FP8 --trust-remote-code --tensor-parallel-size 8
88
- ```
89
-
90
- > Note: the vLLM path is wired up but has had less testing than SGLang here. If you hit
91
- > trouble, SGLang is the better-trodden route.
92
-
93
  ### Query it
94
 
95
  Standard OpenAI multimodal messages — the image is delivered as `image_url`:
 
7
  - vision-language
8
  - glm
9
  - sglang
 
10
  base_model:
11
  - zai-org/GLM-5.2
12
  ---
 
16
  **GLM-5.2 with sight.** A vision-language model that bolts the MoonViT vision encoder from
17
  [Kimi-K2.6](https://huggingface.co/moonshotai/Kimi-K2.6) onto
18
  [GLM-5.2](https://huggingface.co/zai-org/GLM-5.2) through a trained PatchMerger projector —
19
+ shipped as **one self-contained checkpoint** you can hand straight to SGLang.
20
 
21
  GLM-5.2 is a strong open reasoning model with no vision input. This checkpoint adds it,
22
  without touching a single GLM weight: the text backbone and the vision tower are both frozen
 
44
 
45
  ## Quickstart
46
 
47
+ SGLang needs a small out-of-tree plugin, because `Glm5vForConditionalGeneration` is not yet an
48
+ upstream architecture. It ships inside this repo, so there is nothing else to clone:
49
 
50
  ```bash
51
  hf download baseten/GLM-5.2-Vision-FP8 --include 'plugins/*' --local-dir ./glm5v
 
76
  --port 30000
77
  ```
78
 
 
 
 
 
 
 
 
 
 
 
 
 
 
79
  ### Query it
80
 
81
  Standard OpenAI multimodal messages — the image is delivered as `image_url`: