beyoru commited on
Commit
e05fa1f
·
verified ·
1 Parent(s): 2f308ad

Add README.md: sample serving config

Browse files
Files changed (1) hide show
  1. README.md +14 -0
README.md CHANGED
@@ -71,6 +71,20 @@ no advantage there (the LM weights are identical) and costs 0.9 GB extra VRAM.
71
 
72
  ---
73
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
74
  ## Serving
75
 
76
  ### vLLM (recommended, verified)
 
71
 
72
  ---
73
 
74
+ ## Ready-made configs
75
+
76
+ | file | what | status |
77
+ |---|---|---|
78
+ | [`docker-compose.vllm.yaml`](./docker-compose.vllm.yaml) | vLLM, 1 GPU, vision enabled | **verified** |
79
+ | [`docker-compose.sglang-flashqla.yaml`](./docker-compose.sglang-flashqla.yaml) | SGLang + FlashQLA GDN prefill | untested |
80
+
81
+ FlashQLA applies here because 30 of the 40 layers are `linear_attention` (Gated DeltaNet) — exactly
82
+ the layers its TileLang kernel accelerates during prefill. It is opt-in via a single flag
83
+ (`--linear-attn-prefill-backend flashqla`); the default path is byte-identical to stock SGLang.
84
+
85
+ **Windowed-MTP does not apply to this model.** `mtp_num_hidden_layers = 0`, so the checkpoint carries
86
+ no MTP/NEXTN head to self-speculate with. That technique needs `Qwen/Qwen3.6-35B-A3B` (mtp = 1).
87
+
88
  ## Serving
89
 
90
  ### vLLM (recommended, verified)