yujiepan commited on
Commit
0a57c72
·
verified ·
1 Parent(s): 6c0013a

Preserve native mixed NVFP4/MXFP8 Inkling 2 without model-specific plugins

Browse files
Files changed (1) hide show
  1. README.md +1 -2
README.md CHANGED
@@ -35,8 +35,7 @@ Use SGLang with Inkling 2 support. No model-specific package is required.
35
 
36
  ```bash
37
  SGLANG_BUILD_RUST_EXTS=none pip install 'git+https://github.com/sgl-project/sglang.git@f726172d760dea737c7f1e797c9f32efe14d1685#subdirectory=python'
38
- hf download tiny-random/inkling-2 --local-dir inkling-2-tiny
39
- sglang serve --model-path ./inkling-2-tiny --tp 2 --ep 2 \
40
  --dtype bfloat16 --attention-backend fa4 --enable-multimodal \
41
  --context-length 2048 --mem-fraction-static 0.2 \
42
  --disable-custom-all-reduce \
 
35
 
36
  ```bash
37
  SGLANG_BUILD_RUST_EXTS=none pip install 'git+https://github.com/sgl-project/sglang.git@f726172d760dea737c7f1e797c9f32efe14d1685#subdirectory=python'
38
+ sglang serve --model-path tiny-random/inkling-2 --tp 2 --ep 2 \
 
39
  --dtype bfloat16 --attention-backend fa4 --enable-multimodal \
40
  --context-length 2048 --mem-fraction-static 0.2 \
41
  --disable-custom-all-reduce \