|
Download code/SETUP.md from FlyCockpit/GLM-5.2-NVFP4-AQLM-Abliterated-Vision: direct link, hf CLI and curl.
- Browser
- Download file 966 Bytes
-
https://huggingface.co/FlyCockpit/GLM-5.2-NVFP4-AQLM-Abliterated-Vision/resolve/main/code/SETUP.md
- Command line
-
hf download hf://FlyCockpit/GLM-5.2-NVFP4-AQLM-Abliterated-Vision/code/SETUP.md
-
curl -L -o SETUP.md https://huggingface.co/FlyCockpit/GLM-5.2-NVFP4-AQLM-Abliterated-Vision/resolve/main/code/SETUP.md
966 Bytes
New-box setup
The vLLM changes now live in the fork github.com/jarrelscy/vllm-glm52-sm120
(branch glm52-sm120) — everything is committed, there are no patches to apply.
bash code/run_rtx6000.sh [1m|500k|250k]— does everything (clones the fork, builds the venv withVLLM_USE_PRECOMPILED=1, installs flashinfer git-main for the SM120 sparse-MLA path, sets up the JIT toolchain, fetches weights, serves).- Or use the fork's Docker image (RTX PRO 6000):
docker build -f Dockerfile.glm52-sm120 -t glm52-sm120 .thendocker run --gpus all --ipc=host -p 8001:8001 -v <weights>:/models/1m -e PARALLEL=tp4-1m-mtp glm52-sm120(see the Dockerfile header for allPARALLELmodes: pp4-1m, tp4-1m-mtp, tp2pp2, …).
- Or use the fork's Docker image (RTX PRO 6000):
- Verify with
code/tools/verify_sm120.pyagainst the golden bundle (upload it separately or re-capture viacode/tools/capture_golden.py). - Full runbook:
code/PLAN.md; history:code/glm52-RUNLOG.md.