NSLM_VRAM_RESERVE_MB |
512 |
VRAM to keep free |
NSLM_EXPERT_MIN_MB |
6144 |
VRAM to keep for experts when the KV cache is large |
NSLM_KV_VRAM_MB, NSLM_EXPERT_VRAM_MB |
|
Fixed VRAM shares for the KV cache and the experts |
NSLM_CACHE_STATS=1 |
off |
Write the expert cache hits at exit |
NSLM_NO_GRAPH=1 |
off |
Do not use CUDA graphs for decode (K2-Horizon-MoVA and Qwen3.8-Flash-Next) |
NSLM_NO_PREDICT=1 |
off |
Do not predict the experts of the next layer during decode (K2-Horizon-MoVA) |
Q4E_PROFILE=1 |
off |
Qwen3.8-Flash-Next: write the GPU time of each forward stage at exit |
Q4E_ROUTES=FILE |
off |
Qwen3.8-Flash-Next: write the routed experts of each forward, for tools/qwen4_cache_sim.py |
Q4E_NO_PREFETCH=1 |
off |
Qwen3.8-Flash-Next: do not copy the next layer's experts while a prompt layer computes |
Q4E_PLE_THREADS=N |
CPU cores, at most 16 |
Qwen3.8-Flash-Next: host threads for the PLE rows of a prompt |
NSLM_SEED_GEMM_F32=1 |
off |
Keep seed weights in f32 in the prompt GEMM. This changed neither KLD nor NLL, and it costs about 30% of prompt speed at 4K |
NSLM_PREFILL_RANGE |
8192, or 32768 when the KV cache is partly in host memory |
Prompt rows that the engine computes layer by layer at a time. Each range copies the earlier host rows again |
NSLM_MLA_DEC_KEYS |
8192 (128 MiB of VRAM) |
MLA keys that the engine decompresses at a time for prompt rows. 0 attends prompts absorbed, as decode does |
NSLM_MLA_SCALAR=1 |
off |
Use the earlier one-pass latent kernel, for comparison |
