Buckets:
| # Environment variables | |
| ## Both engines | |
| | Variable | Default | Use | | |
| |---|---|---| | |
| | `NSLM_MODEL_DIR` | | The model folder for the tests (Windows build) | | |
| ## Metal engine | |
| | Variable | Default | Use | | |
| |---|---|---| | |
| | `NSLM_KV_DENSE=1` | off | Commit all KV cache memory at the start. The default pages the cache (macOS 26.4 or later) | | |
| | `NSLM_BATCH_GEMM=N` | off | Run decode steps of N or more slots through the prompt GEMMs. Faster at large batches. Gives up bit-for-bit equality | | |
| ## CUDA engine | |
| | Variable | Default | Use | | |
| |---|---|---| | |
| | `NSLM_VRAM_RESERVE_MB` | 512 | VRAM to keep free | | |
| | `NSLM_EXPERT_MIN_MB` | 6144 | VRAM to keep for experts when the KV cache is large | | |
| | `NSLM_KV_VRAM_MB`, `NSLM_EXPERT_VRAM_MB` | | Fixed VRAM shares for the KV cache and the experts | | |
| | `NSLM_CACHE_STATS=1` | off | Write the expert cache hits at exit | | |
| | `NSLM_NO_GRAPH=1` | off | Do not use CUDA graphs for decode (K2-Horizon-MoVA and Qwen3.8-Flash-Next) | | |
| | `NSLM_NO_PREDICT=1` | off | Do not predict the experts of the next layer during decode (K2-Horizon-MoVA) | | |
| | `Q4E_PROFILE=1` | off | Qwen3.8-Flash-Next: write the GPU time of each forward stage at exit | | |
| | `Q4E_ROUTES=FILE` | off | Qwen3.8-Flash-Next: write the routed experts of each forward, for `tools/qwen4_cache_sim.py` | | |
| | `Q4E_NO_PREFETCH=1` | off | Qwen3.8-Flash-Next: do not copy the next layer's experts while a prompt layer computes | | |
| | `Q4E_PLE_THREADS=N` | CPU cores, at most 16 | Qwen3.8-Flash-Next: host threads for the PLE rows of a prompt | | |
| | `NSLM_SEED_GEMM_F32=1` | off | Keep seed weights in f32 in the prompt GEMM. This changed neither KLD nor NLL, and it costs about 30% of prompt speed at 4K | | |
| | `NSLM_PREFILL_RANGE` | 8192, or 32768 when the KV cache is partly in host memory | Prompt rows that the engine computes layer by layer at a time. Each range copies the earlier host rows again | | |
| | `NSLM_MLA_DEC_KEYS` | 8192 (128 MiB of VRAM) | MLA keys that the engine decompresses at a time for prompt rows. 0 attends prompts absorbed, as decode does | | |
| | `NSLM_MLA_SCALAR=1` | off | Use the earlier one-pass latent kernel, for comparison | | |
| ## MLX loader | |
| | Variable | Default | Use | | |
| |---|---|---| | |
| | `NSLM_MAX_CONTEXT` | | The longest context (tokens) that you plan to use. The loader picks the MLA cache format from it | | |
| | `OMLX_K2_MODEL` | | The oMLX K2-Horizon model file, for `make test-mlx` and the MLX tools | | |
Xet Storage Details
- Size:
- 2.4 kB
- Xet hash:
- 3735db6bdf3046b3b2712dfa01ecbb1ead2183b37f9979671732382b7334f277
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.