Buckets:
The model folder format
A model folder is a Hugging Face folder with safetensors shards, config.json, the tokenizer files and the MLX
loader. The engine reads it with nslm/model_st.h.
Tensor encodings
Each tensor NAME.weight is stored in one of these encodings:
| Encoding | Entries |
|---|---|
| BF16 | NAME.weight |
| Affine Q8, Q4 (group 64, MLX layout; the reader also takes Q4 group 32) | NAME.weight (U32), NAME.scales, NAME.biases |
Seeds, P = 4 (SEED4P4, 4.5 bits per weight) |
NAME.seeds (U16), NAME.coefs (U16), NAME.codes (U8), NAME.exp_bias (I32) |
Seeds, P = 3 (SEED4, 4.0 bits per weight) |
NAME.seeds (U16), NAME.nibbles (U16: 3 int4 coefficients and the exponent code in bits 0 to 3), NAME.exp_bias (I32) |
Read SeedLM for how the engine makes weights from seeds.
Trellis expert banks
A folder can also hold trellis-coded experts as bank files, L{layer}_{gate_proj,up_proj,down_proj}.blkt
(nslm-moe --codec tq). The engine and the C reference add them with ns_add_banks. Each bank becomes the tensor
language_model.model.layers.L.mlp.switch_mlp.PROJ.weight, in place of the folder's tensor of that name if it has
one. The safetensors shards must stay whole, because the reader opens every shard that the index names.
A bank of layer n can be the MTP head's (mtp.layers.0.mlp.switch_mlp.PROJ.weight). That is so when the folder has
the routers (mlp.gate.weight) of layer n - 1 and of the head, and no router for layer n. For Qwen3.8-Flash-Next, it
is layer 48.
nslm-qwen4-pack --tq writes a folder without the routed experts, so its banks are the only experts.
| Part | Contents |
|---|---|
Header (56 bytes, MoeBlkHeader in nslm/moe.h) |
magic NSLMBLKT, rows, cols, experts E, L (n_seeds), K (n_exp), T (refit), n0, the encode's seconds |
rel[E], wrel[E] (F32) |
Each expert's relative error: plain and imatrix-weighted, square roots |
| Words (U32) | [E][rows][nslm_tq_row_words(cols, T, L, K)]: each row's blocks of T weights as one bit stream |
| Scales (U16, BF16) | [E][rows][ceil(cols / T)]: one scale for each block |
Read Trellis-coded experts for the code and the bit stream.
Shard layout
The writer pads each shard header to a 16 KiB page. Entries whose size is a multiple of the page come first. The engine maps those entries into GPU buffers without a copy.
MLA entries
An MLA model has two more keys in config.json:
| Key | Value |
|---|---|
mla_ranks |
The latent rank of each layer. A multiple of 32, at most 1024. 0 keeps the layer's GQA attention (a hybrid model) |
mla_rope_dim |
128 |
The engine reads the MLA tensors from model-mla-delta.safetensors. For each layer, the file has the tensors
self_attn.mla.{kv_a_x, kv_a_v, k_rope_proj, q_rope_mix, q_lat, v_up}. These tensors are BF16, or they use the
encoding that nslm-mova-pack --mla sets. The engine does not use k_proj. The dense layers fold v_proj into
kv_a_x.
MLX loader keys
| Key | Use |
|---|---|
model_file |
The loader file: nanoseedlm_k2.py, nanoseedlm_qwen4.py or k2_mla.py. The NanoSeedLM loaders need nanoseedlm_seeds.py beside them |
mla_max_context |
The longest context that you plan to use |
mla_kv |
The MLA cache format. Read The KV cache |
Xet Storage Details
- Size:
- 3.38 kB
- Xet hash:
- 373d3d6299e59c0cc9a6a9355031efb7ad1167615dcd1c06158fc83d5a92f540
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.