MiMo-V2.6-Pro-EXL3

Experimental sequential quantization checkpoint. Not a ready-to-serve model.

Completed full-corpus joint-PV layers: [1, 2, 3, 4]. Each completed layer's run traversed all 18,006,461 training tokens. Validation alone selects the retained checkpoint, which may precede the end of the corpus pass or retain the initial fit when it is better. The selected step and corpus coverage are recorded in reports.

Cold experts use mixed-rate EXL3 trellis codes at <=2 actual packed bits per weight. Initial trellis fitting uses a calibration subset; subsequent full-corpus PV jointly optimizes all input/output scales against the combined routed layer output while holding trellis codes fixed. No neurons are removed. The exact original 1,325 hot experts (5.00075% globally) retain their NVFP4 weights and assignment; this is not a separate 5% allocation per layer.

Later layers use inputs propagated through earlier accepted EXL3 layers. Validation and audit each use 16,384 held-out tokens. reports/layer_errors.csv records EXL3 validation and audit reconstruction errors. ARVQ comparisons are disabled; previously published comparison reports are historical. Relative L2 errors are layer reconstruction metrics, not whole-model accuracy, perplexity, or KL.

Cold weights use standard EXL3 codebooks, not the experimental FP4-component codebooks. Evaluation decodes weights to BF16. These custom mixed-rate per-expert artifacts still require a complete serving loader; this is not an off-the-shelf EXL3 model directory. Only listed layers are complete. Backbone tensors are not included until separately published.

Use decode.py with the official EXL3 1.5.1 runtime and a compatible CUDA/PyTorch wheel to reconstruct an expert:

python decode.py layers/layer_00001/expert_00000/selected.bin expert_00000.pt

Source: XiaomiMiMo/MiMo-V2.6-Pro-RL, revision 73875d00b30a89ef8cc353a0b60b0e9f9561952d. See sequential_manifest.json, per-layer manifests, and receipts for status, provenance, and SHA-256 hashes. The earlier FP4-named research repository is separate.