MiMo-V2.6-Flash-REAP50-GGUF

These are GGUF conversions of MiMo-V2.6-Flash-REAP50, which is XiaomiMiMo/MiMo-V2.6-Flash-RL with 128 of its 256 routed experts kept in each MoE layer. They are for llama.cpp users with large unified-memory or multi-GPU machines. The main file, MXFP4_MOE, is 86.06 GiB; its experts are a direct repack of the source MXFP4 blocks, not a re-quantization.

Status: never loaded, never served, never benchmarked. We have not run these files, or the checkpoint they come from, for generation. There is no serve log, no quality evaluation, no needle-in-a-haystack test and no measured speed. Any "~5 tok/s" figure associated with this release was memory-bandwidth arithmetic.

No video. The GGUF route cannot take MiMo's native video input: llama.cpp's multimodal library (libmtmd) has no video modality in its core, and the mmproj here declares only a vision (image) encoder and an audio encoder. Recent llama.cpp builds add an optional, model-agnostic helper that samples frames with ffmpeg and feeds them in as images. It has not been tested with this model. For video, use the safetensors checkpoint with a runtime that implements MiMo's video path (also untested).

Model overview

Source checkpoint	patrickbdevaney/MiMo-V2.6-Flash-REAP50
Original model	XiaomiMiMo/MiMo-V2.6-Flash-RL (MIT)
GGUF architecture	mimo2 (general.architecture), written by llama.cpp's convert_hf_to_gguf.py
Parameters	about 157B in the language model after pruning, about 15B active per token. Computed from the source safetensors headers; see the source card
Layers	48 (mimo2.block_count): layer 0 dense, layers 1–47 MoE. 9 full-attention layers and 39 sliding-window layers (window 128)
Experts	128 per MoE layer (mimo2.expert_count), top-8 (expert_used_count), sigmoid gating, no shared expert
Context length	1,048,576 (mimo2.context_length). Not tested; calibration never exceeded 4,095 tokens
Licence	MIT

Files

All sizes are from the Hub file metadata. Tensor types are read from each file's GGUF header.

File	Size	Contents
MiMo-V2.6-Flash-REAP50-MXFP4_MOE.gguf	92,403,924,672 B = 86.06 GiB	Trunk without MTP. 141 expert tensors (47 layers × gate/up/down) in MXFP4, repacked from the source blocks. Attention, layer-0 FFN, token embedding and output head in BF16 (dequantized from the source FP8 / BF16). Norms, routers and attention sinks in F32. file_type 38
MiMo-V2.6-Flash-REAP50-Q2_K.gguf	66,187,787,968 B = 61.64 GiB	Trunk without MTP, mixed precision despite the name: ffn_gate_exps / ffn_up_exps in Q2_K, ffn_down_exps kept in MXFP4, attention, layer-0 FFN and token embedding in Q8_0, output head in Q6_K. llama-quantize reported 3.36 BPW. No importance matrix was used. file_type 38
mmproj-MiMo-V2.6-Flash-REAP50-BF16.gguf	2,748,509,920 B = 2.56 GiB	Multimodal projector (clip): vision encoder (28 blocks, projector mimovl) and audio encoder (24 blocks, projector mimo_audio, 20-codebook RVQ), BF16 / F32
mmproj-MiMo-V2.6-Flash-REAP50-Q8_0.gguf	1,564,438,240 B = 1.46 GiB	The same projector in Q8_0 / F32
mtp-MiMo-V2.6-Flash-REAP50-BF16.gguf	4,481,535,904 B = 4.17 GiB	The base model's 3 MTP layers (mimo2.nextn_predict_layers 3, dense FFN) with their own token embedding and output head, BF16 / F32
mtp-MiMo-V2.6-Flash-REAP50-Q8_0.gguf	2,383,728,544 B = 2.22 GiB	The same MTP file in Q8_0 / F32

No other quantizations exist in this repo. A Q3_K_S file, an .imatrix file and a Q4_K_M / IQ* ladder were never published.

Which version should I use

	When
MXFP4_MOE	The closest file to the source checkpoint: the experts are bit-for-bit the source FP4 values. Needs about 86 GiB for weights before KV cache
Q2_K (mixed)	Saves 24.4 GiB, but re-quantizes the gate/up experts from 4-bit to 2-bit. Its quality cost has not been measured
safetensors repo	Needed for video input, or for runtimes other than llama.cpp

Nothing has been measured, so none of these can be recommended on quality or speed.

Quickstart (untested)

llama.cpp has a mimo2 model implementation, and the converter that wrote these files is llama.cpp's own. We have not loaded these files with any build, so the commands below are illustrative only:

# text + image/audio, OpenAI-compatible server (untested)
llama-server \
  -m MiMo-V2.6-Flash-REAP50-MXFP4_MOE.gguf \
  --mmproj mmproj-MiMo-V2.6-Flash-REAP50-BF16.gguf \
  -ngl 99 --port 8080

We have not checked how a llama.cpp build should load the separate mtp-*.gguf files for speculative decoding.

Evaluation

Not yet evaluated. There is no measurement of any kind, on these files or on the source checkpoint: no perplexity or KL against the unpruned model, no generative or task benchmark, no multimodal test, no long-context test, no speed or memory figure. Equivalence between MXFP4_MOE and the safetensors checkpoint was not checked end to end either. The expert bytes are a direct repack, but the BF16 attention path and the llama.cpp kernels are untested.

The only numbers that exist are selection-time diagnostics on the calibration corpus (worst-domain retained routed mass 0.9935 for audio, mean 0.9964). They are described on the source card. They are a proxy, not a quality result.

Hardware & memory

• Built on: one NVIDIA Jetson AGX Thor. The conversion ran on 2026-09-25 with llama.cpp's convert_hf_to_gguf.py (--no-mtp --outtype bf16). The Q2_K file was made with llama-quantize build 8678 (19a4efa22) using per-tensor type overrides.
• Run on: nothing.
• Fit (estimated, not tested): the weights in each file take about their file size. An FP8 KV cache costs about 11.5 KB per token on the 9 full-attention layers, about 11.25 GiB at 1,048,576 tokens. The 39 SWA layers keep a 128-token window. llama.cpp's default F16 KV cache would double that. Leave room for runtime workspace and the mmproj.

How it was made

1. Pruning (source checkpoint): expert saliency was collected over 49.0M tokens of calibration data in nine domains. Six domains are text (agentic, general, code, math, science, finance); three use real image, audio and video run through the model's towers. Every source dataset was permissively licensed. Sequences were at most 4,095 tokens. The prune set was solved with the HOPE objective (arXiv 2609.18916) on the REAP saliency criterion (arXiv 2510.13999), at a uniform 50% per layer, with the top 8% of each domain protected and candidates ranked by their worst domain. A router refit did not beat slicing the teacher's routers in any of the 47 layers, so all routers are the teacher's. No weights were re-quantized and none were fine-tuned.
2. Conversion: convert_hf_to_gguf.py repacked each layer's 128 kept experts from the source MXFP4 layout into GGML MXFP4 blocks. The conversion log records "repacked 128 experts to MXFP4" for all 141 expert tensors. It dequantized the other weight matrices to BF16 and kept norms and router tensors in F32. The 3 MTP layers were exported to a separate file, and the vision and audio towers to mmproj files.
3. Q2_K file: llama-quantize --allow-requantize from the MXFP4_MOE file, with type overrides ffn_gate_exps / ffn_up_exps → q2_k, ffn_down_exps → mxfp4, attn_*, layer-0 FFN and token_embd → q8_0, output → q6_k.

Pipeline code: patrickbdevaney/xiaomi-2.6-flash-REAP.

Limitations

• Unevaluated, as above. Pruning half the experts is lossy and cannot be undone.
• No video input through llama.cpp (see the note at the top). Image and audio rely on llama.cpp's mimovl and mimo_audio projector support and are untested with these files.
• Long context is unverified beyond the 4,095-token calibration ceiling.
• The MTP layers are the base model's, unchanged. Their acceptance rate on the pruned trunk has not been measured. The base repo's separate DFlash drafter is not included.
• The Q2_K name follows the file's dominant expert type. It is not a stock llama.cpp Q2_K preset: file_type is 38 (MXFP4_MOE) and the down-projection experts stay MXFP4.

Changelog / corrections

• 2026-09-25: MXFP4_MOE, Q2_K, mmproj and mtp files published.
• 2026-10-12: card corrected: base model is XiaomiMiMo/MiMo-V2.6-Flash-RL; "Official" removed (not affiliated with Xiaomi); evaluation status stated; HOPE selection described accurately; unmeasured claims dropped.
• 2026-10-24: correction: never served or benchmarked; "~5 tok/s" was bandwidth arithmetic; the 1M-context fit is sizing arithmetic; retention figures are a selection-time proxy; these files have never been loaded, so all commands are untested.
• 2026-11-06: card rewritten and verified against repo files and build artifacts; a 2026-10-26 automated rewrite contained errors.

License & attribution

MIT, inherited from the base model. The base model, its architecture, its multimodal towers and its MTP layers are the work of the Xiaomi MiMo team: XiaomiMiMo/MiMo-V2.6-Flash-RL. This is an independent community derivative, not affiliated with or endorsed by Xiaomi. GGUF conversion and quantization used llama.cpp (MIT).

Citation

@misc{devaney2026mimov26flashreap50gguf,
  title        = {MiMo-V2.6-Flash-REAP50-GGUF: GGUF conversions of a 50\% routed-expert-pruned MiMo-V2.6-Flash-RL},
  author       = {Patrick Devaney},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF}}
}