Model card rewritten and verified against repo files and build artifacts (2026-11-06)

README.md

@@ -1,146 +1,165 @@
 ---
-license: apache-2.0
-base_model: XiaomiMiMo/MiMo-V2.6-Flash
+license: mit
+base_model: XiaomiMiMo/MiMo-V2.6-Flash-RL
 base_model_relation: quantized
-library_name: safetensors
+library_name: transformers
 pipeline_tag: image-text-to-text
-language:
-  - en
-  - zh
 tags:
-  - mimo
-  - mimo-v2.6-flash
-  - xiaomi
-  - moe
-  - reap
-  - expert-pruning
-  - mxfp4
-  - fp8
-  - multimodal
-  - omni
-  - mtp
+- mimo_v2
+- moe
+- pruned
+- expert-pruning
+- reap
+- hope
+- mxfp4
+- fp8
+- multimodal
+- vision
+- audio
+- video
+- mtp
+- custom_code
 ---
 
-<div align="center">
+# MiMo-V2.6-Flash-REAP50
 
-# MiMo-V2.6-Flash — REAP-50 (MXFP4 + FP8 Master Checkpoint)
+This is [`XiaomiMiMo/MiMo-V2.6-Flash-RL`](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL) with half of its routed experts removed: each of the 47 MoE layers goes from 256 experts to 128. The weights come to **86.1 GiB** in 65 safetensors shards, small enough to fit a single 128 GB unified-memory machine (NVIDIA Jetson AGX Thor). That fit is an estimate, not a test. It is meant for people who want the omni-modal MiMo-V2.6-Flash model on one box of that size.
 
-[![Base Model](https://img.shields.io/badge/Base_Model-XiaomiMiMo%2FMiMo--V2.6--Flash-blue)](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash)
-[![Compression](https://img.shields.io/badge/REAP--50-128%2F256_Experts_Kept_(164B_Total_%2F_11B_Active)-orange)](#pruning-method)
-[![Format](https://img.shields.io/badge/Format-Native_MXFP4_%2B_FP8_(85.8_GiB)-green)](#what-changed-vs-the-base-model)
-[![Hardware](https://img.shields.io/badge/Fits_In-128GB_Unified_Memory-76b900)](#model-summary)
-[![License](https://img.shields.io/badge/License-Apache_2.0-yellow.svg)](https://opensource.org/licenses/Apache-2.0)
+> **Status: never served, never benchmarked.** The checkpoint was published without being run for generation.
+> There is no serve log, no quality evaluation, no needle-in-a-haystack test and no measured throughput on any
+> hardware. Any decode speed you may have seen attached to this release (for example "~5 tok/s on Thor") came from
+> memory-bandwidth arithmetic. Nobody measured it.
 
-**85.8 GiB bit-exact 50% REAP expert-pruned master checkpoint of `XiaomiMiMo/MiMo-V2.6-Flash` (315B → 164B total, 11B active) in native OCP MXFP4 + FP8 + BF16.**
+## Model overview
 
-| Sibling Artifact | Format | Size | Description |
-|---|---|---|---|
-| **[`MiMo-V2.6-Flash-REAP50`](https://huggingface.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50)** *(this repo)* | Native MXFP4 + FP8 + BF16 (`safetensors`) | **85.8 GiB** | Bit-exact 128-expert master checkpoint (with vision/video/audio encoders + 3 MTP layers) |
-| **[`MiMo-V2.6-Flash-REAP50-GGUF`](https://huggingface.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF)** | GGUF (`MXFP4_MOE`, `Q3_K_S`, `Q2_K`, `mmproj`, `mtp`) | **64.8–86.7 GiB** | Single-file GGUFs + multimodal projector + 3-layer MTP draft head |
+| | |
+|---|---|
+| Base model | [`XiaomiMiMo/MiMo-V2.6-Flash-RL`](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL) (MIT) |
+| Architecture | `MiMoV2ForCausalLM` (`model_type: mimo_v2`), custom code included (`modeling_mimo_v2.py`) |
+| Parameters | about 157B in the language model after pruning (about 309B before); about 15B active per token. These are computed from the safetensors headers, not taken from a card (see the note below) |
+| Layers | 48: layer 0 is dense, layers 1–47 are MoE. 9 full-attention layers (0, 5, 11, 17, 23, 29, 35, 41, 47) and 39 sliding-window layers (window 128) |
+| Experts | **128 of 256** routed experts kept per MoE layer, top-8 routing, no shared expert (`sigmoid` scoring, `noaux_tc`, `n_group 1`) |
+| Attention | 64 query heads; 4 KV heads on full-attention layers and 8 on SWA layers; head dim 192 (K) / 128 (V) |
+| Context length | 1,048,576 in `config.json`. Not tested: calibration sequences were at most 4,095 tokens |
+| Vocabulary | 152,576, untied embeddings |
+| MTP | 3 next-token-prediction layers (`num_nextn_predict_layers: 3`) in `model_mtp.safetensors`. They have dense FFNs, so pruning did not touch them |
+| Input modalities | text, image, video and audio (vision tower, audio encoder, and `audio_tokenizer/`). The towers are unchanged. **None of these modalities has been tested on the pruned model** |
+| Size on disk | 92,448,660,520 bytes = **86.10 GiB** of model weights (64 `model_pp0_ep*_shard0.safetensors` + `model_mtp.safetensors`). The whole repository is 87.86 GiB once you add `audio_tokenizer/model.safetensors` (1.74 GiB) and the tokenizer files |
+| Licence | MIT, inherited from the base model |
+
+Parameter note: 6,016 expert tensors (47 layers × 128 experts) × 3 projections × 4096 × 2048 = 151.4B expert parameters, plus 6.0B non-expert language-model parameters (attention, layer-0 FFN, routers, embeddings, LM head). Restoring the 128 dropped experts per layer gives 308.8B, which matches the base model's stated 309B. Active parameters (8 experts per layer plus all non-expert weights) are about 15.4B, the same as the base, because pruning does not change top-k. The vision tower (0.73B), audio encoder (0.26B) and MTP layers (0.99B) are counted separately.
+
+### Precision per component (from the safetensors headers)
+
+| Component | Stored as | Changed by this release? |
+|---|---|---|
+| Routed experts (`mlp.experts.*`) | MXFP4: packed E2M1 nibbles (`U8`) + E8M0 block scales (`U8`), block 32. This is Xiaomi's source format | Dropped or kept whole. Kept experts' bytes are copied unchanged and renumbered 0–127 |
+| Routers (`mlp.gate.weight`, `e_score_correction_bias`) | BF16 / F32 | Sliced to the 128 kept rows. Values unchanged (see *Routers*) |
+| Attention `qkv_proj` | FP8 E4M3 with F32 128×128 block scales | No |
+| Attention `o_proj` | BF16 (excluded from FP8 at source) | No |
+| Layer-0 dense FFN | FP8 E4M3 with block scales | No |
+| Embeddings, LM head, norms | BF16 | No |
+| Vision tower, audio encoder | BF16 | No |
+| MTP layers | FP8 E4M3 (`qkv_proj`, FFN) + BF16 | No |
 
-</div>
+Experts already ship as 4-bit MXFP4 in the source checkpoint, so re-quantizing them buys almost nothing. Pruning is the only real compression lever, and this release uses only that.
 
----
+## Which version should I use
 
-**MiMo-V2.6-Flash-REAP50** is a **50% expert-pruned** version of [`XiaomiMiMo/MiMo-V2.6-Flash`](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash) (315B total / 11B active omni-modal MoE) produced using **REAP** (*Router-weighted Expert Activation Pruning*, [arXiv:2510.13999](https://arxiv.org/abs/2510.13999)), reducing each of the 59 MoE layers from **256 routed experts to 128 routed experts** while keeping **top-8 active routing unchanged**.
+| Repository | Format | Size | Notes |
+|---|---|---|---|
+| **MiMo-V2.6-Flash-REAP50** (this repo) | safetensors: MXFP4 experts, FP8 / BF16 elsewhere | 86.10 GiB weights | The only artifact here that keeps the model's native video path. Needs a runtime with MiMo-V2 support and a 128-expert config. Untested |
+| [MiMo-V2.6-Flash-REAP50-GGUF](https://huggingface.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF) | GGUF `MXFP4_MOE` (86.06 GiB) and a mixed MXFP4/Q2_K file named `Q2_K` (61.64 GiB), plus `mmproj` and `mtp` GGUFs | 61.6–86.1 GiB | For llama.cpp. Text, image and audio at most: llama.cpp has no native path for MiMo video. Never loaded |
 
-Total checkpoint size drops from **157 GiB → 85.8 GiB** (164B total parameters, **11B active parameters**), allowing the full model (including its vision/video/audio encoders and 3-layer MTP speculative decoding head) to fit comfortably in a **single 128 GB unified-memory system** (NVIDIA Jetson AGX Thor, DGX Spark, Mac Studio, or AMD Strix Halo).
+Nothing has been measured, so neither repo can be recommended on quality or speed.
 
----
+## Quickstart
 
-## ✨ Highlights
+No inference command has been run against this checkpoint. All that is known to work is downloading it:
 
-- **Bit-Exact Native OCP MXFP4 Survivor Weights (`85.8 GiB`, Down from `157 GiB`):** All 128 surviving routed experts per MoE layer retain their exact upstream packed OCP MXFP4 (`weight_packed` `U8` + `weight_scale` `F8_E8M0`) bytes without re-quantization.
-- **Unmodified Multimodal Encoders, Attention & 3-Layer MTP Head:** Keeps Layer 0 (dense MLP), all 60 hybrid MLA attention layers (full + sliding-window SWA in FP8 `E4M3` + `F32` block scales), the vision (`vision_tower`), video (`video_tower`), and audio (`audio_tower`) encoders + multi-modal aligners (BF16), and the 3-layer Multi-Token Prediction (`mtp`) head **100% bit-identical to the base checkpoint**.
-- **6-Bucket Domain-Balanced REAP-50 Pruning:** Calibrated across 298 sequences (`610,304` tokens) spanning English general/encyclopedic, software engineering & code, competition mathematics, Chinese (`zh`), agentic tool-use, and synthetic omni-modal embeddings, with union top-16 specialist protection per domain (`35–75` specialists locked per layer).
-- **Single-Box 128 GB Fit:** Leaves **~42 GiB of unified-memory headroom** on a 128 GB Jetson AGX Thor or DGX Spark for KV cache and multimodal activations.
+```bash
+hf download patrickbdevaney/MiMo-V2.6-Flash-REAP50 --local-dir MiMo-V2.6-Flash-REAP50
+```
 
----
+The repo ships Xiaomi's remote code (`modeling_mimo_v2.py`, `configuration_mimo_v2.py`, `auto_map` in `config.json`) and the base model's chat template and processor config. A runtime that loads the base `MiMo-V2.6-Flash-RL` checkpoint should load this one, because only the expert count changed (`n_routed_experts: 128`). **That is untested.** It needs about 86 GiB for the weights alone, before any KV cache.
 
-## 📋 Model Summary
+## Evaluation
 
-| Property | Base ([`XiaomiMiMo/MiMo-V2.6-Flash`](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash)) | **This Model (`MiMo-V2.6-Flash-REAP50`)** |
-|---|---|---|
-| **Total Parameters** | ~315B | **~164B** |
-| **Active Parameters / Token** | ~11B | **~11B (unchanged)** |
-| **Decoder Layers** | 60 (Layer 0 dense + Layers 1–59 MoE) | **60 (Layer 0 dense + Layers 1–59 MoE; [`./config.json`](./config.json))** |
-| **Routed Experts per MoE Layer** | 256 (`n_group=8`, `topk_group=4`) | **128 (`n_group=8`, `topk_group=4`, 16 experts/group)** |
-| **Active Experts per Token (`num_experts_per_tok`)** | 8 (`routed_scaling_factor=2.827`) | **8 (`routed_scaling_factor=2.827`)** |
-| **Expert Precision** | Native MXFP4 (`E2M1` + `E8M0` scale, block=32) | **Native MXFP4 (bit-exact slice, zero re-quantization)** |
-| **Attention Precision** | Native FP8 (`E4M3` + `F32` block scale 128×128) | **Native FP8 (bit-exact copy)** |
-| **Vision / Video / Audio Encoders** | BF16 (`vision_tower`, `video_tower`, `audio_tower`) | **BF16 (bit-exact copy, unpruned; [`./audio_tokenizer/model.safetensors`](./audio_tokenizer/model.safetensors))** |
-| **MTP Draft Head (`mtp.layers.0..2`)** | 3 dense+attention layers (FP8/BF16) | **Preserved bit-exact ([`./model_mtp.safetensors`](./model_mtp.safetensors))** |
-| **Total Checkpoint Size** | 157 GiB | **85.8 GiB (64 EP shards `model_pp0_ep0..63_shard0.safetensors` + [`./model_mtp.safetensors`](./model_mtp.safetensors); [`./model.safetensors.index.json`](./model.safetensors.index.json))** |
-| **License** | Apache-2.0 | **Apache-2.0** |
+**Not yet evaluated.** This checkpoint has no end-to-end measurement of any kind:
 
----
+- no perplexity, no teacher-forced agreement against the unpruned model, and no generative or task benchmark;
+- no image, audio or video test;
+- no long-context or needle-in-a-haystack test;
+- no throughput or memory measurement.
 
-## Evaluation Status Note (2026-10-24)
+The only numbers that exist are selection-time diagnostics on the calibration corpus (see *How it was made*). They measure how much routed output mass the kept experts carry on the data used to pick them. They are not a quality result and should not be read as one.
 
-- **Surviving weights are 100% bit-exact slices of `XiaomiMiMo/MiMo-V2.6-Flash`.** No post-pruning router ridge-regression or weight modification was applied (`e_score_correction_bias` and `gate.weight` for the 128 kept experts are copied verbatim from the base checkpoint; see [`scripts/apply_mask.py`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP/blob/main/scripts/apply_mask.py) and [`wiki/98-router-kd-global.md`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP/blob/main/wiki/98-router-kd-global.md)).
-- **Teacher-vs-student KL / NLL benchmark status:** A paired held-out teacher-vs-student KL/NLL table (of the kind published on [`GLM-5.3-Flash-REAP50-FP8-v2`](https://huggingface.co/patrickbdevaney/GLM-5.3-Flash-REAP50-FP8-v2) and [`DeepSeek-V4.1-Flash-1017-REAP192-MXFP4`](https://huggingface.co/patrickbdevaney/DeepSeek-V4.1-Flash-1017-REAP192-MXFP4)) has **not yet been run** for `MiMo-V2.6-Flash-REAP50` because `MiMo-V2.6-Flash`'s custom `mimo_v2_flash` architecture ([`./modeling_mimo_v2.py`](./modeling_mimo_v2.py)) is not yet wired into our standalone teacher-logit evaluation harness. See the companion [`MiMo-V2.6-Flash-REAP50-GGUF`](https://huggingface.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF) card for additional details on GGUF conversion and runtime support.
+## Hardware & memory
 
----
+- **Produced on:** one NVIDIA Jetson AGX Thor (128 GB unified memory). That machine ran the calibration pass, the expert selection and the checkpoint write.
+- **Run on:** nothing. The model has not been loaded for inference anywhere.
+- **Fit estimate (arithmetic only).** The 86.1 GiB of weights plus an FP8 KV cache at the full 1,048,576 tokens comes to about 97.4 GiB before runtime workspace. The KV figure is about 11.25 GiB: 9 full-attention layers × 4 KV heads × (192 + 128) dims × 1 byte, about 11.5 KB per token, while the 39 SWA layers hold only a 128-token window. That total sits under the roughly 117 GiB usable on Thor, but it has not been tried. A BF16 KV cache would double the KV term.
 
-## What Changed vs. the Base Model
+## How it was made
 
-### Preserved Bit-for-Bit (Zero Re-quantization)
-- **Layer 0 (Dense MLP + Full MLA Attention):** 100% untouched.
-- **All 60 Attention Layers (`self_attn` + `layer_scale`):** Full MLA layers (`0, 5, 11, 17, 23, 29, 35, 41, 47, 53, 59`) and Sliding-Window MLA layers (all others, `sliding_window_size=512`) are copied byte-for-byte in native FP8 (`F8_E4M3` + `F32` `weight_scale_inv`).
-- **Multimodal Towers (`vision_tower`, `video_tower`, `audio_tower`, `multi_modal_projector`):** 100% untouched in `BF16` (plus [`./audio_tokenizer/model.safetensors`](./audio_tokenizer/model.safetensors)).
-- **MTP Speculative Decoding Head (`mtp.layers.0..2`):** 100% untouched in [`./model_mtp.safetensors`](./model_mtp.safetensors).
-- **Surviving 128 Routed Experts per Layer:** Their MXFP4 `weight_packed` (`U8`) and `weight_scale` (`F8_E8M0`) bytes are sliced directly from the base tensors without dequantizing or re-quantizing ([`scripts/mimo_shards.py`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP/blob/main/scripts/mimo_shards.py)).
+1. **Calibration.** Expert statistics were accumulated over **49.0M tokens** in 27 chunks of packed 4,096-token rows. Every document was truncated at 4,095 tokens. The pass ran layer by layer through the unpruned model, one layer resident at a time. The corpus has nine domains:
 
----
+   | domain | tokens | | domain | tokens |
+   |---|---:|---|---|---:|
+   | agentic | 11,501,033 | | image | 3,001,238 |
+   | ballast (general) | 10,003,073 | | science | 2,980,200 |
+   | code | 9,503,748 | | audio | 1,502,892 |
+   | math | 7,502,972 | | video | 1,502,610 |
+   | finance | 1,503,455 | | | |
 
-## Pruning Method
+   The image, audio and video domains are real media run through the model's own towers and processor (with their captions), 6.0M tokens in all, rather than text captions alone. Sources were limited to permissively licensed datasets, with licences checked against the Hugging Face API; a source with no declared licence was excluded. Non-English coverage was not measured separately.
 
-### 1. Group-Balanced REAP Saliency (`n_group = 8`, `16` experts kept per group)
-`MiMo-V2.6-Flash` uses **grouped sigmoid top-k routing** (`n_routed_experts = 256`, `n_group = 8` groups of 32 experts, `topk_group = 4`, `num_experts_per_tok = 8`, `routed_scaling_factor = 2.827`; see [`research/MIMO_V26_FLASH_SURVEY_2026-09-23.md`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP/blob/main/research/MIMO_V26_FLASH_SURVEY_2026-09-23.md)). Because the router reshapes experts into `[n_group, n_routed_experts // n_group]`, pruning **exactly 16 of 32 experts within each of the 8 routing groups** preserves the exact group topology (`n_group = 8`, 16 experts per group) so existing inference kernels (`vLLM`, `SGLang`, `Transformers`, `llama.cpp`) work out-of-the-box with only `n_routed_experts: 128` updated in [`./config.json`](./config.json).
+2. **Selection.** Per-expert saliency used the REAP criterion (`reap_1_1_1`, router-weighted expert activation; [Lasby et al., 2025](https://arxiv.org/abs/2510.13999)). The prune set was solved with the **HOPE** objective (arXiv 2609.18916). HOPE minimises `pᵀFp`, the output error of a prune set, interaction terms between experts included; REAP corresponds to the same objective with those off-diagonal terms set to zero. The ratio was 0.50, uniform across all 47 MoE layers (128 kept per layer). The top 8% of experts in each domain were protected from pruning (`protect_frac 0.08`), and candidates were compared by their **worst** domain rather than their mean.
 
-For every MoE layer $l \in \{1, \dots, 59\}$ and expert $j \in \{0, \dots, 255\}$, we streamed 610,304 calibration tokens layer-by-layer on an NVIDIA Jetson AGX Thor (128 GB unified memory) via [`scripts/mimo_saliency.py`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP/blob/main/scripts/mimo_saliency.py) and [`scripts/calib_pass.py`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP/blob/main/scripts/calib_pass.py), executing the exact `MiMoV2FlashMoE` group-topk routing and dequantizing the selected MXFP4 experts on the fly to compute the **REAP saliency score** ([arXiv:2510.13999](https://arxiv.org/abs/2510.13999)):
+   Selection-time diagnostics for the published mask (`artifacts/masks/mask.json` in the pipeline repo). These are retained routed output mass on the calibration corpus, a proxy and not an evaluation:
 
-$$S_j = \frac{1}{|T_j|} \sum_{t \in T_j} g_{t,j} \cdot \left\| W_{\text{down},j}\!\left(\text{SiLU}(W_{\text{gate},j} h_t) \odot (W_{\text{up},j} h_t)\right) \right\|_2$$
+   | domain | retained | | domain | retained |
+   |---|---:|---|---|---:|
+   | audio (worst) | 0.9935 | | finance | 0.9974 |
+   | image | 0.9937 | | ballast | 0.9976 |
+   | video | 0.9938 | | code | 0.9983 |
+   | science | 0.9973 | | agentic | 0.9983 |
+   | math | 0.9974 | | **mean** | **0.9964** |
 
-### 2. Six-Domain Calibration & Specialist Protection
-Calibration data (`298 sequences × 2048 tokens = 610,304 tokens`; defined in [`scripts/mimo_corpus_spec.py`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP/blob/main/scripts/mimo_corpus_spec.py) and [`wiki/91-capability-coverage.md`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP/blob/main/wiki/91-capability-coverage.md)) was balanced across 6 buckets:
-1. **`general`** — English encyclopedic & long-form analytical text
-2. **`code`** — Multi-language software engineering, algorithms, systems code
-3. **`math`** — Olympiad & undergraduate mathematical proofs and derivations
-4. **`zh`** — Simplified & Traditional Chinese technical and literary text
-5. **`tool`** — Multi-turn agentic tool-calling, JSON schemas, and reasoning traces
-6. **`multimodal`** — Text sequences interleaved with synthetic vision/audio projector embeddings matching the empirical norm/distribution of `multi_modal_projector` outputs ([`scripts/media_loaders.py`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP/blob/main/scripts/media_loaders.py))
+   HOPE objective `pᵀFp` = 0.01624.
 
-Within each layer:
-- The **top-16 experts per domain** (by per-domain REAP score $S_j^{(d)}$ with $f_j^{(d)} \ge 16$) are locked as protected domain specialists (35 to 75 specialists locked per layer; [`scripts/reap_select.py`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP/blob/main/scripts/reap_select.py)).
-- Remaining slots in each of the 8 routing groups (`32 → 16` experts per group) are filled by cross-domain normalized REAP score (with a frequency floor $f_j \ge 40$).
+3. **Routers.** A router-only refit by output matching against the unpruned teacher was tried on all 47 layers. In every layer the refit failed to beat simply slicing the teacher's router (improvement +0.0%), so all 47 routers are the teacher's weights sliced to the kept experts. Only about 1.1% of routed slots in the calibration data had gone to experts that were pruned.
 
----
+4. **Write-out.** Whole expert tensors were dropped and the survivors renumbered, with each router's row order checked against the new numbering. No weights were re-quantized and no healing or fine-tuning was applied.
 
-## Companion GGUF Repository
+Code, gates and the method write-ups are in the pipeline repository: [patrickbdevaney/xiaomi-2.6-flash-REAP](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP).
 
-For `llama.cpp` inference ([`MiMo-V2.6-Flash-REAP50-MXFP4_MOE.gguf`](https://huggingface.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF/blob/main/MiMo-V2.6-Flash-REAP50-MXFP4_MOE.gguf), [`MiMo-V2.6-Flash-REAP50-Q2_K.gguf`](https://huggingface.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF/blob/main/MiMo-V2.6-Flash-REAP50-Q2_K.gguf), plus multimodal `mmproj` and `mtp` GGUFs), see **[`patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF`](https://huggingface.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF)**.
+## Limitations
 
----
+- **Unevaluated.** Nothing shows that this model performs as well as the original. Pruning 50% of experts is lossy and cannot be undone, and losses can be concentrated in one capability even when the averages look fine.
+- **Long context is unverified.** Calibration never saw more than 4,095 tokens, and the 1M figure is config plus memory arithmetic.
+- **Multimodal inputs are untested.** The towers are unchanged, and the calibration fed real image, audio and video through them. No multimodal output from the pruned model has been checked.
+- **Draft heads.** The base repo's separate `dflash/` speculative-decoding drafter is **not** included: it was trained against the unpruned model, and nobody has checked it against this one. The 3 native MTP layers are included unchanged; their acceptance rate on the pruned trunk has not been measured.
+- `README_upstream.md` in this repo is an early draft of this card, not Xiaomi's model card. Some of its wording (for example "fits and serves") was never verified. Use this README instead.
+- Calibration covered the nine domains above. Other domains and languages were not measured.
 
-## 🔎 Provenance & Verification Ledger
+## Changelog / corrections
 
-Every claim, architectural parameter, and pruning step on this card can be audited directly against the files in this repository and the companion open-source pipeline:
+- **2026-09-25:** checkpoint published (weights unchanged since then).
+- **2026-10-24:** correction: never served or benchmarked; "~5 tok/s" was bandwidth arithmetic; the 1M-context fit is sizing arithmetic; per-domain retention is a selection-time proxy. No weights were changed.
+- **2026-11-06:** card rewritten and verified against repo files and build artifacts; a 2026-10-26 automated rewrite contained errors.
 
-| Claim / Artifact | Primary Verification Source |
-|---|---|
-| **Upstream Base Model (`315B` MXFP4+FP8, Apache-2.0)** | [`XiaomiMiMo/MiMo-V2.6-Flash`](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash) · [`./README_upstream.md`](./README_upstream.md) |
-| **REAP Pruning Formulation** | Lasby et al., 2025 — [*REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression*](https://arxiv.org/abs/2510.13999) (`arXiv:2510.13999`) |
-| **Model Config, Modeling Code & Shard Index** | [`./config.json`](./config.json) · [`./modeling_mimo_v2.py`](./modeling_mimo_v2.py) · [`./configuration_mimo_v2.py`](./configuration_mimo_v2.py) · [`./model.safetensors.index.json`](./model.safetensors.index.json) |
-| **MTP Draft Head & Audio Tokenizer Weights** | [`./model_mtp.safetensors`](./model_mtp.safetensors) · [`./audio_tokenizer/model.safetensors`](./audio_tokenizer/model.safetensors) · [`./audio_tokenizer/config.json`](./audio_tokenizer/config.json) |
-| **MiMo-V2.6-Flash REAP-50 Pipeline Repository** | [`patrickbdevaney/xiaomi-2.6-flash-REAP`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP) (`main` @ `329123f`) · [`wiki/README.md`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP/blob/main/wiki/README.md) · [`wiki/00-log.md`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP/blob/main/wiki/00-log.md) |
-| **Architecture Survey & SOTA REAP Methodology** | [`research/MIMO_V26_FLASH_SURVEY_2026-09-23.md`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP/blob/main/research/MIMO_V26_FLASH_SURVEY_2026-09-23.md) · [`research/REAP_METHODOLOGY_SOTA_2026-09-23.md`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP/blob/main/research/REAP_METHODOLOGY_SOTA_2026-09-23.md) |
-| **Saliency Streaming, Selection & Bit-Exact Shard Slicing** | [`scripts/mimo_saliency.py`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP/blob/main/scripts/mimo_saliency.py) · [`scripts/reap_select.py`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP/blob/main/scripts/reap_select.py) · [`scripts/apply_mask.py`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP/blob/main/scripts/apply_mask.py) · [`scripts/mimo_shards.py`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP/blob/main/scripts/mimo_shards.py) |
+## License & attribution
 
----
+MIT, inherited from the base model. The base model, its architecture, its multimodal towers and its MTP layers are the work of the Xiaomi MiMo team: [`XiaomiMiMo/MiMo-V2.6-Flash-RL`](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL). This is an independent community derivative, not affiliated with or endorsed by Xiaomi. The expert-saliency criterion is REAP ([arXiv 2510.13999](https://arxiv.org/abs/2510.13999)); the prune-set objective is HOPE (arXiv 2609.18916).
 
-## Citation & Acknowledgments
+## Citation
 
-- **Base Model:** [Xiaomi MiMo Team — `XiaomiMiMo/MiMo-V2.6-Flash`](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash) (Apache-2.0 License)
-- **Pruning Algorithm:** [REAP: Router-weighted Expert Activation Pruning (arXiv:2510.13999)](https://arxiv.org/abs/2510.13999)
-- **REAP-50 Pipeline:** [`patrickbdevaney/xiaomi-2.6-flash-REAP`](https://github.com/patrickbdevaney/xiaomi-2.6-flash-REAP)
+```bibtex
+@misc{devaney2026mimov26flashreap50,
+  title        = {MiMo-V2.6-Flash-REAP50: 50\% routed-expert pruning of MiMo-V2.6-Flash-RL},
+  author       = {Patrick Devaney},
+  year         = {2026},
+  howpublished = {\url{https://huggingface.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50}}
+}
+```