MiMo-V2.6-Distill-Qwen-9B-MTP-Repacked

MiMo-V2.6-Distill-Qwen-9B is a 9-billion-parameter agentic model from Xiaomi MiMo, built via supervised fine-tuning of Qwen3.5-9B on MiMo-generated data spanning coding, general-purpose agent tasks, visual coding, and cybersecurity, released as an SFT checkpoint intended as a starting point for open research in agentic reinforcement learning. It was trained on a weighted mixture of 77.4B total tokens (27.2B loss-bearing) split roughly evenly across code (29.9%), general (28.5%), visual (27.4%), and cyber (14.2%) domains, and shows dramatic gains over its Qwen3.5-9B base across nearly every reported benchmark — jumping from 5.0 to 30.3 on AutomationBench v1.0.6, 5.7 to 31.3 on internal cyber evaluation, 27.0 to 37.1 on Terminal-Bench 2.1, and 19.5 to 51.6 on internal coding evaluation, alongside solid improvements on SWE-bench Verified (61.1) and SWE-bench Pro (44.6). It ships with its tokenizer and MiMo v2.6 chat template, is served via SGLang with a dedicated mimo reasoning parser and explicit thinking-mode control through an OpenAI-compatible API, and is fine-tuned from the Apache-2.0-licensed Qwen3.5-9B base.

MTP — Speculative Decoding: Yes | GGUF — MiMo-V2.6-Distill-Qwen-9B-MTP-Repacked(GGUF)

Multi-Token Prediction (MTP) GGUF is a specialized GGUF model file format extension that integrates speculative decoding directly into the model weights to significantly accelerate local inference. Unlike traditional speculative decoding which requires a separate, smaller "draft" model, MTP GGUF files include additional output heads within the main model architecture that predict multiple future tokens in a single forward pass.

Model Files [GGUF]

File Name	Quant Type	File Size	File Link	Description
MiMo-V2.6-Distill-Qwen-9B-MTP.BF16.gguf	BF16	18.4 GB	Link	Full BF16 weights. Highest quality, largest file size.
MiMo-V2.6-Distill-Qwen-9B-MTP.Q3_K_L.gguf	Q3_K_L	5.05 GB	Link	Lower quality but usable, good for low RAM availability.
MiMo-V2.6-Distill-Qwen-9B-MTP.Q3_K_M.gguf	Q3_K_M	4.74 GB	Link	Low quality.
MiMo-V2.6-Distill-Qwen-9B-MTP.Q4_K_M.gguf	Q4_K_M	5.78 GB	Link	Good quality, default size for most use cases, recommended.
MiMo-V2.6-Distill-Qwen-9B-MTP.Q4_K_S.gguf	Q4_K_S	5.49 GB	Link	Slightly lower quality with more space savings, recommended.
MiMo-V2.6-Distill-Qwen-9B-MTP.Q5_K_M.gguf	Q5_K_M	6.64 GB	Link	High quality, recommended.
MiMo-V2.6-Distill-Qwen-9B-MTP.Q5_K_S.gguf	Q5_K_S	6.47 GB	Link	High quality, recommended.
MiMo-V2.6-Distill-Qwen-9B-MTP.Q6_K.gguf	Q6_K	7.56 GB	Link	Very high quality, near perfect, recommended.
MiMo-V2.6-Distill-Qwen-9B-MTP.mmproj-bf16.gguf	mmproj-bf16	922 MB	Link	Multimodal projection file in BF16 format. Used for vision/language models.

MTP-Repacked [Transformers, BF16]

File Name	File Type	File Size	File Link	Description
chat_template.jinja	Jinja Template	3.92 kB	Link	Chat template for formatting conversations and prompts.
config.json	JSON	2.78 kB	Link	Core model architecture and configuration parameters.
generation_config.json	JSON	126 Bytes	Link	Default text generation parameters (e.g., temperature, top_p).
merges.txt	Text	3.35 MB	Link	Byte-Pair Encoding (BPE) merge rules for the tokenizer.
model-00001-of-00004.safetensors	Safetensors	5.18 GB	Link	Model weight shard 1/4. Required to load the model.
model-00002-of-00004.safetensors	Safetensors	5.13 GB	Link	Model weight shard 2/4. Required to load the model.
model-00003-of-00004.safetensors	Safetensors	5.17 GB	Link	Model weight shard 3/4. Required to load the model.
model-00004-of-00004.safetensors	Safetensors	3.83 GB	Link	Model weight shard 4/4. Required to load the model.
model.safetensors.index.json	JSON	70.4 kB	Link	Index file mapping model weights to their respective shards.
preprocessor_config.json	JSON	443 Bytes	Link	Configuration for preprocessing vision/image inputs.
processor_config.json	JSON	1.19 kB	Link	Configuration for the multimodal processor (text + vision).
tokenizer.json	JSON	20 MB	Link	Complete tokenizer vocabulary and encoding rules.
tokenizer_config.json	JSON	1.17 kB	Link	Tokenizer-specific configuration parameters.
video_preprocessor_config.json	JSON	385 Bytes	Link	Configuration for preprocessing video inputs.
vocab.json	JSON	6.72 MB	Link	Tokenizer vocabulary mapping tokens to IDs.

Run Inference (llama.cpp)

/path/to/llama.cpp/build/bin/llama-cli \
  -m /path/to/models/MiMo-V2.6-Distill-Qwen-9B-MTP.BF16.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  -ngl 99 \
  -c 8192 \
  --cache-type-k q8_0 \
  --seed 42 \
  -n 8192 \
  -st \
  -p "Write the code for swapping variables."

llama.cpp

LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp

MTP-repackaged and quantized using llama.cpp by — prithivMLmods.