Instructions to use patrickbdevaney/MiMo-V2.6-Flash-REAP50 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="patrickbdevaney/MiMo-V2.6-Flash-REAP50", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("patrickbdevaney/MiMo-V2.6-Flash-REAP50", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "patrickbdevaney/MiMo-V2.6-Flash-REAP50"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "patrickbdevaney/MiMo-V2.6-Flash-REAP50",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50

SGLang

How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "patrickbdevaney/MiMo-V2.6-Flash-REAP50" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "patrickbdevaney/MiMo-V2.6-Flash-REAP50",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "patrickbdevaney/MiMo-V2.6-Flash-REAP50" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "patrickbdevaney/MiMo-V2.6-Flash-REAP50",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50 with Docker Model Runner:
```
docker model run hf.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50
```

patrickbdevaney commited on 21 days ago

Commit

8127c52

verified ·

1 Parent(s): be6ffa5

Card corrections: describe only measured results

Browse files

Files changed (1) hide show

README.md +27 -4

README.md CHANGED Viewed

@@ -14,10 +14,31 @@ tags:
 # MiMo-V2.6-Flash-REAP50
 `XiaomiMiMo/MiMo-V2.6-Flash-RL` with **50% of its routed experts removed** — 256
-experts per layer down to **128** — so that it fits and serves on a single
-NVIDIA Jetson AGX Thor (117 GiB unified memory). 86.1 GiB across 65 shards.
-Vision, audio and video input are preserved; `audio_tokenizer/` ships with the checkpoint.
 ## How the experts were chosen
@@ -68,6 +89,8 @@ no router here is worse than simply slicing the teacher's.
 - The `dflash/` speculative-decoding draft head from the source repo is **not** included: it was
   trained against the unpruned expert set and is not valid for this checkpoint.
 - Pruned MoE experts do not come back. This is a lossy, irreversible transform of the base model.
 ## Provenance

 # MiMo-V2.6-Flash-REAP50
 `XiaomiMiMo/MiMo-V2.6-Flash-RL` with **50% of its routed experts removed** — 256
+experts per layer down to **128** — sized to fit a single NVIDIA Jetson AGX Thor
+(117 GiB unified memory). 86.1 GiB across 65 shards. It has **not yet been served or
+benchmarked**; see *Corrections (2026-10-24)*.
+The vision and audio towers are untouched and `audio_tokenizer/` ships with the checkpoint;
+image, audio and video inputs have not been tested on the pruned model.
+## Corrections (2026-10-24)
+Earlier wording implied more than has been measured. **No weights were changed**; only the
+description is corrected.
+1. **Never served or benchmarked.** This checkpoint was published without being run for
+   generation. No quality evaluation (perplexity, generative or task accuracy) and no throughput
+   measurement has been made on any hardware. Decode-speed figures associated with this release
+   (for example "~5 tok/s" on Jetson AGX Thor) are memory-bandwidth arithmetic, not measurements.
+2. **1M context is an estimate, not a tested capability.** That the weights plus a 1M-token KV cache
+   fit in 117 GiB comes from sizing arithmetic (about 115 GiB, a ~2 GiB margin). No long-context run
+   or needle-in-a-haystack test has been done, and calibration covered sequences of at most 4,095
+   tokens.
+3. **The retention figures are a proxy.** The per-domain numbers are retained gated output mass on
+   the calibration corpus, computed at selection time. They are not an end-to-end quality result.
+4. **Planned:** a serving measurement on Jetson AGX Thor (decode tok/s, plus needle-in-a-haystack
+   at long context) and an end-to-end comparison against the unpruned model. This note will be
+   replaced with those results.
 ## How the experts were chosen
 - The `dflash/` speculative-decoding draft head from the source repo is **not** included: it was
   trained against the unpruned expert set and is not valid for this checkpoint.
 - Pruned MoE experts do not come back. This is a lossy, irreversible transform of the base model.
+- Not yet served: no throughput, long-context or end-to-end quality measurement exists for this
+  checkpoint (see *Corrections (2026-10-24)*).
 ## Provenance