Instructions to use patrickbdevaney/MiMo-V2.6-Flash-REAP50 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="patrickbdevaney/MiMo-V2.6-Flash-REAP50", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("patrickbdevaney/MiMo-V2.6-Flash-REAP50", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "patrickbdevaney/MiMo-V2.6-Flash-REAP50"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "patrickbdevaney/MiMo-V2.6-Flash-REAP50",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50

SGLang

How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "patrickbdevaney/MiMo-V2.6-Flash-REAP50" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "patrickbdevaney/MiMo-V2.6-Flash-REAP50",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "patrickbdevaney/MiMo-V2.6-Flash-REAP50" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "patrickbdevaney/MiMo-V2.6-Flash-REAP50",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50 with Docker Model Runner:
```
docker model run hf.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50
```

patrickbdevaney commited on 21 days ago

Commit

39f6bbe

verified ·

1 Parent(s): 8127c52

Add files using upload-large-folder tool

Browse files

Files changed (1) hide show

README.md +4 -27

README.md CHANGED Viewed

@@ -14,31 +14,10 @@ tags:
 # MiMo-V2.6-Flash-REAP50
 `XiaomiMiMo/MiMo-V2.6-Flash-RL` with **50% of its routed experts removed** — 256
-experts per layer down to **128** — sized to fit a single NVIDIA Jetson AGX Thor
-(117 GiB unified memory). 86.1 GiB across 65 shards. It has **not yet been served or
-benchmarked**; see *Corrections (2026-10-24)*.
-The vision and audio towers are untouched and `audio_tokenizer/` ships with the checkpoint;
-image, audio and video inputs have not been tested on the pruned model.
-## Corrections (2026-10-24)
-Earlier wording implied more than has been measured. **No weights were changed**; only the
-description is corrected.
-1. **Never served or benchmarked.** This checkpoint was published without being run for
-   generation. No quality evaluation (perplexity, generative or task accuracy) and no throughput
-   measurement has been made on any hardware. Decode-speed figures associated with this release
-   (for example "~5 tok/s" on Jetson AGX Thor) are memory-bandwidth arithmetic, not measurements.
-2. **1M context is an estimate, not a tested capability.** That the weights plus a 1M-token KV cache
-   fit in 117 GiB comes from sizing arithmetic (about 115 GiB, a ~2 GiB margin). No long-context run
-   or needle-in-a-haystack test has been done, and calibration covered sequences of at most 4,095
-   tokens.
-3. **The retention figures are a proxy.** The per-domain numbers are retained gated output mass on
-   the calibration corpus, computed at selection time. They are not an end-to-end quality result.
-4. **Planned:** a serving measurement on Jetson AGX Thor (decode tok/s, plus needle-in-a-haystack
-   at long context) and an end-to-end comparison against the unpruned model. This note will be
-   replaced with those results.
 ## How the experts were chosen
@@ -89,8 +68,6 @@ no router here is worse than simply slicing the teacher's.
 - The `dflash/` speculative-decoding draft head from the source repo is **not** included: it was
   trained against the unpruned expert set and is not valid for this checkpoint.
 - Pruned MoE experts do not come back. This is a lossy, irreversible transform of the base model.
-- Not yet served: no throughput, long-context or end-to-end quality measurement exists for this
-  checkpoint (see *Corrections (2026-10-24)*).
 ## Provenance

 # MiMo-V2.6-Flash-REAP50
 `XiaomiMiMo/MiMo-V2.6-Flash-RL` with **50% of its routed experts removed** — 256
+experts per layer down to **128** — so that it fits and serves on a single
+NVIDIA Jetson AGX Thor (117 GiB unified memory). 86.1 GiB across 65 shards.
+Vision, audio and video input are preserved; `audio_tokenizer/` ships with the checkpoint.
 ## How the experts were chosen
 - The `dflash/` speculative-decoding draft head from the source repo is **not** included: it was
   trained against the unpruned expert set and is not valid for this checkpoint.
 - Pruned MoE experts do not come back. This is a lossy, irreversible transform of the base model.
 ## Provenance