Instructions to use kueizen/Marco-Nano-Instruct-REAP15 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kueizen/Marco-Nano-Instruct-REAP15 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="kueizen/Marco-Nano-Instruct-REAP15")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("kueizen/Marco-Nano-Instruct-REAP15")
model = AutoModelForCausalLM.from_pretrained("kueizen/Marco-Nano-Instruct-REAP15", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use kueizen/Marco-Nano-Instruct-REAP15 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kueizen/Marco-Nano-Instruct-REAP15"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kueizen/Marco-Nano-Instruct-REAP15",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/kueizen/Marco-Nano-Instruct-REAP15

SGLang

How to use kueizen/Marco-Nano-Instruct-REAP15 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "kueizen/Marco-Nano-Instruct-REAP15" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kueizen/Marco-Nano-Instruct-REAP15",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "kueizen/Marco-Nano-Instruct-REAP15" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kueizen/Marco-Nano-Instruct-REAP15",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use kueizen/Marco-Nano-Instruct-REAP15 with Docker Model Runner:
```
docker model run hf.co/kueizen/Marco-Nano-Instruct-REAP15
```

Marco-Nano-Instruct-REAP15

Marco-Nano-Instruct (8.6B total, ~0.6B active, 232 experts, 8 active per token) with 15% of its experts removed by REAP: 198 of 232 experts kept. These are the full-precision (bf16) safetensors weights, for fine-tuning, re-quantising or running with transformers.

For ready-to-run quantised files at every pruning ratio, see kueizen/Marco-Nano-Instruct-REAP-GGUF.

Load it

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "kueizen/Marco-Nano-Instruct-REAP15"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto")

How this was made

Pruning: REAP (Router-weighted Expert Activation Pruning, Cerebras Research), using the reference implementation. REAP scores each expert by its router weight times the size of its output over a calibration set, then removes the lowest-scoring experts whole. It is one-shot: no retraining. Seed 42.
Calibration set: theblackcat102/evol-codealpaca-v1 (train split, shuffled, seed 42), 8 batches per category, batch size 2, max sequence length 512 tokens.
Experts removed: 15% (198 of 232 kept).

We have not run task benchmarks on these checkpoints (MMLU, coding, multilingual). Test them on your own workload before relying on them.

License and credits

Derived from ATH-MaaS/Marco-Nano-Instruct and released under the same Apache 2.0 licence. What we changed: removed experts with REAP. Nothing else was modified or retrained. REAP is by Cerebras Research: paper, code. All credit for the base model goes to its authors.

Published by Kueizen.

Downloads last month: 193

Safetensors

Model size

7B params

Tensor type

BF16

Model tree for kueizen/Marco-Nano-Instruct-REAP15

Base model

ATH-MaaS/Marco-Nano-Instruct

Finetuned

(4)

this model

Paper for kueizen/Marco-Nano-Instruct-REAP15

REAP the Experts: Why Pruning Prevails for One-Shot MoE compression

Paper • 2510.13999 • Published Oct 15, 2025 • 23