Instructions to use Yo-Purple/Meditor-3-4B-MoE with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Yo-Purple/Meditor-3-4B-MoE with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Yo-Purple/Meditor-3-4B-MoE", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Yo-Purple/Meditor-3-4B-MoE", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Yo-Purple/Meditor-3-4B-MoE with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Yo-Purple/Meditor-3-4B-MoE"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Yo-Purple/Meditor-3-4B-MoE",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Yo-Purple/Meditor-3-4B-MoE

SGLang

How to use Yo-Purple/Meditor-3-4B-MoE with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Yo-Purple/Meditor-3-4B-MoE" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Yo-Purple/Meditor-3-4B-MoE",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Yo-Purple/Meditor-3-4B-MoE" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Yo-Purple/Meditor-3-4B-MoE",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Yo-Purple/Meditor-3-4B-MoE with Docker Model Runner:
```
docker model run hf.co/Yo-Purple/Meditor-3-4B-MoE
```

Meditor-3-4B-MoE — Shared-Backbone Dynamic Specialist MoE

Meditor-3-4B-MoE is the flagship native PyTorch Mixture-of-Experts architecture in the Meditor series, built on an intact 4.02B Qwen3-4B shared dense core paired with 8 specialist rank-8 LoRA weight-delta experts (math, logic, code, science, constraints, strategy, creative, crisp) and a 2.90M parameter PromptRouter.

For every user prompt, the PromptRouter inspects pooled hidden states from the shared backbone before generating the first token, selects 1 to 3 specialist domain experts, and dynamically modulates the shared linear projections (q/k/v/o/gate/up/down_proj) across all 36 layers prior to decoding.

Standalone GGUF Edition: Looking for llama.cpp / LM Studio compatibility? See Yo-Purple/Meditor-3-4B-Universal.

Meditor Model Series

Generation & Variant	Hugging Face Repo	Base Backbone	Params (Total / Active)	UniBench-100 Mean (/10)	Pass Rate	VRAM Footprint
Meditor 1 (Native MoE)	`Yo-Purple/Meditor-1-2.2B-A500M-MoE`	Qwen2.5-0.5B	2.22B / 0.57B	2.09 (37/100) · 2.42 (Best M1)	1.0% (Best M1)	4.16 GB (BF16)
Meditor 1 (Universal)	`Yo-Purple/Meditor-1-500M-Universal`	Qwen2.5-0.5B (`qwen2`)	0.49B / 0.49B	2.42 (100/100 BF16)	1.0%	0.93 GB / GGUF
Meditor 2 (Native HMoE)	`Yo-Purple/Meditor-2-3.4B-A1.8B-HMoE`	SmolLM2-1.7B	3.41B / 1.83B	3.28 (37/100) · 4.12 (Best M2)	22.0% (Best M2)	6.42 GB (BF16)
Meditor 2 (Universal)	`Yo-Purple/Meditor-2-3.4B-Universal`	SmolLM2-1.7B (`qwen2_moe`)	3.41B / 1.83B	4.12 (100/100 BF16) · 4.00 (Q4_K_M)	22.0% / 18.0%	6.38 GB / GGUF
Meditor 3 (Native MoE) (This Model)	`Yo-Purple/Meditor-3-4B-MoE`	Qwen3-4B (`meditor3`)	4.16B / 4.04B–4.07B	7.31 (100/100, Rank #1/36)	69.0% (7.0% Uniq)	5.43 GiB peak (`bf16_low_vram`)
Meditor 3 (Universal GGUF)	`Yo-Purple/Meditor-3-4B-Universal`	Qwen3-4B (`k2-horizon`)	4.16B (`BF16`) · `Q8_0` · `Q4_K_M`	Compiled GGUF from M3 (`7.31` native)	LM Studio / llama.cpp	4.37–5.95 GiB (quantized)

Universal LLM Benchmark (`UniBench-100`) Cross-Generation Comparison

All benchmark comparisons below come directly from the unified UniBench-100 evaluation suite (100 open-ended synthesis questions across 10 cognitive domains, graded out of 10.0 with pass threshold >= 6.0 by gemini-3.8-flash).

1. Cross-Generation Progression (Full 100/100 Runs)

Rank (of 36 Runs)	Model Run	Series Generation	Evaluated	Mean Score (/10)	Pass Rate (>=6.0)	Unique Solve Rate	Eval Speed
#1	Meditor-3-4B-MoE (Released)	Meditor 3 (Native)	100 / 100	7.31	69.0%	7.0%	35.7 tok/s
#2	Meditor-3 Prior Uncontested (BF16)	Meditor 3 (Prior R&D)	100 / 100	4.62	30.0%	1.0%	35.7 tok/s
#5	Meditor 3 Beta (BF16)	Meditor 3 (Prior R&D)	100 / 100	4.31	22.0%	1.0%	19.0 tok/s
#11	Meditor 2 Universal (BF16)	Meditor 2 Best	100 / 100	4.12	22.0%	0.0%	27.5 tok/s
#18	Meditor 2 Universal (Q4_K_M)	Meditor 2 Quantized	100 / 100	4.00	18.0%	0.0%	141.2 tok/s
#29	Meditor 2 (Native HMoE)	Meditor 2 Native	37 / 100	3.28	8.1%	0.0%	48.7 tok/s
#30	Meditor 1 Universal (BF16)	Meditor 1 Best	100 / 100	2.42	1.0%	0.0%	163.4 tok/s
#31	Meditor 1 Universal (Quant)	Meditor 1 Quantized	100 / 100	2.35	0.0%	0.0%	171.3 tok/s
#33	Meditor 1 (Native MoE)	Meditor 1 Native	37 / 100	2.09	2.7%	0.0%	141.9 tok/s

Key Takeaway: Meditor-3-4B-MoE delivers +3.19 points (+77.4% relative score jump) over the best Meditor 2 run (4.12), +4.89 points (3.02x score) over the best Meditor 1 run (2.42), and +2.69 points over the strongest earlier Meditor 3 research run (4.62).

2. Per-Domain Score Breakdown (Best Runs of Meditor 1, Meditor 2, and Meditor 3)

UniBench-100 Domain	Meditor 1 Best (`2.42`)	Meditor 2 Best (`4.12`)	Meditor 3 Released (`7.31`)	Gain vs M2	M3 Pass Rate
Critical Thinking & Argument Evaluation	3.51	5.98	9.09	`+3.11`	100.0%
Creative & Stylistic Expression	2.91	4.95	8.38	`+3.43`	90.0%
Humanities, Philosophy & Ethics	2.30	4.91	8.35	`+3.44`	80.0%
Physical & Life Sciences	1.68	3.38	7.41	`+4.03`	80.0%
Mathematics & Quantitative Estimation	2.36	1.99	7.27	`+5.28`	60.0%
Factuality & Hallucination Resistance	1.32	5.56	7.09	`+1.53`	60.0%
Real-World Tactical Strategy & Decision-Making	2.10	3.72	6.98	`+3.26`	60.0%
Instruction Following & Strict Constraints	3.65	4.62	6.82	`+2.20`	50.0%
Coding & Software Engineering	2.15	3.26	6.46	`+3.20`	70.0%
Logic & Abstract Reasoning	2.19	2.82	5.27	`+2.45`	40.0%

3. Full Leaderboard Landscape

Architecture Specifications (`Meditor-3-4B-MoE`)

Architectural Parameter	Value
Model Class	`Meditor3ForCausalLM` (`meditor3`, wraps `Qwen3ForCausalLM` via Hugging Face `auto_map`)
Shared Backbone Core	`Qwen/Qwen3-4B` (`4,022,468,096` params, `36` layers, `d_model=2560`, `d_ff=9728`, `32Q / 8KV` heads, `head_dim=128`)
Specialist Expert Bank	`8` domain LoRA cartridges (`r=8, alpha=8`), `16,515,072` params per expert (`132,120,576` total across all 8)
Pre-Generation Router	`PromptRouter` (`2,899,465` params) reading `[mean_pool, last_token]` hidden states
Total Native Parameters	`4,157,488,137` (`4.16B`) · Active per Prompt: `4,038,983,168` (1 expert) to `4,072,013,312` (3 experts)
Adaptive Selection Policy	`1–3` experts per prompt (`cum_threshold=0.72`, `min_prob=0.14`, `min_ratio_to_top1=0.25`, scale `0.35–1.00`)
Context Window & RoPE	`40,960` tokens (`rope_theta=1,000,000`)
On-Disk BF16 Weight Footprint	`7.7499 GiB` (shards + 8 experts + router — under `8.0 GiB`)

The 8 Domain Specialist Experts

Index	Key	Specialist Profile	Target Focus
`0`	`math`	Math & Quantitative	Arithmetic, algebra, combinatorics, probability, verified step-by-step calculation
`1`	`logic`	Deductive Logic & Proofs	Formal deduction, constraint puzzles, causal inference, fallacy analysis
`2`	`code`	Code & Systems	Algorithms, data structures, concurrency, systems design, production code
`3`	`science`	Science & Fact-Check	Physics, STEM reasoning, empirical grounding, false-premise rejection
`4`	`constraints`	Precision & Constraints	Strict formatting, exact word/bullet counts, lipograms, JSON schemas
`5`	`strategy`	Strategy & Synthesis	Tactical planning, triage, tradeoff analysis, executive synthesis
`6`	`creative`	Creative & Humanities	Metaphor, storytelling, philosophy, ethics, rhetoric, expressive prose
`7`	`crisp`	Concise & Direct	Direct answers and bounded reasoning traces without overthinking

Local Hardware & Probe Verification (RTX 4060 8 GB)

Verification Metric	Observed Result
AutoModel Load (`bf16_low_vram`)	`5.26s` load · `4.937 GiB` CUDA allocated (`4.938 GiB` reserved)
Peak CUDA VRAM (Generation)	`5.227 GiB` allocated (`5.432 GiB` reserved) — strictly `< 7.0 GiB`
Probe10 Baseline Comparison	7.02 / 10 (`7/10` pass) vs Raw Qwen3-4B `4.28 / 10` (`+2.74` uplift)
UniBench-100 Eval Speed	`35.7 tok/s` (`Q8_0` backbone + dynamic Python-routed LoRA deltas)
Interactive Web UI & API	Included (`server.py` + `static/index.html` + `chat.py` + OpenAI `/v1/chat/completions`)

Quickstart

1. Standard Hugging Face `transformers` (`trust_remote_code=True`)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "Yo-Purple/Meditor-3-4B-MoE"
tok = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id, trust_remote_code=True)

messages = [{"role": "user", "content": "Solve x^2 - 7x + 12 = 0 briefly."}]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
inputs = tok(prompt, return_tensors="pt")

with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=128, do_sample=False, pad_token_id=tok.eos_token_id)

print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

2. Local Interactive Studio UI & OpenAI-Compatible API

pip install torch transformers safetensors bitsandbytes fastapi uvicorn huggingface_hub
python server.py --backend auto --port 8008
# Open http://127.0.0.1:8008/ (or pass --backend pytorch_4bit for < 3.5 GB VRAM)

Or run interactively in the terminal:

python chat.py --backend auto

License

Released under Apache 2.0. Built on Qwen/Qwen3-4B.

Downloads last month: 25

Safetensors

Model size

4B params

Tensor type

BF16

Model tree for Yo-Purple/Meditor-3-4B-MoE

Base model

Qwen/Qwen3-4B-Base

Finetuned

Qwen/Qwen3-4B