Instructions to use M1n1A1/MiniAI-Quata2-4b-v0.3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use M1n1A1/MiniAI-Quata2-4b-v0.3 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="M1n1A1/MiniAI-Quata2-4b-v0.3") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("M1n1A1/MiniAI-Quata2-4b-v0.3") model = AutoModelForMultimodalLM.from_pretrained("M1n1A1/MiniAI-Quata2-4b-v0.3", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use M1n1A1/MiniAI-Quata2-4b-v0.3 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf M1n1A1/MiniAI-Quata2-4b-v0.3:Q4_K_M # Run inference directly in the terminal: llama cli -hf M1n1A1/MiniAI-Quata2-4b-v0.3:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf M1n1A1/MiniAI-Quata2-4b-v0.3:Q4_K_M # Run inference directly in the terminal: llama cli -hf M1n1A1/MiniAI-Quata2-4b-v0.3:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf M1n1A1/MiniAI-Quata2-4b-v0.3:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf M1n1A1/MiniAI-Quata2-4b-v0.3:Q4_K_M
Use Docker
docker model run hf.co/M1n1A1/MiniAI-Quata2-4b-v0.3:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use M1n1A1/MiniAI-Quata2-4b-v0.3 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "M1n1A1/MiniAI-Quata2-4b-v0.3" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "M1n1A1/MiniAI-Quata2-4b-v0.3", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/M1n1A1/MiniAI-Quata2-4b-v0.3:Q4_K_M
- SGLang
How to use M1n1A1/MiniAI-Quata2-4b-v0.3 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "M1n1A1/MiniAI-Quata2-4b-v0.3" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "M1n1A1/MiniAI-Quata2-4b-v0.3", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "M1n1A1/MiniAI-Quata2-4b-v0.3" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "M1n1A1/MiniAI-Quata2-4b-v0.3", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use M1n1A1/MiniAI-Quata2-4b-v0.3 with Ollama:
ollama run hf.co/M1n1A1/MiniAI-Quata2-4b-v0.3:Q4_K_M
- Unsloth Desktop
- Pi
How to use M1n1A1/MiniAI-Quata2-4b-v0.3 with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf M1n1A1/MiniAI-Quata2-4b-v0.3:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "M1n1A1/MiniAI-Quata2-4b-v0.3:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use M1n1A1/MiniAI-Quata2-4b-v0.3 with Docker Model Runner:
docker model run hf.co/M1n1A1/MiniAI-Quata2-4b-v0.3:Q4_K_M
- Lemonade
How to use M1n1A1/MiniAI-Quata2-4b-v0.3 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull M1n1A1/MiniAI-Quata2-4b-v0.3:Q4_K_M
Run and chat with the model
lemonade run user.MiniAI-Quata2-4b-v0.3-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use M1n1A1/MiniAI-Quata2-4b-v0.3 with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf M1n1A1/MiniAI-Quata2-4b-v0.3:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default M1n1A1/MiniAI-Quata2-4b-v0.3:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use M1n1A1/MiniAI-Quata2-4b-v0.3 with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf M1n1A1/MiniAI-Quata2-4b-v0.3:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "M1n1A1/MiniAI-Quata2-4b-v0.3:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
MiniAI Quata2 (4B) โ v0.3
v0.2 plus reinforcement learning: better at maths, science questions and following instructions, ties Qwen3.5-9B on skill benchmarks.
Quata2 4B v0.3 is MiniAI's compact model from Belgrade. It is Quata2 4B v0.2 trained further with GRPO reinforcement learning on answers that can be checked automatically: instruction-following rules, maths with known answers, and code with unit tests. It runs on one 8 GB GPU (~2.8 GB at Q4_K_M), and it was trained on one 8 GB GPU too.
Highlights
- Release-set mean 77.5, up from 75.4 (v0.2) and 75.1 (Qwen3.5-4B, its original base).
- Against v0.2: 4 real wins, 0 real losses. GPQA Diamond +11.1, IFEval +6.7, MATH-500 +4.0, GSM8K +3.8.
- Against Qwen3.5-4B: 4 real wins (GPQA Diamond +13.6, BFCL tool calling +4.7, MATH-500 +3.2, Belebele Serbian +3.2), 1 real loss (TruthfulQA โ2.8).
- Ties Qwen3.5-9B on GSM8K, MATH-500, MMLU-Pro, GPQA Diamond, IFEval and BFCL (paired, same questions). The 9B is still ahead on knowledge (MMLU, TruthfulQA, HellaSwag, Serbian knowledge) and HumanEval+.
- Thinking mode: GPQA Diamond 71.2% (v0.2: 62.6%), MMLU-Pro 80.5%, GSM8K 87.0%.
- Shorter, decisive answers: on GPQA it ran out of room on 21% of questions with thinking off (v0.2: 49%).
- Still knows who it is (100% with no system prompt) and keeps customer personas (100%).
How it was made
- v0.2: Qwen3.5-4B โ Serbian continued pretraining โ fine-tune (~31,000 examples) โ 80/20 blend with Qwen3.5-4B.
- Prompts: 8,000 tasks with answers a program can check: instruction-following constraints (word counts, formats, casing, forbidden characters), maths with a known final answer, and coding tasks with unit tests. Decontaminated against the evaluation sets (13-gram overlap with IFEval, GSM8K, MMLU, HumanEval, MBPP, MATH-500, AIME).
- Filter: v0.2 answered each prompt 4 times; we kept the 817 prompts where its answers got different scores. Prompts it always passes or always fails teach nothing.
- GRPO: 600 steps, 2 prompts ร 8 answers per step, LoRA rank 32, learning rate 2e-5, thinking off, answers up to 640 tokens. Each answer is scored by the checker (rule check, exact answer, or tests passed); answers cut off by the length limit score nothing. About 6 hours on one RTX 4060 Ti 8 GB. The LoRA is merged into the full weights.
Benchmarks (thinking off)
Same harness, prompts, settings and question samples for every model; Q4_K_M GGUF on llama.cpp, no system prompt. โ / โ = a real win / loss against v0.2 (beyond 2 standard errors, paired on the same questions); unmarked rows are ties. Best 4B per row in bold.
| Benchmark | Quata2 4B v0.3 | Quata2 4B v0.2 | Qwen3.5 4B | Qwen3 4B 2507 | Gemma 4 E4B | Qwen3.5 9B |
|---|---|---|---|---|---|---|
| MMLU | 71.8 | 73.0 | 70.6 | 64.6 | 65.0 | 80.0 |
| MMLU-Pro | 71.3 | 68.7 | 71.3 | 66.3 | 66.0 | 72.3 |
| GPQA Diamond โ | 51.0 | 39.9 | 37.4 | 29.3 | 40.4 | 49.5 |
| ARC-Challenge | 92.6 | 91.6 | 92.4 | 92.0 | 89.2 | 95.0 |
| HellaSwag | 86.4 | 86.0 | 86.0 | 84.6 | 80.6 | 92.4 |
| TruthfulQA (MC1) | 70.2 | 71.0 | 73.0 | 67.0 | 66.2 | 77.6 |
| GSM8K โ | 89.4 | 85.6 | 86.6 | 88.8 | 86.4 | 89.6 |
| MATH-500 โ | 83.4 | 79.4 | 80.2 | 77.2 | 77.2 | 83.6 |
| IFEval (strict) โ | 85.5 | 78.8 | 82.3 | 83.6 | 84.7 | 83.8 |
| BFCL tool calling | 89.9 | 88.7 | 85.2 | 91.7 | 89.9 | 88.6 |
| HumanEval+ | 79.3 | 77.4 | 75.6 | 84.8 | 85.4 | 86.6 |
| MBPP+ | 67.7 | 66.9 | 64.8 | 63.0 | 67.5 | 63.5 |
| Belebele Serbian | 87.8 | 86.6 | 84.6 | 84.2 | 86.8 | 90.2 |
| Global-MMLU Serbian | 58.6 | 60.0 | 58.4 | 50.0 | 51.8 | 67.0 |
| Serbian LLM Eval (generative) | 77.4 | 76.8 | 78.2 | 73.9 | 75.9 | 82.4 |
| Mean | 77.5 | 75.4 | 75.1 | โ | โ | 80.1 |
Thinking mode
Thinking on, up to 32k tokens, sampling 0.6 / 0.95 / 20, same 200 / 198 questions for both.
| Benchmark | v0.3 | v0.2 | Finished within 32k (v0.3 / v0.2) |
|---|---|---|---|
| GSM8K | 87.0 | 85.0 | 100% / 100% |
| MMLU-Pro | 80.5 | 81.0 | 100% / 98% |
| GPQA Diamond โ | 71.2 | 62.6 | 98% / 88% |
MATH-500, SuperGPQA, BIG-Bench Hard and AIME in thinking mode have not been rerun for v0.3 yet (v0.2: 95.0 / 47.0 / 86.7 / 51.7).
Quata's own checks
Targeted tests with the website's system prompt. Small sets, so read them as directional.
| Check | v0.3 | v0.2 | Qwen3.5 4B | Items |
|---|---|---|---|---|
| Knows it is Quata2 (no system prompt) | 100.0 | 100.0 | 0.0 | 36 |
| Never mentions itself unprompted | 100.0 | 100.0 | 98.3 | 300 |
| Stays in a customer persona | 100.0 | 100.0 | 97.0 | 100 |
| Rejects a false premise | 100.0 | 75.0 | 100.0 | 4 |
| Holds a correct answer under pushback | 90.0 | 75.0 | 65.0 | 20 |
| Fixes a real mistake when challenged | 87.5 | 80.0 | 95.0 | 40 |
| Knows where companies are | 91.7 | 91.7 | 91.7 | 12 |
| Facts about Serbia | 50.0 | 75.0 | 62.5 | 8 |
| Real tool-using agent tasks, solved | 81.5 | 85.4 | 80.1 | 151 |
| Multi-step agent tasks, solved | 72.5 | 78.4 | 74.5 | 102 |
None of the differences between v0.3 and v0.2 here are beyond noise.
Reading the numbers: what is noise
Every benchmark is a sample of questions, so every score has a margin of error. We call a difference real only when it is larger than twice its standard error, computed pair by pair on the same questions. On 500 questions that is about 2โ4 points; on the small checks above it is 15โ25 points. Anything smaller is a tie.
Get started
Run it with Transformers
from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration
model_id = "M1n1A1/MiniAI-Quata2-4b-v0.3"
tok = AutoTokenizer.from_pretrained(model_id)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
model_id, torch_dtype="auto", device_map="auto"
)
messages = [{"role": "user", "content": "Ko te je napravio?"}]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True,
enable_thinking=False) # True = think step by step first
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
The safetensors include the Qwen3.5 vision tower (unchanged from v0.2); the GGUF is text-only.
Thinking mode: enable_thinking=True, or put /think / /no_think in a message. Give it room when it thinks:
our thinking-mode numbers use up to 32,000 new tokens.
Run it with llama.cpp
llama-server -m MiniAI-Quata2-4b-v0.3-Q4_K_M.gguf --jinja -c 8192
Recommended sampling: temperature 0.7, top_p 0.8, top_k 20 (thinking: 0.6, 0.95, 20).
Details
- Architecture: Qwen3.5-4B, 4.2B language parameters (+0.33B vision, +0.12B MTP), 32 layers, hidden size 2560, hybrid attention
- Weights: Quata2 4B v0.2 + merged GRPO LoRA (rank 32); vision tower, MTP head and embeddings unchanged
- Context length: 262,144 tokens in the model config; RL used answers up to 640 tokens
- Precision: bf16 safetensors (
9.7 GB with MTP); Q4_K_M GGUF (2.8 GB) - Languages: English first; Serbian, Croatian and Bosnian improved, grammar still behind English
- License: Apache 2.0 (same as the base model)
Honest caveats
- RL improves skills, not knowledge: knowledge benchmarks (MMLU, Global-MMLU Serbian, TruthfulQA) are unchanged or slightly lower, and TruthfulQA is a real loss against Qwen3.5-4B.
- Agent tasks and Serbia facts are a little lower than v0.2 (within noise); in one rerun it named a weekday without checking it with its date tool. Check answers that depend on tools.
- Qwen3 4B 2507 and Gemma 4 E4B are still ahead on HumanEval+; Qwen3 4B 2507 on BFCL.
- The wider public suite from the v0.2 card (MMLU-Redux, SuperGPQA, Winogrande, BBH, MuSR, DROP, Belebele Croatian, MMLU-ProX Serbian, YugoGPT's log-likelihood suite) has not been rerun for v0.3 yet.
Citation
@misc{miniai2026quata2v03,
title = {MiniAI Quata2 4B v0.3},
author = {{MiniAI}},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/M1n1A1/MiniAI-Quata2-4b-v0.3}},
note = {Quata2 4B v0.2 trained further with GRPO on verifiable rewards}
}
- Downloads last month
- 94
Model tree for M1n1A1/MiniAI-Quata2-4b-v0.3
Base model
Qwen/Qwen3.5-4B-Base