Instructions to use M1n1A1/MiniAI-Quata2-9b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use M1n1A1/MiniAI-Quata2-9b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="M1n1A1/MiniAI-Quata2-9b") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("M1n1A1/MiniAI-Quata2-9b") model = AutoModelForMultimodalLM.from_pretrained("M1n1A1/MiniAI-Quata2-9b", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use M1n1A1/MiniAI-Quata2-9b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf M1n1A1/MiniAI-Quata2-9b:Q4_K_M # Run inference directly in the terminal: llama cli -hf M1n1A1/MiniAI-Quata2-9b:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf M1n1A1/MiniAI-Quata2-9b:Q4_K_M # Run inference directly in the terminal: llama cli -hf M1n1A1/MiniAI-Quata2-9b:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf M1n1A1/MiniAI-Quata2-9b:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf M1n1A1/MiniAI-Quata2-9b:Q4_K_M
Use Docker
docker model run hf.co/M1n1A1/MiniAI-Quata2-9b:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use M1n1A1/MiniAI-Quata2-9b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "M1n1A1/MiniAI-Quata2-9b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "M1n1A1/MiniAI-Quata2-9b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/M1n1A1/MiniAI-Quata2-9b:Q4_K_M
- SGLang
How to use M1n1A1/MiniAI-Quata2-9b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "M1n1A1/MiniAI-Quata2-9b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "M1n1A1/MiniAI-Quata2-9b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "M1n1A1/MiniAI-Quata2-9b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "M1n1A1/MiniAI-Quata2-9b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use M1n1A1/MiniAI-Quata2-9b with Ollama:
ollama run hf.co/M1n1A1/MiniAI-Quata2-9b:Q4_K_M
- Unsloth Desktop
- Pi
How to use M1n1A1/MiniAI-Quata2-9b with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf M1n1A1/MiniAI-Quata2-9b:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "M1n1A1/MiniAI-Quata2-9b:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use M1n1A1/MiniAI-Quata2-9b with Docker Model Runner:
docker model run hf.co/M1n1A1/MiniAI-Quata2-9b:Q4_K_M
- Lemonade
How to use M1n1A1/MiniAI-Quata2-9b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull M1n1A1/MiniAI-Quata2-9b:Q4_K_M
Run and chat with the model
lemonade run user.MiniAI-Quata2-9b-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use M1n1A1/MiniAI-Quata2-9b with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf M1n1A1/MiniAI-Quata2-9b:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default M1n1A1/MiniAI-Quata2-9b:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use M1n1A1/MiniAI-Quata2-9b with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf M1n1A1/MiniAI-Quata2-9b:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "M1n1A1/MiniAI-Quata2-9b:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
MiniAI Quata2 (9B)
Qwen3.5-9B fine-tuned and trained with reinforcement learning: ahead of its base on maths, instruction following, tool calling and coding, with no real losses.
Quata2 9B is MiniAI's larger model from Belgrade. It is Qwen3.5-9B with the Quata2 fine-tune and GRPO reinforcement learning on answers that can be checked automatically: instruction-following rules, maths with known answers and code with unit tests. It is the same recipe as Quata2 4B v0.3, scaled up. It runs on one 8 GB GPU (~5.8 GB at Q4_K_M) and was trained on one 8 GB GPU (RTX 4060 Ti) too.
Highlights
- Release-set mean 81.8, against 80.1 for Qwen3.5-9B (its base) and 77.5 for Quata2 4B v0.3.
- Against Qwen3.5-9B: 5 real wins, 0 real losses. IFEval +6.5 (90.3%), BFCL tool calling +3.8, MATH-500 +3.4, GSM8K +3.0, coding (HumanEval+ and MBPP+ together) +3.1.
- Agents: solves 93.4% of real tool-using tasks on the MiniAI website's tools (Qwen3.5-9B: 80.8%, real win) and 91.2% of the multi-step ones (80.4%, also a real win).
- Serbian: at or above its base on every Serbian benchmark: Belebele Serbian 91.6%, Serbian LLM Eval 82.7%. On YugoGPT's own log-likelihood suite it scores 62.2, against 60.5 for YugoGPT 7B (a tie within the margin).
- Knows who it is: says it is Quata2 with no system prompt 94% of the time (Qwen3.5-9B: 0%), keeps customer personas 100% of the time and never mentions itself unprompted.
How it was made
- Fine-tune: Qwen3.5-9B + LoRA (rank 16) on the Quata2 instruction set used for the 4B v0.2 (27,000 conversations, 9.4M trained tokens: English and Serbian chat, tools, identity, reasoning), 2 epochs, learning rate 5e-5. The result is blended 80/20 with Qwen3.5-9B (weight-space ensemble), which keeps the base's knowledge. No Serbian continued pretraining: on the 4B it cost more on general benchmarks than it gave on Serbian.
- Prompts: from the same decontaminated pool as v0.3 (instruction-following constraints, maths with a known final answer, coding with unit tests), the 1,445 the 4B could not solve every time. The 9B answered each 4 times; we kept the 590 where its answers got different scores (385 instruction following, 106 code, 99 maths).
- GRPO: 600 steps, 2 prompts ร 8 answers per step, LoRA rank 32, learning rate 2e-5, thinking off, answers up to 640 tokens, no KL penalty. Each answer is scored by its checker; answers cut off by the length limit are left out of the loss. About 17 hours on one RTX 4060 Ti 8 GB. The LoRA is merged into the full weights.
Benchmarks (thinking off)
Same harness, prompts, settings and question samples for every model; Q4_K_M GGUF on llama.cpp, no system prompt (our GGUF is quantized with the same importance matrix as the Qwen3.5-9B GGUF it is compared against). โ = a real win against Qwen3.5-9B (beyond 2 standard errors, paired on the same questions); unmarked rows are ties. Best per row in bold.
| Benchmark | Quata2 9B | Qwen3.5 9B | Quata2 4B v0.3 | Qwen3.5 4B |
|---|---|---|---|---|
| MMLU | 77.6 | 80.0 | 71.8 | 70.6 |
| MMLU-Pro | 74.0 | 72.3 | 71.3 | 71.3 |
| GPQA Diamond | 54.0 | 49.5 | 51.0 | 37.4 |
| ARC-Challenge | 94.8 | 95.0 | 92.6 | 92.4 |
| HellaSwag | 91.8 | 92.4 | 86.4 | 86.0 |
| TruthfulQA (MC1) | 77.8 | 77.6 | 70.2 | 73.0 |
| GSM8K โ | 92.6 | 89.6 | 89.4 | 86.6 |
| MATH-500 โ | 87.0 | 83.6 | 83.4 | 80.2 |
| IFEval (strict) โ | 90.3 | 83.8 | 85.5 | 82.3 |
| BFCL tool calling โ | 92.4 | 88.6 | 89.9 | 85.2 |
| HumanEval+ | 89.6 | 86.6 | 79.3 | 75.6 |
| MBPP+ | 66.7 | 63.5 | 67.7 | 64.8 |
| Belebele Serbian | 91.6 | 90.2 | 87.8 | 84.6 |
| Global-MMLU Serbian | 64.2 | 67.0 | 58.6 | 58.4 |
| Serbian LLM Eval (generative) | 82.7 | 82.4 | 77.4 | 78.2 |
| Mean | 81.8 | 80.1 | 77.5 | 75.1 |
HumanEval+ and MBPP+ count as one coding row in the mean; together they are a real win (+3.1).
Serbian LLM Eval (YugoGPT's suite)
Log-likelihood, 7 tasks ร 500 questions, the same script, prompts and questions for every model (4-bit).
| Task | Quata2 9B | Qwen3.5 9B | YugoGPT 7B | Quata2 4B v0.2 |
|---|---|---|---|---|
| ARC-Challenge | 49.4 | 48.4 | 41.0 | 43.0 |
| ARC-Easy | 72.2 | 69.6 | 60.6 | 66.4 |
| BoolQ | 83.6 | 82.2 | 67.2 | 84.4 |
| HellaSwag | 60.0 | 59.6 | 69.2 | 54.0 |
| OpenBookQA | 41.8 | 41.6 | 43.2 | 37.0 |
| PIQA | 68.8 | 69.4 | 75.0 | 67.8 |
| Winogrande | 59.8 | 60.8 | 67.6 | 60.2 |
| Mean | 62.2 | 61.7 | 60.5 | 59.0 |
Quata2 9B is far ahead of YugoGPT at reading and reasoning in Serbian (BoolQ +16.4, ARC-Easy +11.6, ARC-Challenge +8.4); YugoGPT, pretrained on much more Serbian text, is ahead at completing natural Serbian sentences (HellaSwag, PIQA, Winogrande). The mean difference (+1.7) is within the margin (ยฑ2.2).
Quata's own checks
Targeted tests with the website's system prompt. Small sets, so read them as directional.
| Check | Quata2 9B | Qwen3.5 9B | Quata2 4B v0.3 | Items |
|---|---|---|---|---|
| Knows it is Quata2 (no system prompt) | 94.4 | 0.0 | 100.0 | 36 |
| Never mentions itself unprompted | 100.0 | 98.0 | 100.0 | 300 |
| Stays in a customer persona | 100.0 | 99.0 | 100.0 | 100 |
| Rejects a false premise | 100.0 | 75.0 | 100.0 | 4 |
| Fixes a real mistake when challenged | 92.5 | 90.0 | 87.5 | 40 |
| Holds a correct answer under pushback | 50.0 | 60.0 | 90.0 | 20 |
| Knows where companies are | 91.7 | 91.7 | 91.7 | 12 |
| Facts about Serbia | 87.5 | 87.5 | 50.0 | 8 |
| Real tool-using agent tasks, solved โ | 93.4 | 80.8 | 81.5 | 151 |
| Multi-step agent tasks, solved โ | 91.2 | 80.4 | 72.5 | 102 |
On "holds a correct answer under pushback" it never switched to the wrong answer it was pushed towards. In a rerun, 6 of the misses kept the right answer politely ("Wikipedia says 1862; if you're sure it was 1834, check another source") and are scoring misses; in the other 5 it searched again, found nothing, and said it was not certain instead of standing by its first answer.
Reading the numbers: what is noise
Every benchmark is a sample of questions, so every score has a margin of error. We call a difference real only when it is larger than twice its standard error, computed pair by pair on the same questions. On 500 questions that is about 2โ4 points; on the small checks above it is 8โ30 points. Anything smaller is a tie.
Get started
Run it with Transformers
from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration
model_id = "M1n1A1/MiniAI-Quata2-9b"
tok = AutoTokenizer.from_pretrained(model_id)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
model_id, torch_dtype="auto", device_map="auto"
)
messages = [{"role": "user", "content": "Ko te je napravio?"}]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True,
enable_thinking=False) # True = think step by step first
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
The safetensors include the Qwen3.5 vision tower and MTP layers, unchanged from Qwen3.5-9B; the GGUF is text-only.
Thinking mode: enable_thinking=True, or put /think / /no_think in a message. Give it room when it thinks.
Run it with llama.cpp
llama-server -m MiniAI-Quata2-9b-Q4_K_M.gguf --jinja -c 8192
On an 8 GB GPU with a desktop running, offload most layers (-ngl 30) if the full model does not fit.
Recommended sampling: temperature 0.7, top_p 0.8, top_k 20 (thinking: 0.6, 0.95, 20).
Details
- Architecture: Qwen3.5-9B, 8.95B language parameters (+0.46B vision, +0.24B MTP), 32 layers, hidden size 4096, hybrid attention
- Weights: Qwen3.5-9B + Quata2 fine-tune (80/20 blend) + merged GRPO LoRA (rank 32); vision tower and MTP layers unchanged
- Context length: 262,144 tokens in the model config; training used conversations up to 1,792 tokens and RL answers up to 640
- Precision: bf16 safetensors (
19 GB); Q4_K_M GGUF (5.8 GB) - Languages: English first; Serbian, Croatian and Bosnian, grammar still behind English
- License: Apache 2.0 (same as the base model)
Honest caveats
- Fine-tuning and RL improve skills, not knowledge: MMLU and Global-MMLU Serbian are 2โ3 points below Qwen3.5-9B (within noise), HellaSwag and ARC are level.
- When a re-check search finds nothing, it can give up a correct answer and say it is unsure (see the checks above).
- With thinking off it ran out of room on 29% of GPQA Diamond questions at our answer limit (Qwen3.5-9B: 47%).
- Thinking-mode benchmarks and the wider public suite from the 4B v0.2 card (MMLU-Redux, SuperGPQA, BBH, MuSR, DROP, Belebele Croatian, MMLU-ProX Serbian) have not been run for the 9B yet.
- Serbian is good but not native-level; YugoGPT is still better at natural sentence completion in Serbian.
Citation
@misc{miniai2026quata2_9b,
title = {MiniAI Quata2 9B},
author = {{MiniAI}},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/M1n1A1/MiniAI-Quata2-9b}},
note = {Qwen3.5-9B fine-tuned and trained with GRPO on verifiable rewards}
}
- Downloads last month
- -