---
license: apache-2.0
base_model: openbmb/MiniCPM5-2B
base_model_relation: finetune
language:
- en
tags:
- decision-model
- system-one
- decision-index
- gguf
- ollama
---
# Aurelis 2B
Aurelis is a small **decision model**: it reads a `state` and a set of typed questions (multiple choice `choice`, or
yes/no `noul`) and returns a probability for every option, one forward pass per question. It speaks the same
`/v1/systemone` interface as the other models on the [Decision Index](https://huggingface.co/spaces/multimodalart/jev-decision-index).
It is [MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B) (2.5B parameters) with a merged LoRA adapter (rank 16).
## Results
**Decision Index 0.3, public suite** (all 37 benchmarks, 140,620 requests, scored by the
[kit](https://github.com/apolinario/decision-index)), one run of the published Ollama build; run files in
[MrScratchcat/aurelis-decision-index](https://huggingface.co/datasets/MrScratchcat/aurelis-decision-index). The other
models' numbers are from the board's data of 2026-10-16. Scores are the kit's chance-corrected skill: 0 is guessing, 100
is every answer right.
| | Aurelis 2B | LiquidAI d1-3B | vLLM-SR d3-nano | CoCo-Decision-4B | Perplexity Decider Flash v1.2 | Cloudflare clef-flash | Cloudflare clef |
|---|---:|---:|---:|---:|---:|---:|---:|
| parameters | 2.5B | 3.1B | 2.2B | 4.2B | 4.5B | 9.7B | 27.8B |
| **public index** | **51.29** | 48.99 | 42.22 | 55.38 | 60.06 | 56.15 | 61.71 |
| Knowledge & Reasoning | 32.2 | 24.3 | 25.3 | 34.8 | 41.4 | 53.0 | 52.8 |
| Language Understanding | 54.1 | 56.5 | 42.2 | 63.2 | 65.1 | 47.2 | 61.3 |
| Retrieval & Classification | 54.2 | 52.8 | 58.8 | 59.9 | 65.2 | 52.2 | 62.8 |
| Tools & Automation | 79.5 | 74.5 | 58.9 | 76.8 | 81.0 | 81.9 | 80.7 |
| Arts & Human Taste | 35.9 | 38.9 | 22.4 | 40.2 | 46.6 | 48.4 | 48.9 |
On the board of 2026-10-16 (174 models) 51.29 is place 65, and the highest public index of any model under 4B parameters.
The three models of 4.5B or less above it are CoCo-Decision-4B, vLLM-SR d3-mini (54.90) and Perplexity Decider Flash
v1.2. Against Cloudflare's clef-flash, almost four times its size, Aurelis is 4.9 points behind and ahead on 11 of the 37
benchmarks, among them When2Call (87.0 against 53.8), CLINC150, VAST, PhishNChips and BBH; it is furthest behind where
knowledge or multi-step reasoning decides (MuSR, WinoGrande, CRUXEval, CLadder).
All 37 benchmarks (bold: best of the seven)
| benchmark | area | Aurelis 2B | LiquidAI d1-3B | vLLM-SR d3-nano | CoCo-Decision-4B | Perplexity Decider Flash v1.2 | Cloudflare clef-flash | Cloudflare clef |
|---|---|---:|---:|---:|---:|---:|---:|---:|
| GPQA Diamond | Knowledge | 21.8 | 4.8 | 9.5 | 28.6 | 25.9 | **35.4** | 31.3 |
| WinoGrande | Knowledge | 51.7 | 55.5 | 59.7 | 67.2 | 77.3 | **95.1** | 87.1 |
| GSM8K | Knowledge | 27.9 | 19.0 | 42.2 | 25.9 | **61.2** | 36.3 | 52.1 |
| ChessBench | Knowledge | 9.0 | 8.3 | 0.6 | 10.2 | 14.1 | 16.2 | **17.8** |
| MuSR | Knowledge | 30.6 | 31.3 | 35.7 | 37.6 | 36.8 | **78.0** | 74.2 |
| SATA-Bench | Knowledge | **38.6** | 30.4 | 28.8 | 33.1 | 10.5 | 35.8 | 32.6 |
| CRUXEval | Knowledge | 9.0 | 0.0 | 7.6 | 11.5 | 40.7 | **78.6** | **78.6** |
| CLadder | Knowledge | 71.0 | 13.2 | 20.7 | 67.6 | 70.8 | **95.3** | 88.2 |
| HLE | Knowledge | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| MMLU-Pro | Knowledge | 31.8 | 38.0 | 36.5 | 49.2 | 54.0 | 60.9 | **61.7** |
| BBH | Knowledge | **62.1** | 60.0 | 33.9 | 47.7 | 61.5 | 54.9 | 61.8 |
| ContractNLI | Language | 74.8 | 71.9 | 55.0 | 77.6 | 75.3 | **77.6** | 73.0 |
| ANLI | Language | 46.2 | 43.6 | 34.0 | 54.1 | 54.1 | 38.8 | **55.0** |
| HellaSwag | Language | 79.6 | 85.6 | 87.9 | 92.1 | 91.8 | **98.1** | 97.6 |
| ACOS | Language | **36.1** | 35.7 | 10.2 | 34.1 | 33.4 | 23.6 | 31.0 |
| FinEntity | Language | 72.9 | 82.4 | 81.7 | 80.2 | 86.0 | **95.7** | 94.2 |
| iSarcasmEval | Language | 19.4 | 17.6 | 22.9 | 39.2 | **47.8** | 0.0 | 31.0 |
| VAST | Language | 59.4 | 65.5 | 35.5 | **66.7** | 65.3 | 24.4 | 39.0 |
| NLI4CT | Language | 45.9 | 49.6 | 44.7 | 61.5 | 65.6 | 58.3 | **66.9** |
| RAGTruth | Language | 49.4 | 53.5 | 0.0 | 59.4 | **63.7** | 0.0 | 57.6 |
| BANKING77 | Retrieval | 86.3 | 87.8 | 91.2 | 89.6 | 90.0 | 90.7 | **94.0** |
| CLINC150 | Retrieval | 89.6 | 91.1 | 95.1 | 92.1 | 93.3 | 66.7 | **97.4** |
| BRIGHT | Retrieval | 19.1 | 32.8 | 31.0 | 32.9 | 38.5 | 30.1 | **40.6** |
| Amazon ESCI | Retrieval | 41.3 | 38.8 | 33.7 | 46.0 | **47.0** | 46.6 | 46.5 |
| PhishNChips | Retrieval | **64.2** | 7.7 | 34.0 | 18.0 | 45.1 | 50.1 | 58.7 |
| HoVer | Retrieval | 18.1 | 48.2 | 59.8 | **74.0** | 72.4 | 22.4 | 31.2 |
| BFCL | Tools | 92.3 | 85.7 | 89.4 | 94.5 | 94.3 | **98.3** | 97.9 |
| ToolRet | Tools | 56.6 | 60.4 | 55.2 | 44.0 | 59.7 | 61.0 | **64.3** |
| API-Bank | Tools | 85.0 | 77.7 | 55.5 | 69.9 | 86.2 | **93.0** | 91.8 |
| Home appliances | Tools | 72.7 | 63.6 | 12.5 | 87.5 | 75.0 | **97.7** | 80.7 |
| When2Call | Tools | **87.0** | 82.1 | 76.4 | 86.0 | 86.2 | 53.8 | 63.3 |
| BPoMP | Arts | 88.5 | 64.3 | 14.3 | 87.0 | 83.2 | 90.8 | **94.1** |
| Humicroedit | Arts | 25.8 | 27.3 | 21.1 | 31.3 | 23.1 | **50.1** | 33.1 |
| POP909 | Arts | 8.4 | 23.3 | 9.5 | 7.2 | **25.5** | 0.7 | 14.8 |
| cfcolor | Arts | 14.1 | 23.4 | 18.3 | 28.7 | **36.0** | 31.9 | 34.0 |
| Habermas | Arts | 43.1 | 40.2 | 8.1 | 22.4 | 44.3 | **59.0** | 54.5 |
| New Yorker | Arts | 35.8 | 55.0 | 63.3 | 64.7 | **67.6** | 57.9 | 62.8 |
**Parts of the [Artificial Analysis Decision Index](https://artificialanalysis.ai/evaluations/artificial-analysis-decision-index)**,
on the exact items its [methodology](https://artificialanalysis.ai/methodology/decision-index) lists: 3,423 When2Call tasks
(options shuffled per task, as AA does) and 513 HLE multiple-choice questions. Aurelis's numbers are measured here, the
others are AA's of 2026-10-17. Aurelis is not on that board, its AA-Omniscience part is private and WildGuardTest was not
run, so there is no AA index for it. Aurelis was trained on When2Call's train split, never its test.
| decision model | maker | When2Call | HLE-MultiChoice | AA index |
|---|---|---:|---:|---:|
| **Aurelis 2B** (measured here) | Leptos | 92.6 | 12.1 | — |
| d1-ultra | Liquid AI | 92.6 | 15.0 | 65.2 |
| Celeris-1 Decision | Celeris | 83.7 | 13.3 | 52.3 |
| GLiDE | Fastino | 83.7 | 10.5 | 53.9 |
| Microsoft-Decision-1 | Microsoft | 81.9 | 10.9 | 53.8 |
| Jev 1.14 | TypeSafe | 80.8 | 18.9 | 57.1 |
| Jev 1.13 | TypeSafe | 80.7 | 20.9 | 57.5 |
| Mercury Decide | Inception | 79.3 | 10.1 | 52.7 |
| d1 | Liquid AI | 77.4 | 11.5 | 50.3 |
| Decider v1.1 27B | Perplexity | 76.5 | 15.4 | 53.1 |
| GPT-6 Luna (Decisions) | OpenAI | 74.1 | 11.7 | 48.5 |
| Clef | Cloudflare | 71.0 | 11.1 | 48.6 |
| Clef-Flash | Cloudflare | 64.4 | 10.3 | 45.8 |
| Clef-Omni | Cloudflare | 62.1 | 9.0 | 44.3 |
## Use
**Ollama** (0.40.5 or newer):
```sh
ollama pull mrscratchcat22/aurelis:2b
curl -s localhost:11434/v1/systemone -d '{
"model": "mrscratchcat22/aurelis:2b",
"state": "Customer: my card was charged twice for one order and I need the money back before Friday.",
"questions": {
"route": {"type": "choice", "instructions": "Which team handles this?",
"criteria": {"billing": "Billing", "technical": "Technical support", "account": "Account"}},
"urgent": {"type": "noul", "instructions": "Is this urgent?"}
}
}'
```
The GGUF here (`aurelis-2b-q8_0.gguf`, with `Modelfile`) is that same build: decision type `pplx-decider` and the
trained prompt as `tokenizer.chat_template.systemone`, for llama.cpp's `/v1/systemone` (b11669 or newer) and Ollama.
Other sizes, from Q4_K_M (1.56 GB) to F16, each measured against this one, are in
[MrScratchcat/Aurelis-2B-GGUF](https://huggingface.co/MrScratchcat/Aurelis-2B-GGUF)
(`ollama pull hf.co/MrScratchcat/Aurelis-2B-GGUF:Q4_K_M`).
**Transformers**, with `aurelis.py` from this repo (same prompt and probabilities as Ollama; on 23 test questions the same
answers, largest probability difference 0.009, which is the 8-bit rounding of the GGUF):
```python
from aurelis import Aurelis
decide = Aurelis("MrScratchcat/Aurelis-2B")
decide("Customer: my card was charged twice for one order and I need the money back before Friday.",
{"route": {"type": "choice", "instructions": "Which team handles this?",
"criteria": {"billing": "Billing", "technical": "Technical support", "account": "Account"}},
"urgent": {"type": "noul", "instructions": "Is this urgent?"}})
# {'route': {'type': 'choice', 'choice': 'billing', 'probabilities': {'billing': 0.84, 'technical': 0.09, 'account': 0.07}},
# 'urgent': {'type': 'noul', 'noul': 0.91}}
```
Prompt format, if you write your own runner: one user message per question holding the JSON
`{"state": ..., "question": ..., "options": [{"label": "A", "key": ..., "description": ...}, ...]}` (Python `json.dumps`
spacing, `ensure_ascii=False`; an object state is compact JSON text; a yes/no question is `false`/`No` then `true`/`Yes`),
then the assistant turn `\n\n\n\n`, and the answer is the softmax over the option letters. No `` in front.
## Limits
- At most 24 options per question (more need two rounds, as the Decision Index engine does).
- Trained on prompts of up to 4,096 tokens; Ollama serves it with a 16,384-token window, where long states still work but
were seen less in training.
- English. Small model: knowledge-heavy questions (GPQA, HLE, MMLU-Pro) stay near what its size allows.
## Training
61,901 examples, 3,869 steps, LoRA rank 16 (alpha 32), learning rate 5e-5, on one AMD RX 9070 XT. The output layer carries a
calibration temperature of 1.55, fitted on the held-out split and built into the weights. Data:
- training splits of public datasets (NLI, claim verification, tool calling and tool selection, classification, multiple
choice, ...) laid out as decision requests, with an overlap check that drops every row whose text also occurs in the
Decision Index suite; no test set of any benchmark was trained on;
- [tasksource/tasksource-jev-typed-decisions](https://huggingface.co/datasets/tasksource/tasksource-jev-typed-decisions) (5,924 rows);
- a few dozen rows from small game simulations and hand-written decisions.
## License
Apache-2.0, as the base model.