basal-omni-2.0-mini (preview)

GitHub Website

basal-omni-2.0-mini (preview) is an early release of the small model of basal-omni 2.0, on Google Gemma 4 E4B. The final basal-omni 2.0 will be released later together with the whole family: nano, mini, max and ultra. This preview lets you try the 2.0 mini now; the final mini may differ from it.

What it is. Instead of writing an answer, the model reads a state (a message, a document, retrieved passages, an agent trace, a web page as JSON). It then answers a typed question about it (choice, yes/no noul, score, and in the basal engine multi, act and tool). The answer is a calibrated probability for each allowed answer, in one forward pass, without generating text. The answer never falls outside the options you give, and the probability says how sure the model is.

What is new in 2.0.

  • Two-way reading: the model reads the state in both directions before it answers (served by the basal engine 2.5 or later; see Limitations for long prompts).
  • Higher accuracy on our hidden Polish and English decision benchmark (Werdykt) and on the held-out tests below.
  • More decisions can be automated at a given error target (Coverage below).
  • The same typed questions and the same API as basal-omni-1.0.

Results

Served with the public basal engine 2.8.0 from the released files on one NVIDIA H100: --mode eager, bf16, one option order (the default), the shipped CALIBRATION.json. Werdykt was read with basal 2.5 (see below).

benchmark basal-omni-2.0-mini (preview)
Werdykt v1, macro (hidden PL + EN, 10 categories Γ— 2 languages, 5,000 items) 0.811 (95% interval 0.801–0.821)
Werdykt v1, Polish / English 0.807 / 0.814
PL decisions (held-out test, 7,081 items) 0.951
PL general (knowledge, exams, reading; 3,079 items) 0.767
PL-Real (300 real Polish documents) 0.960
EN external panel (10 public English decision tasks, 3,842 items) 0.811
EN corpus test (1,479 items) 0.692
Coverage at 1% error (share decided automatically with the shipped threshold) 58.3% at 0.58% observed error
Coverage at 5% error 87.8% at 4.0% observed error
  • Werdykt v1: measured with basal 2.5 (build d72e614), --mode fast, one option order, on one NVIDIA RTX PRO 6000 Blackwell (bf16). The public engine 2.8.0 gives the same probabilities on our parity set (largest difference 5e-6). Weakest categories: rules (0.382; compute exact thresholds and dates in code), long documents (0.694) and contracts (0.726). Strongest: retrieval (0.982), real documents (0.978), routing (0.966).
  • Coverage: on the held-out PL decisions and EN corpus tests (8,560 decisions), with the per-type thresholds of the shipped CALIBRATION.json as the server applies them. At the 1% target, score questions have no certified threshold and go to a person; per type: choice 78.2% at 0.74%, yes/no 43.8% at 0.15%. At 5%: choice 94.7%, yes/no 88.0%, score 47.6%. The test figures stored inside CALIBRATION.json come from the calibration fit (an earlier engine build) and differ slightly; the numbers here are what 2.8.0 serves.

English external panel by task

The EN external panel row above is the mean of these ten public English decision tasks (same run as Results).

task items accuracy
FinancialPhraseBank 300 0.957
Belebele (English) 300 0.940
WinoGrande 300 0.907
Circa 300 0.837
TabFact 300 0.830
BIG-Bench Hard 540 0.811
RAGTruth 300 0.780
ContractNLI 300 0.777
TruthfulQA (binary) 902 0.649
JudgeBench 300 0.627
mean of the ten tasks 3,842 0.811

Werdykt against other 4B typed-decision models

The same 5,000 hidden Werdykt v1 items (Polish and English), the same client and the same scoring, over each model's typed-decision API. Each model is served by its maker's own server. The lead is basal-omni-2.0-mini's paired difference over the same items, in points, with a 95% bootstrap interval.

model Werdykt v1 [95% interval] Polish / English basal-omni-2.0-mini lead
basal-omni-2.0-mini (preview) 0.811 [0.801–0.821] 0.807 / 0.814 –
torchcast-decision-4b (v1.3) 0.799 [0.788–0.809] 0.787 / 0.810 +1.2 [+0.1, +2.2]
KnowLine-4B-Gen5.5ΒΉ 0.776 [0.766–0.787] 0.764 / 0.788 +3.4 [+2.3, +4.5]
h2o-lightning-4b (v1.1) 0.753 [0.742–0.764] 0.745 / 0.761 +5.8 [+4.7, +6.9]
Bielik-One-4B-r1 0.696 [0.684–0.709] 0.687 / 0.705 +11.5 [+10.2, +12.6]
  • Read between 8 Oct and 13 Nov 2026; torchcast-decision-4b, KnowLine-4B and Bielik-One-4B on one NVIDIA RTX PRO 6000 Blackwell, h2o-lightning-4b on one H100, basal-omni-2.0-mini as in Results.
  • ΒΉ KnowLine-4B was served in FP8; the other models in bf16.
  • Werdykt is hidden, so only these totals are shown.

Accuracy by document length

Accuracy on Werdykt (5,000 items, Polish and English; a hidden set, so only these totals are shown) by the length of the state, in characters (a token is about 2 characters; 1,000 characters is about half a page). 81.1% overall. 95% Wilson intervals in brackets.

state length items accuracy
0-1k 709 71.4% [68, 75]
1k-2k 2,598 80.0% [78, 82]
2k-4k 796 92.3% [90, 94]
4k-8k 394 97.5% [95, 99]
8k-16k 128 80.5% [73, 86]
16k+ 375 65.9% [61, 70]

Short states are mostly rules and abstention items, which are harder for every model, so read the rows as accuracy per bucket, not as the effect of length alone. The long-document family keeps the task and varies only the length:

state length (long-document family) items accuracy
8k-20k 226 74.3% [68, 80]
20k-30k 194 70.1% [63, 76]
30k+ 80 53.8% [43, 64]

Same run as the Werdykt row above. The intervals are about the items, not about run-to-run noise.

Quick start

1. Install into a fresh uv environment:

uv venv --python 3.12 --allow-existing ~/basal-env && source ~/basal-env/bin/activate
uv pip install torch==2.11.0 torchvision==0.26.0 --index-url https://download.pytorch.org/whl/cu128
uv pip install "basal[media] @ https://github.com/rkinas/basal/archive/refs/tags/v2.8.0.tar.gz"

2. Start the server. It prints basal: model ... ready:

basal-serve --model Remek/basal-omni-2.0-mini-preview --name basal-omni-2.0-mini-preview --port 8000

On an NVIDIA GPU the engine picks the mode itself: fast (CUDA graphs) on Hopper with FlashAttention 3 and on Blackwell workstation GPUs, eager (PyTorch, bf16) on other GPUs. The weights take about 16 GB. A two-way model is served with --mode eager or --mode fast only; on Apple Silicon pass --mode eager --device mps, because the engine's default there does not serve two-way models (PyTorch on Metal; several times slower than on an NVIDIA GPU, 32 GB of memory or more recommended). One option order is asked by default; --orders 2 uses CALIBRATION.o2.json automatically.

3. Ask questions:

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "state": "Klient: od wczoraj nie mogę zalogować się do bankowości internetowej, system pokazuje błąd hasła.",
  "questions": {
    "dept":   {"type": "choice", "instructions": "Do którego działu skierować zgłoszenie?",
               "criteria": {"cards": "Reklamacje kart", "online": "Wsparcie bankowoΕ›ci elektronicznej", "loans": "Kredyty"}},
    "urgent": {"type": "noul", "instructions": "Czy klient nie może korzystać z usługi?"}}}'

Full API (multi, act, tool, option keys, max_error): API.

Speed

One question per request, one option order, on one NVIDIA H100 (bf16), basal 2.8.0:

measurement --mode fast --mode eager
sequential, 200 requests: p50 / p95 per question 11 ms / 17 ms 54 ms / 73 ms
32 concurrent clients, 1,000 requests 123.5 decisions/s 59.7 decisions/s
sequential, states of 4,000-5,000 tokens: p50 per question 98 ms 212 ms
16 questions about one state in one request, 8 concurrent 200 decisions/s 94 decisions/s
  • fast is the default on Hopper GPUs with FlashAttention 3; eager was measured with --media on.
  • GPU memory while serving: 20.6 GB (fast), 31.7 GB (eager with --media on).
  • --orders 2 costs about twice as much per question.

Files

  • model.safetensors, tokenizer, chat_template.jinja, processor_config.json: the model (Gemma 4 E4B architecture, 262k vocabulary). Gemma's image and audio encoders are included and unchanged.
  • CALIBRATION.json: per-type temperatures and certified confidence thresholds for 1% / 5% error, fitted on the served engine (--mode eager, bf16, one option order).
  • CALIBRATION.o2.json: the same for --orders 2 (answers read in both option orders and combined).
  • basal.json: prompt format 2 and the readout protocol.
  • SHA256SUMS: checksums of the model files.

How it works

The model receives a fixed chat prompt with the state, the question and coded options (A–Z, then two-character codes, up to 255). The assistant turn is prefilled with {"answer": ", and the decision is the softmax over the next-token logits of the option codes only: one forward pass, no text generation. The server applies the calibrated temperature of the question type from CALIBRATION.json.

Use the confidence. CALIBRATION.json stores thresholds for a target error of 1% or 5% among accepted decisions. They were chosen on calibration data before testing. Accept decisions above the threshold automatically and route the rest to a person. At the 1% target, score questions have no certified threshold and are deferred.

Limitations

  • Preview. This is an early release of the 2.0 mini; the final 2.0 family (nano, mini, max, ultra) comes later. Validate it on your own documents before relying on it.
  • Long prompts: the state is read two-way up to 4,096 prompt tokens (8,192 tokens per request). Longer prompts are read one way, as in basal-omni-1.0, and the answer does not say which reading was used. Accuracy on the longest documents is lower (see Accuracy by document length).
  • Serving features: for a two-way model the engine turns off the state cache, the prefix cache, reading one state once for many questions (SOAM) and the long lane. Ask several questions about one state in one request.
  • Images and audio: Gemma 4's encoders are included, but this preview was trained and evaluated on text only, and --media on needs --mode eager. Use it for text; media answers are not evaluated here.
  • Exact arithmetic and thresholds are a weak spot of one-pass decisions (Werdykt rules 0.382): compute them in code.
  • Option order: one order is asked by default; --orders 2 reduces sensitivity to option order at about twice the cost.
  • Engine 2.8.0: bf16 only; no FP8 build of this preview.
  • GGUF and MLX (experimental, not recommended): basal-omni-2.0-mini-preview-GGUF and basal-omni-2.0-mini-preview-MLX read the state one way only, so they answer differently from this model (their cards give the measured cost). For the model on this card, use basal-serve --mode fast or --mode eager (on a Mac --mode eager --device mps).
  • Review: decisions with serious consequences for people should be reviewed by a person.

Citation

@software{kinas2026basal20,
  title   = {basal-2.0: typed-decision models and inference engine for Polish and English},
  author  = {Kinas, Remigiusz},
  year    = {2026},
  url     = {https://github.com/rkinas/basal}
}

License and attribution

Apache-2.0. Fine-tuned from google/gemma-4-E4B-it (Apache-2.0).

Downloads last month
94
Safetensors
Model size
8B params
Tensor type
BF16
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for Remek/basal-omni-2.0-mini-preview

Finetuned
(480)
this model
Quantizations
2 models