Instructions to use Remek/basal-omni-2.0-mini-preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Remek/basal-omni-2.0-mini-preview with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Remek/basal-omni-2.0-mini-preview") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("Remek/basal-omni-2.0-mini-preview") model = AutoModelForMultimodalLM.from_pretrained("Remek/basal-omni-2.0-mini-preview", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Remek/basal-omni-2.0-mini-preview with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Remek/basal-omni-2.0-mini-preview" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Remek/basal-omni-2.0-mini-preview", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Remek/basal-omni-2.0-mini-preview
- SGLang
How to use Remek/basal-omni-2.0-mini-preview with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Remek/basal-omni-2.0-mini-preview" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Remek/basal-omni-2.0-mini-preview", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Remek/basal-omni-2.0-mini-preview" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Remek/basal-omni-2.0-mini-preview", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Remek/basal-omni-2.0-mini-preview with Docker Model Runner:
docker model run hf.co/Remek/basal-omni-2.0-mini-preview
basal-omni-2.0-mini (preview)
basal-omni-2.0-mini (preview) is an early release of the small model of basal-omni 2.0, on Google Gemma 4 E4B. The final basal-omni 2.0 will be released later together with the whole family: nano, mini, max and ultra. This preview lets you try the 2.0 mini now; the final mini may differ from it.
What it is. Instead of writing an answer, the model reads a state (a message, a document, retrieved passages, an
agent trace, a web page as JSON). It then answers a typed question about it (choice, yes/no noul, score, and in
the basal engine multi, act and tool). The answer is a calibrated probability for each allowed answer, in one
forward pass, without generating text. The answer never falls outside the options you give, and the probability says
how sure the model is.
What is new in 2.0.
- Two-way reading: the model reads the state in both directions before it answers (served by the basal engine 2.5 or later; see Limitations for long prompts).
- Higher accuracy on our hidden Polish and English decision benchmark (Werdykt) and on the held-out tests below.
- More decisions can be automated at a given error target (Coverage below).
- The same typed questions and the same API as basal-omni-1.0.
Results
Served with the public basal engine 2.8.0 from the released files on one NVIDIA H100: --mode eager, bf16, one option order (the default), the shipped CALIBRATION.json. Werdykt was read with basal 2.5 (see below).
| benchmark | basal-omni-2.0-mini (preview) |
|---|---|
| Werdykt v1, macro (hidden PL + EN, 10 categories Γ 2 languages, 5,000 items) | 0.811 (95% interval 0.801β0.821) |
| Werdykt v1, Polish / English | 0.807 / 0.814 |
| PL decisions (held-out test, 7,081 items) | 0.951 |
| PL general (knowledge, exams, reading; 3,079 items) | 0.767 |
| PL-Real (300 real Polish documents) | 0.960 |
| EN external panel (10 public English decision tasks, 3,842 items) | 0.811 |
| EN corpus test (1,479 items) | 0.692 |
| Coverage at 1% error (share decided automatically with the shipped threshold) | 58.3% at 0.58% observed error |
| Coverage at 5% error | 87.8% at 4.0% observed error |
- Werdykt v1: measured with basal 2.5 (build d72e614),
--mode fast, one option order, on one NVIDIA RTX PRO 6000 Blackwell (bf16). The public engine 2.8.0 gives the same probabilities on our parity set (largest difference 5e-6). Weakest categories: rules (0.382; compute exact thresholds and dates in code), long documents (0.694) and contracts (0.726). Strongest: retrieval (0.982), real documents (0.978), routing (0.966). - Coverage: on the held-out PL decisions and EN corpus tests (8,560 decisions), with the per-type thresholds of the shipped
CALIBRATION.jsonas the server applies them. At the 1% target,scorequestions have no certified threshold and go to a person; per type: choice 78.2% at 0.74%, yes/no 43.8% at 0.15%. At 5%: choice 94.7%, yes/no 88.0%, score 47.6%. The test figures stored insideCALIBRATION.jsoncome from the calibration fit (an earlier engine build) and differ slightly; the numbers here are what 2.8.0 serves.
English external panel by task
The EN external panel row above is the mean of these ten public English decision tasks (same run as Results).
| task | items | accuracy |
|---|---|---|
| FinancialPhraseBank | 300 | 0.957 |
| Belebele (English) | 300 | 0.940 |
| WinoGrande | 300 | 0.907 |
| Circa | 300 | 0.837 |
| TabFact | 300 | 0.830 |
| BIG-Bench Hard | 540 | 0.811 |
| RAGTruth | 300 | 0.780 |
| ContractNLI | 300 | 0.777 |
| TruthfulQA (binary) | 902 | 0.649 |
| JudgeBench | 300 | 0.627 |
| mean of the ten tasks | 3,842 | 0.811 |
Werdykt against other 4B typed-decision models
The same 5,000 hidden Werdykt v1 items (Polish and English), the same client and the same scoring, over each model's typed-decision API. Each model is served by its maker's own server. The lead is basal-omni-2.0-mini's paired difference over the same items, in points, with a 95% bootstrap interval.
| model | Werdykt v1 [95% interval] | Polish / English | basal-omni-2.0-mini lead |
|---|---|---|---|
| basal-omni-2.0-mini (preview) | 0.811 [0.801β0.821] | 0.807 / 0.814 | β |
| torchcast-decision-4b (v1.3) | 0.799 [0.788β0.809] | 0.787 / 0.810 | +1.2 [+0.1, +2.2] |
| KnowLine-4B-Gen5.5ΒΉ | 0.776 [0.766β0.787] | 0.764 / 0.788 | +3.4 [+2.3, +4.5] |
| h2o-lightning-4b (v1.1) | 0.753 [0.742β0.764] | 0.745 / 0.761 | +5.8 [+4.7, +6.9] |
| Bielik-One-4B-r1 | 0.696 [0.684β0.709] | 0.687 / 0.705 | +11.5 [+10.2, +12.6] |
- Read between 8 Oct and 13 Nov 2026; torchcast-decision-4b, KnowLine-4B and Bielik-One-4B on one NVIDIA RTX PRO 6000 Blackwell, h2o-lightning-4b on one H100, basal-omni-2.0-mini as in Results.
- ΒΉ KnowLine-4B was served in FP8; the other models in bf16.
- Werdykt is hidden, so only these totals are shown.
Accuracy by document length
Accuracy on Werdykt (5,000 items, Polish and English; a hidden set, so only these totals are shown) by the length of the state, in characters (a token is about 2 characters; 1,000 characters is about half a page). 81.1% overall. 95% Wilson intervals in brackets.
| state length | items | accuracy |
|---|---|---|
| 0-1k | 709 | 71.4% [68, 75] |
| 1k-2k | 2,598 | 80.0% [78, 82] |
| 2k-4k | 796 | 92.3% [90, 94] |
| 4k-8k | 394 | 97.5% [95, 99] |
| 8k-16k | 128 | 80.5% [73, 86] |
| 16k+ | 375 | 65.9% [61, 70] |
Short states are mostly rules and abstention items, which are harder for every model, so read the rows as accuracy per bucket, not as the effect of length alone. The long-document family keeps the task and varies only the length:
| state length (long-document family) | items | accuracy |
|---|---|---|
| 8k-20k | 226 | 74.3% [68, 80] |
| 20k-30k | 194 | 70.1% [63, 76] |
| 30k+ | 80 | 53.8% [43, 64] |
Same run as the Werdykt row above. The intervals are about the items, not about run-to-run noise.
Quick start
1. Install into a fresh uv environment:
uv venv --python 3.12 --allow-existing ~/basal-env && source ~/basal-env/bin/activate
uv pip install torch==2.11.0 torchvision==0.26.0 --index-url https://download.pytorch.org/whl/cu128
uv pip install "basal[media] @ https://github.com/rkinas/basal/archive/refs/tags/v2.8.0.tar.gz"
2. Start the server. It prints basal: model ... ready:
basal-serve --model Remek/basal-omni-2.0-mini-preview --name basal-omni-2.0-mini-preview --port 8000
On an NVIDIA GPU the engine picks the mode itself: fast (CUDA graphs) on Hopper with FlashAttention 3 and on
Blackwell workstation GPUs, eager (PyTorch, bf16) on other GPUs. The weights take about 16 GB. A two-way model is served
with --mode eager or --mode fast only; on Apple Silicon pass --mode eager --device mps, because the engine's default there does
not serve two-way models (PyTorch on Metal; several times slower than on an NVIDIA GPU, 32 GB of memory or more recommended). One option order is asked by default; --orders 2 uses CALIBRATION.o2.json automatically.
3. Ask questions:
curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
"state": "Klient: od wczoraj nie mogΔ zalogowaΔ siΔ do bankowoΕci internetowej, system pokazuje bΕΔ
d hasΕa.",
"questions": {
"dept": {"type": "choice", "instructions": "Do ktΓ³rego dziaΕu skierowaΔ zgΕoszenie?",
"criteria": {"cards": "Reklamacje kart", "online": "Wsparcie bankowoΕci elektronicznej", "loans": "Kredyty"}},
"urgent": {"type": "noul", "instructions": "Czy klient nie moΕΌe korzystaΔ z usΕugi?"}}}'
Full API (multi, act, tool, option keys, max_error): API.
Speed
One question per request, one option order, on one NVIDIA H100 (bf16), basal 2.8.0:
| measurement | --mode fast |
--mode eager |
|---|---|---|
| sequential, 200 requests: p50 / p95 per question | 11 ms / 17 ms | 54 ms / 73 ms |
| 32 concurrent clients, 1,000 requests | 123.5 decisions/s | 59.7 decisions/s |
| sequential, states of 4,000-5,000 tokens: p50 per question | 98 ms | 212 ms |
| 16 questions about one state in one request, 8 concurrent | 200 decisions/s | 94 decisions/s |
fastis the default on Hopper GPUs with FlashAttention 3;eagerwas measured with--media on.- GPU memory while serving: 20.6 GB (
fast), 31.7 GB (eagerwith--media on). --orders 2costs about twice as much per question.
Files
model.safetensors, tokenizer,chat_template.jinja,processor_config.json: the model (Gemma 4 E4B architecture, 262k vocabulary). Gemma's image and audio encoders are included and unchanged.CALIBRATION.json: per-type temperatures and certified confidence thresholds for 1% / 5% error, fitted on the served engine (--mode eager, bf16, one option order).CALIBRATION.o2.json: the same for--orders 2(answers read in both option orders and combined).basal.json: prompt format 2 and the readout protocol.SHA256SUMS: checksums of the model files.
How it works
The model receives a fixed chat prompt with the state, the question and coded options (AβZ, then two-character codes,
up to 255). The assistant turn is prefilled with {"answer": ", and the decision is the softmax over the next-token
logits of the option codes only: one forward pass, no text generation. The server applies the calibrated temperature of
the question type from CALIBRATION.json.
Use the confidence. CALIBRATION.json stores thresholds for a target error of 1% or 5% among accepted decisions.
They were chosen on calibration data before testing. Accept decisions above the threshold automatically and route the
rest to a person. At the 1% target, score questions have no certified threshold and are deferred.
Limitations
- Preview. This is an early release of the 2.0 mini; the final 2.0 family (nano, mini, max, ultra) comes later. Validate it on your own documents before relying on it.
- Long prompts: the state is read two-way up to 4,096 prompt tokens (8,192 tokens per request). Longer prompts are read one way, as in basal-omni-1.0, and the answer does not say which reading was used. Accuracy on the longest documents is lower (see Accuracy by document length).
- Serving features: for a two-way model the engine turns off the state cache, the prefix cache, reading one state once for many questions (SOAM) and the long lane. Ask several questions about one state in one request.
- Images and audio: Gemma 4's encoders are included, but this preview was trained and evaluated on text only, and
--media onneeds--mode eager. Use it for text; media answers are not evaluated here. - Exact arithmetic and thresholds are a weak spot of one-pass decisions (Werdykt rules 0.382): compute them in code.
- Option order: one order is asked by default;
--orders 2reduces sensitivity to option order at about twice the cost. - Engine 2.8.0: bf16 only; no FP8 build of this preview.
- GGUF and MLX (experimental, not recommended): basal-omni-2.0-mini-preview-GGUF and basal-omni-2.0-mini-preview-MLX read the state one way only, so they answer differently from this model (their cards give the measured cost). For the model on this card, use basal-serve
--mode fastor--mode eager(on a Mac--mode eager --device mps). - Review: decisions with serious consequences for people should be reviewed by a person.
Citation
@software{kinas2026basal20,
title = {basal-2.0: typed-decision models and inference engine for Polish and English},
author = {Kinas, Remigiusz},
year = {2026},
url = {https://github.com/rkinas/basal}
}
License and attribution
Apache-2.0. Fine-tuned from google/gemma-4-E4B-it (Apache-2.0).
- Downloads last month
- 94