SPX-CD-Omni

CMDB-1500 leaderboard · Release overview · Dataset

SPX-CD-Omni (Simplex Calibrated Decision Model) by SurdAI is a Gemma 4 12B LoRA adapter for text, image, and audio decisions. It supports up to 1,024 choices, including single-choice, multi-select, binary judgments, and ordinal ratings, and outputs candidate probabilities without generated reasoning.

LoRA rank/alpha: 32/64 · Adapter: ~501 MiB · Runner context: 16,384 tokens.

Download

pip install huggingface_hub
hf download SurdAI/SPX-CD-Omni --local-dir SPX-CD-Omni
cd SPX-CD-Omni

Run inference

Install CUDA-enabled PyTorch for Transformers. Use separate environments for the two backends.

Transformers — BF16 base:

pip install -r requirements-transformers.txt
python infer.py --backend transformers --adapter . \
  --input examples/text.json --output predictions.jsonl

vLLM — AWQ INT4 base:

pip install -r requirements-vllm.txt
python infer.py --backend vllm --base nicklas373/gemma-4-12B-it-AWQ \
  --adapter vllm-adapter --input examples/text.json --output predictions.jsonl

For images or multi-select, use examples/image.json or examples/multi_select.json as the input. See these files for the input format and python infer.py --help for options.

Run the audio test

pip install -r requirements-audio.txt
python smoke_audio.py --adapter vllm-adapter --output audio-results.json

Tests four bundled speech clips and silent controls. Base and adapter each scored 3/4.

Results

Self-evaluated with the AWQ INT4 base.

Benchmark / subset	Effort	Questions	Accuracy ↑	I ↑	C ↑	ECE ↓	TVD ↓	RPS ↓
CMDB-1500	1	1,500	73.00%	45.77	77.38	0.0246	0.4018	0.1205
CMDB text	1	1,200	71.00%	45.63	75.75	0.0220	0.5013	0.1205
CMDB images (native)	1	300	81.00%	70.43	85.49	0.0612	0.1679	N/A
CMDB-1500	2	1,500	76.27%	48.12	79.46	0.0511	0.4019	0.1173
CMDB text	2	1,200	74.67%	48.04	77.73	0.0476	0.5003	0.1173
CMDB images (native)	2	300	82.67%	73.02	84.59	0.0688	0.1706	N/A
JevBench Public 231	1	231	83.12%	57.35	78.42	0.1300 / 0.1597 / 0.1485	0.0682	0.0304
JevBench Hard 111	1	111	66.67%	38.48	67.01	0.1300 / 0.2531 / 0.2776	0.0682	0.0838
Decision Bench text	1	949	88.20%	83.63	84.12	0.0518	0.2140	N/A
Decision Bench image-origin (text-rendered)	1	122	80.33%	72.49	81.61	0.0760	0.2157	N/A
Decision Bench all (text-rendered)	1	1,071	87.30%	82.37	84.15	0.0514	0.2142	N/A

JevBench ECE is Choice / Noul / Score. N/A means no rating questions. Metric scopes and sample counts are in METRICS.md; reproduction commands are in EVALUATION.md.

Settings

• --effort 1–5: average that many distinct candidate orderings, capped at two for binary questions. Reported results cover efforts 1 and 2.
• --temperature: candidate-softmax temperature, default 1.0.
• --prompt-format: cmdb for CMDB; open-format for JevBench and Decision Bench.

Base: Google Gemma 4 12B, Apache 2.0.