---
license: apache-2.0
base_model: Qwen/Qwen3-Omni-30B-A3B-Instruct
base_model_relation: finetune
library_name: transformers
pipeline_tag: any-to-any
language:
- multilingual
tags:
- multimodal
- text
- audio
- image
- video
- decision-model
- choice
- qwen3-omni
- lora
- custom-code
- safetensors
---

# SPEV

SPEV is a 30B parameter omnimodal decision model that evaluates questions over text,
audio, images and video. It scores the supplied choices in one forward pass and
returns an answer with a probability for each choice.

## Model

- **Backbone:** Qwen3-Omni-30B-A3B Thinker, with a trained LoRA adapter.
- **Decision head:** a shared scorer over contextual representations of the supplied choices.
- **Output:** per-choice probabilities and the highest-probability answer for each question.

## Quickstart

The easiest way to get started is with the hosted API on Sprag (free through October 2026). 
The full API docs are available [here](https://sprag.ai/docs/decisions).

## Usage

Install the dependencies listed in `requirements.txt`, then load the model and processor:

```python
import torch
from transformers import AutoModel, AutoProcessor

model_id = "SpragAI/SPEV"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModel.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map="auto",
    attn_implementation="sdpa",
)

questions = {
    "decision": {
        "instructions": (
            "At the end of this conversation audio, has the most recent "
            "speaker yielded the floor or are they continuing their turn?"
        ),
        "criteria": {
            "keep_listening": (
                "The most recent speaker is continuing their turn; keep listening."
            ),
            "respond": (
                "The most recent speaker has yielded the floor; "
                "another speaker can respond."
            ),
        },
    }
}
inputs = processor(audio="conversation.wav", state={}, questions=questions)
inputs = inputs.to(model.device, dtype=torch.bfloat16)
with torch.inference_mode():
    outputs = model(**inputs)
result = processor.post_process(outputs, questions=questions)

print(result["answers"]["decision"]["choice"])
print(result["answers"]["decision"]["probabilities"])
```

### Inputs

| Argument | Description |
|---|---|
| `audio` | Audio file to evaluate |
| `image` | Image file |
| `video` | Video file; visual frames |
| `state` | JSON context for the decision |
| `questions` | Mapping of question IDs to instructions and named criteria |

Media arguments are optional and can be combined. For text-only questions, supply
text through `state`. The processor resamples audio to the expected 16 kHz; callers
do not supply a sample-rate argument. Supply a video's soundtrack separately as
`audio` when it is part of the evidence.

## Files

| File | Purpose |
|---|---|
| `model-*.safetensors`, `model.safetensors.index.json` | Backbone, adapter and decision-head weights |
| `config.json`, `configuration_spev.py` | Model architecture and scoring configuration |
| `modeling_spev.py` | Transformers model implementation |
| `processing_spev.py`, `decision_utils.py` | Media preparation, question rendering and answer formatting |
| Tokenizer, processor and chat-template files | Text, audio, image and video processing |
| `examples/turn_policy.json` | Evaluated turn-taking question and application threshold |
| `requirements.txt` | Inference dependencies |
| `LICENSE` | Apache-2.0 license |

## Results

| Input | SPEV p50 / p95 | OpenAI p50 / p95 |
|---|---:|---:|
| 5-second audio | 166 / 191 ms | 752 / 1331 ms |
| 20-second audio | 216 / 270 ms | 997 / 1326 ms |
| 512 × 512 image | 183 / 283 ms | 194 / 334 ms |
| 1024 × 1024 image | 322 / 642 ms | 373 / 763 ms |

Warm public-HTTPS latency including RTT, measured from the same client with concurrency one. SPEV ran on an RTX PRO6000 with native vLLM pooling. Each row has 30 measured pairs over six fixtures after three warmup pairs. OpenAI used `gpt-6-luna`; audio includes a fresh `gpt-4o-mini-transcribe` request before Decisions. These measurements cover latency, not accuracy.

## License

Apache-2.0. See [LICENSE](LICENSE) and [third-party notices](THIRD_PARTY_NOTICES.md).