Hugging Face's logo

SpragAI
/
SPEV

Any-to-Any
Transformers
Safetensors
multilingual
spev
feature-extraction
multimodal
text
audio
image
video
decision-model
choice
qwen3-omni
lora
custom-code
custom_code
Model card Files
xet
 Community
SPEV
File size: 4,301 Bytes
7381480
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
235e7a1
 
 
 
 
7381480
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
cfcadc3
7b243bf
 
 
 
 
 
 
 
 
 
7381480
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
---
license: apache-2.0
base_model: Qwen/Qwen3-Omni-30B-A3B-Instruct
base_model_relation: finetune
library_name: transformers
pipeline_tag: any-to-any
language:
- multilingual
tags:
- multimodal
- text
- audio
- image
- video
- decision-model
- choice
- qwen3-omni
- lora
- custom-code
- safetensors
---

# SPEV

SPEV is a 30B parameter omnimodal decision model that evaluates questions over text,
audio, images and video. It scores the supplied choices in one forward pass and
returns an answer with a probability for each choice.

## Model

- **Backbone:** Qwen3-Omni-30B-A3B Thinker, with a trained LoRA adapter.
- **Decision head:** a shared scorer over contextual representations of the supplied choices.
- **Output:** per-choice probabilities and the highest-probability answer for each question.

## Quickstart

The easiest way to get started is with the hosted API on Sprag (free through October 2026). 
The full API docs are available [here](https://sprag.ai/docs/decisions).

## Usage

Install the dependencies listed in `requirements.txt`, then load the model and processor:

```python
import torch
from transformers import AutoModel, AutoProcessor

model_id = "SpragAI/SPEV"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModel.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map="auto",
    attn_implementation="sdpa",
)

questions = {
    "decision": {
        "instructions": (
            "At the end of this conversation audio, has the most recent "
            "speaker yielded the floor or are they continuing their turn?"
        ),
        "criteria": {
            "keep_listening": (
                "The most recent speaker is continuing their turn; keep listening."
            ),
            "respond": (
                "The most recent speaker has yielded the floor; "
                "another speaker can respond."
            ),
        },
    }
}
inputs = processor(audio="conversation.wav", state={}, questions=questions)
inputs = inputs.to(model.device, dtype=torch.bfloat16)
with torch.inference_mode():
    outputs = model(**inputs)
result = processor.post_process(outputs, questions=questions)

print(result["answers"]["decision"]["choice"])
print(result["answers"]["decision"]["probabilities"])
```

### Inputs

| Argument | Description |
|---|---|
| `audio` | Audio file to evaluate |
| `image` | Image file |
| `video` | Video file; visual frames |
| `state` | JSON context for the decision |
| `questions` | Mapping of question IDs to instructions and named criteria |

Media arguments are optional and can be combined. For text-only questions, supply
text through `state`. The processor resamples audio to the expected 16 kHz; callers
do not supply a sample-rate argument. Supply a video's soundtrack separately as
`audio` when it is part of the evidence.

## Files

| File | Purpose |
|---|---|
| `model-*.safetensors`, `model.safetensors.index.json` | Backbone, adapter and decision-head weights |
| `config.json`, `configuration_spev.py` | Model architecture and scoring configuration |
| `modeling_spev.py` | Transformers model implementation |
| `processing_spev.py`, `decision_utils.py` | Media preparation, question rendering and answer formatting |
| Tokenizer, processor and chat-template files | Text, audio, image and video processing |
| `examples/turn_policy.json` | Evaluated turn-taking question and application threshold |
| `requirements.txt` | Inference dependencies |
| `LICENSE` | Apache-2.0 license |

## Results

| Input | SPEV p50 / p95 | OpenAI p50 / p95 |
|---|---:|---:|
| 5-second audio | 166 / 191 ms | 752 / 1331 ms |
| 20-second audio | 216 / 270 ms | 997 / 1326 ms |
| 512 × 512 image | 183 / 283 ms | 194 / 334 ms |
| 1024 × 1024 image | 322 / 642 ms | 373 / 763 ms |

Warm public-HTTPS latency including RTT, measured from the same client with concurrency one. SPEV ran on an RTX PRO6000 with native vLLM pooling. Each row has 30 measured pairs over six fixtures after three warmup pairs. OpenAI used `gpt-6-luna`; audio includes a fresh `gpt-4o-mini-transcribe` request before Decisions. These measurements cover latency, not accuracy.

## License

Apache-2.0. See [LICENSE](LICENSE) and [third-party notices](THIRD_PARTY_NOTICES.md).