correct the qev media c=1 cells after re-measurement and simplify the warm-up wording

1c85c47 verified 7 days ago

10.4 kB

---
license: apache-2.0
base_model:
- OmniJev/OneJev1.6-0.8B
- OmniJev/OneJev1.6-4B
- OmniJev/OneJev1.6-9B
- OmniJev/OneJev1.6-27B
pipeline_tag: text-classification
tags:
- verdict
- system-one
- decision-models
- onejev
- serving
- torch-runtime
---
# OneJev 1.6 (Verdict-ready)

The [OneJev 1.6](https://huggingface.co/OmniJev/OneJev1.6-9B) family of multimodal System One decision models — text, image, and video inputs — packaged for [Verdict](https://github.com/sgyaqing/verdict), a high-performance serving engine for System One decision models.

This repository is a **variant family**: four variants — `0.8b`, `4b`, `9b`, `27b` — each a complete, self-contained package. Pick one with the `:variant` syntax; the bare name lists the variants and exits instead of guessing. One command to serve — the `9b` variant, for example:

```bash
# GPU host
docker run --rm --gpus all -p 8080:8080 \
    -v ~/.cache/verdict:/root/.cache/verdict \
    sgyaqing/verdict:latest-cuda serve verdict-models/OneJev1.6:9b --host 0.0.0.0
# ModelScope:
docker run --rm --gpus all -p 8080:8080 \
    -e VERDICT_USE_MODELSCOPE=1 \
    -v ~/.cache/verdict:/root/.cache/verdict \
    sgyaqing/verdict:latest-cuda serve sgyaqing/OneJev1.6:9b --host 0.0.0.0
```

Endpoints: the standard decision protocols `POST /v1/systemone` and `POST /v1/decisions`, native `POST /v1/choice | /v1/score | /v1/noul`, plus an OpenAI-compatible layer (`/v1/chat/completions`, `/v1/models`).

## Files

| File | Purpose |
|---|---|
| `variants.json` | the variant index |
| `<variant>/model.json` | the manifest — budgets, and the declaration of which official repo holds the checkpoint |
| `SHA256SUMS` | checksums for the files above |

No weights ship here. On first serve Verdict downloads the checkpoint (weights, configs, tokenizer) from the official OmniJev repo named in the variant's manifest, into the local cache.

## Provenance & fidelity

Each variant serves its official checkpoint in bf16 (its own precision), verified against the golden standard (the official OneJev SDK on CPU fp32, 31-case corpus per tier) on CUDA (RTX 4090, A100).

Accuracy PK — against the model author's own server (`qev` 0.1.4); both servers on one machine per tier, graded against the same golden standard. Worst-case deviation across the corpus:

| Variant | Verdict | `qev` | result |
|---|---:|---:|:--|
| `0.8b` | 2.49e-2 | 8.55e-1 | Verdict ahead |
| `4b` | 4.95e-2 | 9.03e-1 | Verdict ahead |
| `9b` | 3.91e-2 | 8.97e-1 | Verdict ahead |
| `27b` | 7.34e-2 | 7.80e-1 | Verdict ahead |

## Performance

Measured on one RTX 4090 (48 GB) for `0.8b`/`4b` and one A100-SXM4 (80 GB) for `9b`/`27b`, end to end over HTTP, against the model author's own server (`qev` 0.1.4) — each cell a 10-second duration window taken after warm-up on both servers. On `9b`/`27b`, rps counts the 200 responses completed inside the window ÷ 10 s, and a cell where a server completed fewer than 2× the concurrency is reported as n/a, not as a measured rate. A ratio above 1x means Verdict is ahead.

### 0.8b

<table>
<thead>
<tr><th>workload</th><th>concurrency</th><th>Verdict rps</th><th><code>qev</code> rps</th><th>rps ratio</th><th>Verdict P50 / P99 (ms)</th><th><code>qev</code> P50 / P99 (ms)</th><th>P50 / P99 ratio</th></tr>
</thead>
<tbody>
<tr><td rowspan="3">text</td><td>1</td><td>116.7</td><td>87.5</td><td>1.33x</td><td>8.90&nbsp;/&nbsp;9.30</td><td>11.37&nbsp;/&nbsp;11.93</td><td>1.28x&nbsp;/&nbsp;1.28x</td></tr>
<tr><td>8</td><td>245.7</td><td>89.9</td><td>2.73x</td><td>33.16&nbsp;/&nbsp;35.56</td><td>89.42&nbsp;/&nbsp;91.05</td><td>2.70x&nbsp;/&nbsp;2.56x</td></tr>
<tr><td>64</td><td>289.5</td><td>89.6</td><td>3.23x</td><td>225.43&nbsp;/&nbsp;228.53</td><td>693.37&nbsp;/&nbsp;928.39</td><td>3.08x&nbsp;/&nbsp;4.06x</td></tr>
<tr><td rowspan="3">image</td><td>1</td><td>37.5</td><td>38.0</td><td>0.99x</td><td>26.86&nbsp;/&nbsp;28.70</td><td>26.31&nbsp;/&nbsp;28.73</td><td>0.98x&nbsp;/&nbsp;1.00x</td></tr>
<tr><td>8</td><td>135.6</td><td>38.4</td><td>3.53x</td><td>59.09&nbsp;/&nbsp;60.98</td><td>205.37&nbsp;/&nbsp;422.47</td><td>3.48x&nbsp;/&nbsp;6.93x</td></tr>
<tr><td>64</td><td>141.2</td><td>44.8</td><td>3.15x</td><td>472.29&nbsp;/&nbsp;479.76</td><td>1639.10&nbsp;/&nbsp;1697.90</td><td>3.47x&nbsp;/&nbsp;3.54x</td></tr>
<tr><td rowspan="3">video</td><td>1</td><td>30.1</td><td>26.8</td><td>1.12x</td><td>32.24&nbsp;/&nbsp;40.02</td><td>31.30&nbsp;/&nbsp;35.30</td><td>0.97x&nbsp;/&nbsp;0.88x</td></tr>
<tr><td>8</td><td>109.2</td><td>31.5</td><td>3.47x</td><td>72.57&nbsp;/&nbsp;83.08</td><td>251.40&nbsp;/&nbsp;464.83</td><td>3.46x&nbsp;/&nbsp;5.59x</td></tr>
<tr><td>64</td><td>115.2</td><td>32.0</td><td>3.60x</td><td>583.80&nbsp;/&nbsp;608.71</td><td>2018.50&nbsp;/&nbsp;2143.50</td><td>3.46x&nbsp;/&nbsp;3.52x</td></tr>
</tbody>
</table>

### 4b

<table>
<thead>
<tr><th>workload</th><th>concurrency</th><th>Verdict rps</th><th><code>qev</code> rps</th><th>rps ratio</th><th>Verdict P50 / P99 (ms)</th><th><code>qev</code> P50 / P99 (ms)</th><th>P50 / P99 ratio</th></tr>
</thead>
<tbody>
<tr><td rowspan="3">text</td><td>1</td><td>38.1</td><td>31.3</td><td>1.22x</td><td>25.98&nbsp;/&nbsp;27.01</td><td>31.99&nbsp;/&nbsp;32.38</td><td>1.23x&nbsp;/&nbsp;1.20x</td></tr>
<tr><td>8</td><td>47.3</td><td>32.3</td><td>1.46x</td><td>163.49&nbsp;/&nbsp;185.22</td><td>253.52&nbsp;/&nbsp;254.76</td><td>1.55x&nbsp;/&nbsp;1.38x</td></tr>
<tr><td>64</td><td>55.2</td><td>32.0</td><td>1.72x</td><td>1275.20&nbsp;/&nbsp;1425.70</td><td>2017.70&nbsp;/&nbsp;2243.00</td><td>1.58x&nbsp;/&nbsp;1.57x</td></tr>
<tr><td rowspan="3">image</td><td>1</td><td>17.2</td><td>16.8</td><td>1.02x</td><td>57.51&nbsp;/&nbsp;60.47</td><td>57.68&nbsp;/&nbsp;60.92</td><td>1.00x&nbsp;/&nbsp;1.01x</td></tr>
<tr><td>8</td><td>51.6</td><td>17.6</td><td>2.93x</td><td>156.16&nbsp;/&nbsp;160.33</td><td>461.55&nbsp;/&nbsp;706.21</td><td>2.96x&nbsp;/&nbsp;4.41x</td></tr>
<tr><td>64</td><td>56.8</td><td>19.2</td><td>2.96x</td><td>1254.90&nbsp;/&nbsp;1267.90</td><td>4074.00&nbsp;/&nbsp;4076.10</td><td>3.25x&nbsp;/&nbsp;3.21x</td></tr>
<tr><td rowspan="3">video</td><td>1</td><td>15.2</td><td>13.2</td><td>1.15x</td><td>65.60&nbsp;/&nbsp;73.36</td><td>64.40&nbsp;/&nbsp;71.10</td><td>0.98x&nbsp;/&nbsp;0.97x</td></tr>
<tr><td>8</td><td>45.3</td><td>16.0</td><td>2.83x</td><td>176.36&nbsp;/&nbsp;239.65</td><td>512.21&nbsp;/&nbsp;572.87</td><td>2.90x&nbsp;/&nbsp;2.39x</td></tr>
<tr><td>64</td><td>51.2</td><td>19.2</td><td>2.67x</td><td>1409.70&nbsp;/&nbsp;1514.20</td><td>4451.10&nbsp;/&nbsp;4452.30</td><td>3.16x&nbsp;/&nbsp;2.94x</td></tr>
</tbody>
</table>

### 9b (A100 80 GB)

<table>
<thead>
<tr><th>workload</th><th>concurrency</th><th>Verdict rps</th><th><code>qev</code> rps</th><th>rps ratio</th><th>Verdict P50 / P99 (ms)</th><th><code>qev</code> P50 / P99 (ms)</th><th>P50 / P99 ratio</th></tr>
</thead>
<tbody>
<tr><td rowspan="3">text</td><td>1</td><td>25.3</td><td>24.4</td><td>1.04x</td><td>40&nbsp;/&nbsp;41</td><td>41&nbsp;/&nbsp;41</td><td>1.03x&nbsp;/&nbsp;1.02x</td></tr>
<tr><td>8</td><td>36.0</td><td>24.0</td><td>1.50x</td><td>219&nbsp;/&nbsp;221</td><td>324&nbsp;/&nbsp;342</td><td>1.48x&nbsp;/&nbsp;1.55x</td></tr>
<tr><td>64</td><td>35.8</td><td>19.2</td><td>1.86x</td><td>1783&nbsp;/&nbsp;2094</td><td>2592&nbsp;/&nbsp;2672</td><td>1.45x&nbsp;/&nbsp;1.28x</td></tr>
<tr><td rowspan="3">image</td><td>1</td><td>21.2</td><td>13.3</td><td>1.59x</td><td>43&nbsp;/&nbsp;45</td><td>74&nbsp;/&nbsp;75</td><td>1.71x&nbsp;/&nbsp;1.67x</td></tr>
<tr><td>8</td><td>22.2</td><td>13.6</td><td>1.63x</td><td>375&nbsp;/&nbsp;422</td><td>587&nbsp;/&nbsp;592</td><td>1.57x&nbsp;/&nbsp;1.40x</td></tr>
<tr><td>64</td><td>20.9</td><td>12.8</td><td>1.63x</td><td>2868&nbsp;/&nbsp;2939</td><td>5258&nbsp;/&nbsp;5260</td><td>1.83x&nbsp;/&nbsp;1.79x</td></tr>
<tr><td rowspan="3">video</td><td>1</td><td>19.2</td><td>11.8</td><td>1.63x</td><td>52&nbsp;/&nbsp;56</td><td>84&nbsp;/&nbsp;88</td><td>1.62x&nbsp;/&nbsp;1.56x</td></tr>
<tr><td>8</td><td>20.5</td><td>12.1</td><td>1.69x</td><td>382&nbsp;/&nbsp;462</td><td>645&nbsp;/&nbsp;653</td><td>1.69x&nbsp;/&nbsp;1.41x</td></tr>
<tr><td>64</td><td>19.3</td><td>n/a&nbsp;&dagger;</td><td>&mdash;</td><td>3100&nbsp;/&nbsp;3348</td><td>&mdash;</td><td>&mdash;</td></tr>
</tbody>
</table>

### 27b (A100 80 GB)

Measured with CUDA graphs disabled (`VERDICT_TORCH_GRAPHS=0`): the bf16 weights alone take most of the 80 GB card, leaving no room for the graph pools.

<table>
<thead>
<tr><th>workload</th><th>concurrency</th><th>Verdict rps</th><th><code>qev</code> rps</th><th>rps ratio</th><th>Verdict P50 / P99 (ms)</th><th><code>qev</code> P50 / P99 (ms)</th><th>P50 / P99 ratio</th></tr>
</thead>
<tbody>
<tr><td rowspan="3">text</td><td>1</td><td>9.1</td><td>8.5</td><td>1.07x</td><td>109&nbsp;/&nbsp;111</td><td>117&nbsp;/&nbsp;118</td><td>1.07x&nbsp;/&nbsp;1.06x</td></tr>
<tr><td>8</td><td>12.0</td><td>8.0</td><td>1.50x</td><td>656&nbsp;/&nbsp;658</td><td>934&nbsp;/&nbsp;966</td><td>1.42x&nbsp;/&nbsp;1.47x</td></tr>
<tr><td>64</td><td>11.2</td><td>n/a&nbsp;&dagger;</td><td>&mdash;</td><td>4466&nbsp;/&nbsp;5737</td><td>&mdash;</td><td>&mdash;</td></tr>
<tr><td rowspan="3">image</td><td>1</td><td>4.0</td><td>5.6</td><td>0.71x</td><td>245&nbsp;/&nbsp;249</td><td>175&nbsp;/&nbsp;179</td><td>0.72x&nbsp;/&nbsp;0.72x</td></tr>
<tr><td>8</td><td>17.6</td><td>5.6</td><td>3.14x</td><td>448&nbsp;/&nbsp;454</td><td>1400&nbsp;/&nbsp;1402</td><td>3.13x&nbsp;/&nbsp;3.09x</td></tr>
<tr><td>64</td><td>16.7</td><td>n/a&nbsp;&dagger;</td><td>&mdash;</td><td>3529&nbsp;/&nbsp;3934</td><td>&mdash;</td><td>&mdash;</td></tr>
<tr><td rowspan="3">video</td><td>1</td><td>3.9</td><td>5.2</td><td>0.75x</td><td>253&nbsp;/&nbsp;274</td><td>184&nbsp;/&nbsp;318</td><td>0.73x&nbsp;/&nbsp;1.16x</td></tr>
<tr><td>8</td><td>16.8</td><td>4.8</td><td>3.50x</td><td>455&nbsp;/&nbsp;569</td><td>1476&nbsp;/&nbsp;1579</td><td>3.24x&nbsp;/&nbsp;2.77x</td></tr>
<tr><td>64</td><td>17.1</td><td>n/a&nbsp;&dagger;</td><td>&mdash;</td><td>3621&nbsp;/&nbsp;4005</td><td>&mdash;</td><td>&mdash;</td></tr>
</tbody>
</table>
At the n/a cells `qev` did not sustain the load: its requests were still in flight when the window closed.

## License
Apache-2.0, as declared by [OmniJev](https://huggingface.co/OmniJev). Credit for the models themselves belongs to their authors; this repository only packages the serving manifests.

	---
	license: apache-2.0
	base_model:
	- OmniJev/OneJev1.6-0.8B
	- OmniJev/OneJev1.6-4B
	- OmniJev/OneJev1.6-9B
	- OmniJev/OneJev1.6-27B
	pipeline_tag: text-classification
	tags:
	- verdict
	- system-one
	- decision-models
	- onejev
	- serving
	- torch-runtime
	---

	# OneJev 1.6 (Verdict-ready)

	The [OneJev 1.6](https://huggingface.co/OmniJev/OneJev1.6-9B) family of multimodal System One decision models — text, image, and video inputs — packaged for [Verdict](https://github.com/sgyaqing/verdict), a high-performance serving engine for System One decision models.

	This repository is a variant family: four variants — `0.8b`, `4b`, `9b`, `27b` — each a complete, self-contained package. Pick one with the `:variant` syntax; the bare name lists the variants and exits instead of guessing. One command to serve — the `9b` variant, for example:

	```bash
	# GPU host
	docker run --rm --gpus all -p 8080:8080 \
	-v ~/.cache/verdict:/root/.cache/verdict \
	sgyaqing/verdict:latest-cuda serve verdict-models/OneJev1.6:9b --host 0.0.0.0

	# ModelScope:
	docker run --rm --gpus all -p 8080:8080 \
	-e VERDICT_USE_MODELSCOPE=1 \
	-v ~/.cache/verdict:/root/.cache/verdict \
	sgyaqing/verdict:latest-cuda serve sgyaqing/OneJev1.6:9b --host 0.0.0.0
	```

	Endpoints: the standard decision protocols `POST /v1/systemone` and `POST /v1/decisions`, native `POST /v1/choice \| /v1/score \| /v1/noul`, plus an OpenAI-compatible layer (`/v1/chat/completions`, `/v1/models`).

	## Files

	\| File \| Purpose \|
	\|---\|---\|
	\| `variants.json` \| the variant index \|
	\| `<variant>/model.json` \| the manifest — budgets, and the declaration of which official repo holds the checkpoint \|
	\| `SHA256SUMS` \| checksums for the files above \|

	No weights ship here. On first serve Verdict downloads the checkpoint (weights, configs, tokenizer) from the official OmniJev repo named in the variant's manifest, into the local cache.

	## Provenance & fidelity

	Each variant serves its official checkpoint in bf16 (its own precision), verified against the golden standard (the official OneJev SDK on CPU fp32, 31-case corpus per tier) on CUDA (RTX 4090, A100).

	Accuracy PK — against the model author's own server (`qev` 0.1.4); both servers on one machine per tier, graded against the same golden standard. Worst-case deviation across the corpus:

	\| Variant \| Verdict \| `qev` \| result \|
	\|---\|---:\|---:\|:--\|
	\| `0.8b` \| 2.49e-2 \| 8.55e-1 \| Verdict ahead \|
	\| `4b` \| 4.95e-2 \| 9.03e-1 \| Verdict ahead \|
	\| `9b` \| 3.91e-2 \| 8.97e-1 \| Verdict ahead \|
	\| `27b` \| 7.34e-2 \| 7.80e-1 \| Verdict ahead \|

	## Performance

	Measured on one RTX 4090 (48 GB) for `0.8b`/`4b` and one A100-SXM4 (80 GB) for `9b`/`27b`, end to end over HTTP, against the model author's own server (`qev` 0.1.4) — each cell a 10-second duration window taken after warm-up on both servers. On `9b`/`27b`, rps counts the 200 responses completed inside the window ÷ 10 s, and a cell where a server completed fewer than 2× the concurrency is reported as n/a, not as a measured rate. A ratio above 1x means Verdict is ahead.

	### 0.8b

	<table>
	<thead>
	<tr><th>workload</th><th>concurrency</th><th>Verdict rps</th><th><code>qev</code> rps</th><th>rps ratio</th><th>Verdict P50 / P99 (ms)</th><th><code>qev</code> P50 / P99 (ms)</th><th>P50 / P99 ratio</th></tr>
	</thead>
	<tbody>
	<tr><td rowspan="3">text</td><td>1</td><td>116.7</td><td>87.5</td><td>1.33x</td><td>8.90 / 9.30</td><td>11.37 / 11.93</td><td>1.28x / 1.28x</td></tr>
	<tr><td>8</td><td>245.7</td><td>89.9</td><td>2.73x</td><td>33.16 / 35.56</td><td>89.42 / 91.05</td><td>2.70x / 2.56x</td></tr>
	<tr><td>64</td><td>289.5</td><td>89.6</td><td>3.23x</td><td>225.43 / 228.53</td><td>693.37 / 928.39</td><td>3.08x / 4.06x</td></tr>
	<tr><td rowspan="3">image</td><td>1</td><td>37.5</td><td>38.0</td><td>0.99x</td><td>26.86 / 28.70</td><td>26.31 / 28.73</td><td>0.98x / 1.00x</td></tr>
	<tr><td>8</td><td>135.6</td><td>38.4</td><td>3.53x</td><td>59.09 / 60.98</td><td>205.37 / 422.47</td><td>3.48x / 6.93x</td></tr>
	<tr><td>64</td><td>141.2</td><td>44.8</td><td>3.15x</td><td>472.29 / 479.76</td><td>1639.10 / 1697.90</td><td>3.47x / 3.54x</td></tr>
	<tr><td rowspan="3">video</td><td>1</td><td>30.1</td><td>26.8</td><td>1.12x</td><td>32.24 / 40.02</td><td>31.30 / 35.30</td><td>0.97x / 0.88x</td></tr>
	<tr><td>8</td><td>109.2</td><td>31.5</td><td>3.47x</td><td>72.57 / 83.08</td><td>251.40 / 464.83</td><td>3.46x / 5.59x</td></tr>
	<tr><td>64</td><td>115.2</td><td>32.0</td><td>3.60x</td><td>583.80 / 608.71</td><td>2018.50 / 2143.50</td><td>3.46x / 3.52x</td></tr>
	</tbody>
	</table>

	### 4b

	<table>
	<thead>
	<tr><th>workload</th><th>concurrency</th><th>Verdict rps</th><th><code>qev</code> rps</th><th>rps ratio</th><th>Verdict P50 / P99 (ms)</th><th><code>qev</code> P50 / P99 (ms)</th><th>P50 / P99 ratio</th></tr>
	</thead>
	<tbody>
	<tr><td rowspan="3">text</td><td>1</td><td>38.1</td><td>31.3</td><td>1.22x</td><td>25.98 / 27.01</td><td>31.99 / 32.38</td><td>1.23x / 1.20x</td></tr>
	<tr><td>8</td><td>47.3</td><td>32.3</td><td>1.46x</td><td>163.49 / 185.22</td><td>253.52 / 254.76</td><td>1.55x / 1.38x</td></tr>
	<tr><td>64</td><td>55.2</td><td>32.0</td><td>1.72x</td><td>1275.20 / 1425.70</td><td>2017.70 / 2243.00</td><td>1.58x / 1.57x</td></tr>
	<tr><td rowspan="3">image</td><td>1</td><td>17.2</td><td>16.8</td><td>1.02x</td><td>57.51 / 60.47</td><td>57.68 / 60.92</td><td>1.00x / 1.01x</td></tr>
	<tr><td>8</td><td>51.6</td><td>17.6</td><td>2.93x</td><td>156.16 / 160.33</td><td>461.55 / 706.21</td><td>2.96x / 4.41x</td></tr>
	<tr><td>64</td><td>56.8</td><td>19.2</td><td>2.96x</td><td>1254.90 / 1267.90</td><td>4074.00 / 4076.10</td><td>3.25x / 3.21x</td></tr>
	<tr><td rowspan="3">video</td><td>1</td><td>15.2</td><td>13.2</td><td>1.15x</td><td>65.60 / 73.36</td><td>64.40 / 71.10</td><td>0.98x / 0.97x</td></tr>
	<tr><td>8</td><td>45.3</td><td>16.0</td><td>2.83x</td><td>176.36 / 239.65</td><td>512.21 / 572.87</td><td>2.90x / 2.39x</td></tr>
	<tr><td>64</td><td>51.2</td><td>19.2</td><td>2.67x</td><td>1409.70 / 1514.20</td><td>4451.10 / 4452.30</td><td>3.16x / 2.94x</td></tr>
	</tbody>
	</table>

	### 9b (A100 80 GB)

	<table>
	<thead>
	<tr><th>workload</th><th>concurrency</th><th>Verdict rps</th><th><code>qev</code> rps</th><th>rps ratio</th><th>Verdict P50 / P99 (ms)</th><th><code>qev</code> P50 / P99 (ms)</th><th>P50 / P99 ratio</th></tr>
	</thead>
	<tbody>
	<tr><td rowspan="3">text</td><td>1</td><td>25.3</td><td>24.4</td><td>1.04x</td><td>40 / 41</td><td>41 / 41</td><td>1.03x / 1.02x</td></tr>
	<tr><td>8</td><td>36.0</td><td>24.0</td><td>1.50x</td><td>219 / 221</td><td>324 / 342</td><td>1.48x / 1.55x</td></tr>
	<tr><td>64</td><td>35.8</td><td>19.2</td><td>1.86x</td><td>1783 / 2094</td><td>2592 / 2672</td><td>1.45x / 1.28x</td></tr>
	<tr><td rowspan="3">image</td><td>1</td><td>21.2</td><td>13.3</td><td>1.59x</td><td>43 / 45</td><td>74 / 75</td><td>1.71x / 1.67x</td></tr>
	<tr><td>8</td><td>22.2</td><td>13.6</td><td>1.63x</td><td>375 / 422</td><td>587 / 592</td><td>1.57x / 1.40x</td></tr>
	<tr><td>64</td><td>20.9</td><td>12.8</td><td>1.63x</td><td>2868 / 2939</td><td>5258 / 5260</td><td>1.83x / 1.79x</td></tr>
	<tr><td rowspan="3">video</td><td>1</td><td>19.2</td><td>11.8</td><td>1.63x</td><td>52 / 56</td><td>84 / 88</td><td>1.62x / 1.56x</td></tr>
	<tr><td>8</td><td>20.5</td><td>12.1</td><td>1.69x</td><td>382 / 462</td><td>645 / 653</td><td>1.69x / 1.41x</td></tr>
	<tr><td>64</td><td>19.3</td><td>n/a &dagger;</td><td>—</td><td>3100 / 3348</td><td>—</td><td>—</td></tr>
	</tbody>
	</table>

	### 27b (A100 80 GB)

	Measured with CUDA graphs disabled (`VERDICT_TORCH_GRAPHS=0`): the bf16 weights alone take most of the 80 GB card, leaving no room for the graph pools.

	<table>
	<thead>
	<tr><th>workload</th><th>concurrency</th><th>Verdict rps</th><th><code>qev</code> rps</th><th>rps ratio</th><th>Verdict P50 / P99 (ms)</th><th><code>qev</code> P50 / P99 (ms)</th><th>P50 / P99 ratio</th></tr>
	</thead>
	<tbody>
	<tr><td rowspan="3">text</td><td>1</td><td>9.1</td><td>8.5</td><td>1.07x</td><td>109 / 111</td><td>117 / 118</td><td>1.07x / 1.06x</td></tr>
	<tr><td>8</td><td>12.0</td><td>8.0</td><td>1.50x</td><td>656 / 658</td><td>934 / 966</td><td>1.42x / 1.47x</td></tr>
	<tr><td>64</td><td>11.2</td><td>n/a &dagger;</td><td>—</td><td>4466 / 5737</td><td>—</td><td>—</td></tr>
	<tr><td rowspan="3">image</td><td>1</td><td>4.0</td><td>5.6</td><td>0.71x</td><td>245 / 249</td><td>175 / 179</td><td>0.72x / 0.72x</td></tr>
	<tr><td>8</td><td>17.6</td><td>5.6</td><td>3.14x</td><td>448 / 454</td><td>1400 / 1402</td><td>3.13x / 3.09x</td></tr>
	<tr><td>64</td><td>16.7</td><td>n/a &dagger;</td><td>—</td><td>3529 / 3934</td><td>—</td><td>—</td></tr>
	<tr><td rowspan="3">video</td><td>1</td><td>3.9</td><td>5.2</td><td>0.75x</td><td>253 / 274</td><td>184 / 318</td><td>0.73x / 1.16x</td></tr>
	<tr><td>8</td><td>16.8</td><td>4.8</td><td>3.50x</td><td>455 / 569</td><td>1476 / 1579</td><td>3.24x / 2.77x</td></tr>
	<tr><td>64</td><td>17.1</td><td>n/a &dagger;</td><td>—</td><td>3621 / 4005</td><td>—</td><td>—</td></tr>
	</tbody>
	</table>
	At the n/a cells `qev` did not sustain the load: its requests were still in flight when the window closed.

	## License
	Apache-2.0, as declared by [OmniJev](https://huggingface.co/OmniJev). Credit for the models themselves belongs to their authors; this repository only packages the serving manifests.