correct the qev media c=1 cells after re-measurement and simplify the warm-up wording

1c85c47 verified 5 days ago

10.4 kB

license: apache-2.0
base_model:
  - OmniJev/OneJev1.6-0.8B
  - OmniJev/OneJev1.6-4B
  - OmniJev/OneJev1.6-9B
  - OmniJev/OneJev1.6-27B
pipeline_tag: text-classification
tags:
  - verdict
  - system-one
  - decision-models
  - onejev
  - serving
  - torch-runtime

OneJev 1.6 (Verdict-ready)

The OneJev 1.6 family of multimodal System One decision models — text, image, and video inputs — packaged for Verdict, a high-performance serving engine for System One decision models.

This repository is a variant family: four variants — 0.8b, 4b, 9b, 27b — each a complete, self-contained package. Pick one with the :variant syntax; the bare name lists the variants and exits instead of guessing. One command to serve — the 9b variant, for example:

# GPU host
docker run --rm --gpus all -p 8080:8080 \
    -v ~/.cache/verdict:/root/.cache/verdict \
    sgyaqing/verdict:latest-cuda serve verdict-models/OneJev1.6:9b --host 0.0.0.0

# ModelScope:
docker run --rm --gpus all -p 8080:8080 \
    -e VERDICT_USE_MODELSCOPE=1 \
    -v ~/.cache/verdict:/root/.cache/verdict \
    sgyaqing/verdict:latest-cuda serve sgyaqing/OneJev1.6:9b --host 0.0.0.0

Endpoints: the standard decision protocols POST /v1/systemone and POST /v1/decisions, native POST /v1/choice | /v1/score | /v1/noul, plus an OpenAI-compatible layer (/v1/chat/completions, /v1/models).

Files

File	Purpose
`variants.json`	the variant index
`<variant>/model.json`	the manifest — budgets, and the declaration of which official repo holds the checkpoint
`SHA256SUMS`	checksums for the files above

No weights ship here. On first serve Verdict downloads the checkpoint (weights, configs, tokenizer) from the official OmniJev repo named in the variant's manifest, into the local cache.

Provenance & fidelity

Each variant serves its official checkpoint in bf16 (its own precision), verified against the golden standard (the official OneJev SDK on CPU fp32, 31-case corpus per tier) on CUDA (RTX 4090, A100).

Accuracy PK — against the model author's own server (qev 0.1.4); both servers on one machine per tier, graded against the same golden standard. Worst-case deviation across the corpus:

Variant	Verdict	`qev`	result
`0.8b`	2.49e-2	8.55e-1	Verdict ahead
`4b`	4.95e-2	9.03e-1	Verdict ahead
`9b`	3.91e-2	8.97e-1	Verdict ahead
`27b`	7.34e-2	7.80e-1	Verdict ahead

Performance

Measured on one RTX 4090 (48 GB) for 0.8b/4b and one A100-SXM4 (80 GB) for 9b/27b, end to end over HTTP, against the model author's own server (qev 0.1.4) — each cell a 10-second duration window taken after warm-up on both servers. On 9b/27b, rps counts the 200 responses completed inside the window ÷ 10 s, and a cell where a server completed fewer than 2× the concurrency is reported as n/a, not as a measured rate. A ratio above 1x means Verdict is ahead.

0.8b

workload	concurrency	Verdict rps	`qev` rps	rps ratio	Verdict P50 / P99 (ms)	`qev` P50 / P99 (ms)	P50 / P99 ratio
text	1	116.7	87.5	1.33x	8.90 / 9.30	11.37 / 11.93	1.28x / 1.28x
	8	245.7	89.9	2.73x	33.16 / 35.56	89.42 / 91.05	2.70x / 2.56x
	64	289.5	89.6	3.23x	225.43 / 228.53	693.37 / 928.39	3.08x / 4.06x
image	1	37.5	38.0	0.99x	26.86 / 28.70	26.31 / 28.73	0.98x / 1.00x
	8	135.6	38.4	3.53x	59.09 / 60.98	205.37 / 422.47	3.48x / 6.93x
	64	141.2	44.8	3.15x	472.29 / 479.76	1639.10 / 1697.90	3.47x / 3.54x
video	1	30.1	26.8	1.12x	32.24 / 40.02	31.30 / 35.30	0.97x / 0.88x
	8	109.2	31.5	3.47x	72.57 / 83.08	251.40 / 464.83	3.46x / 5.59x
	64	115.2	32.0	3.60x	583.80 / 608.71	2018.50 / 2143.50	3.46x / 3.52x

4b

workload	concurrency	Verdict rps	`qev` rps	rps ratio	Verdict P50 / P99 (ms)	`qev` P50 / P99 (ms)	P50 / P99 ratio
text	1	38.1	31.3	1.22x	25.98 / 27.01	31.99 / 32.38	1.23x / 1.20x
	8	47.3	32.3	1.46x	163.49 / 185.22	253.52 / 254.76	1.55x / 1.38x
	64	55.2	32.0	1.72x	1275.20 / 1425.70	2017.70 / 2243.00	1.58x / 1.57x
image	1	17.2	16.8	1.02x	57.51 / 60.47	57.68 / 60.92	1.00x / 1.01x
	8	51.6	17.6	2.93x	156.16 / 160.33	461.55 / 706.21	2.96x / 4.41x
	64	56.8	19.2	2.96x	1254.90 / 1267.90	4074.00 / 4076.10	3.25x / 3.21x
video	1	15.2	13.2	1.15x	65.60 / 73.36	64.40 / 71.10	0.98x / 0.97x
	8	45.3	16.0	2.83x	176.36 / 239.65	512.21 / 572.87	2.90x / 2.39x
	64	51.2	19.2	2.67x	1409.70 / 1514.20	4451.10 / 4452.30	3.16x / 2.94x

9b (A100 80 GB)

workload	concurrency	Verdict rps	`qev` rps	rps ratio	Verdict P50 / P99 (ms)	`qev` P50 / P99 (ms)	P50 / P99 ratio
text	1	25.3	24.4	1.04x	40 / 41	41 / 41	1.03x / 1.02x
	8	36.0	24.0	1.50x	219 / 221	324 / 342	1.48x / 1.55x
	64	35.8	19.2	1.86x	1783 / 2094	2592 / 2672	1.45x / 1.28x
image	1	21.2	13.3	1.59x	43 / 45	74 / 75	1.71x / 1.67x
	8	22.2	13.6	1.63x	375 / 422	587 / 592	1.57x / 1.40x
	64	20.9	12.8	1.63x	2868 / 2939	5258 / 5260	1.83x / 1.79x
video	1	19.2	11.8	1.63x	52 / 56	84 / 88	1.62x / 1.56x
	8	20.5	12.1	1.69x	382 / 462	645 / 653	1.69x / 1.41x
	64	19.3	n/a †	—	3100 / 3348	—	—

27b (A100 80 GB)

Measured with CUDA graphs disabled (VERDICT_TORCH_GRAPHS=0): the bf16 weights alone take most of the 80 GB card, leaving no room for the graph pools.

workload	concurrency	Verdict rps	`qev` rps	rps ratio	Verdict P50 / P99 (ms)	`qev` P50 / P99 (ms)	P50 / P99 ratio
text	1	9.1	8.5	1.07x	109 / 111	117 / 118	1.07x / 1.06x
	8	12.0	8.0	1.50x	656 / 658	934 / 966	1.42x / 1.47x
	64	11.2	n/a †	—	4466 / 5737	—	—
image	1	4.0	5.6	0.71x	245 / 249	175 / 179	0.72x / 0.72x
	8	17.6	5.6	3.14x	448 / 454	1400 / 1402	3.13x / 3.09x
	64	16.7	n/a †	—	3529 / 3934	—	—
video	1	3.9	5.2	0.75x	253 / 274	184 / 318	0.73x / 1.16x
	8	16.8	4.8	3.50x	455 / 569	1476 / 1579	3.24x / 2.77x
	64	17.1	n/a †	—	3621 / 4005	—	—

At the n/a cells `qev` did not sustain the load: its requests were still in flight when the window closed.

License

Apache-2.0, as declared by OmniJev. Credit for the models themselves belongs to their authors; this repository only packages the serving manifests.