sgyaqing commited on
Commit
e41b7fc
·
verified ·
1 Parent(s): 5d44d6a

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +20 -30
README.md CHANGED
@@ -15,21 +15,18 @@ tags:
15
 
16
  The [decider-2b](https://huggingface.co/Mapika/decider-2b) System One decision model (Qwen3.5-2B-Base, 1.9B parameters, 32768-token context, checkpoint revision v11 — the vendor's default), packaged for [Verdict](https://github.com/sgyaqing/verdict) — a high-performance serving engine for System One decision models. Serve it with one command:
17
 
18
- This package serves through the torch runtime, which runs on CUDA only in v0.2, so there is one
19
- image and it needs a GPU. The `-v` mounts cache the checkpoint, so only the first run downloads it.
20
-
21
  ```bash
22
- # GPU host (no CPU image: the torch runtime serves on CUDA in v0.2)
23
  docker run --rm --gpus all -p 8080:8080 \
24
  -v ~/.cache/verdict:/root/.cache/verdict \
25
  sgyaqing/verdict:0.2.0-cuda serve verdict-models/decider-2b --host 0.0.0.0
26
- ```
27
 
28
- If the hub is slow or unreachable, add `-e HF_ENDPOINT=https://hf-mirror.com` — the mirror covers
29
- both the package and the upstream checkpoint, since both are fetched over the Hugging Face path.
30
- Add `-e VERDICT_USE_MODELSCOPE=1` and use `sgyaqing/decider-2b` to fetch from ModelScope instead; the
31
- upstream checkpoint is looked up there under the id the manifest carries (`Mapika/decider-2b`),
32
- which is the same id it has on Hugging Face.
 
33
 
34
  Endpoints: the standard decision protocols `POST /v1/systemone` and `POST /v1/decisions`, native `POST /v1/choice | /v1/score | /v1/noul`, plus an OpenAI-compatible layer (`/v1/chat/completions`, `/v1/models`).
35
 
@@ -54,32 +51,25 @@ The weights are **not redistributed in this repository**. The manifest names the
54
 
55
  Requirements: a CUDA GPU (the torch runtime serves on CUDA in v0.2), `python3.12` with the pinned stack (the Verdict CUDA image ships it), ~5.5 GB VRAM at load (measured: 5654 MiB in the shipped image).
56
 
57
- ## Verified against the official model
58
 
59
- The official forward pass (`decider-ai` 1.9.0, CPU fp32) anchors a 30-case golden corpus
60
- (choice / score / noul, English, mixed lengths up to 2063 tokens). This package's serving path
61
- is graded against it end to end in the shipped configuration (torch runtime, bf16,
62
- torch.compile + CUDA graphs, RTX 4090): **argmax flips: 0**.
63
 
64
- ## Performance
65
 
66
- Measured on one machine (RTX 4090, AMD EPYC 7543, 16 threads), end to end over HTTP with
67
- [`oha`](https://github.com/hatoo/oha) and an identical request body for both servers. The
68
- baseline is the model author's own server (`decider.serve`); the Verdict column is this
69
- package. Latencies are per-request P50 / P99 at the stated concurrency. The ratios are oriented
70
- so that **above 1× means Verdict is ahead** in both: throughput ratio = Verdict ÷ baseline,
71
- latency ratio = baseline ÷ Verdict.
72
 
73
- Both the accuracy and the numbers below are from the vendor's **v11** weights — what
74
- `Mapika/decider-2b` serves today. The manifest names the repository rather than a revision, so a
75
- future upstream release would be picked up on a fresh download rather than silently graded against
76
- these figures.
77
 
78
- | concurrency | Verdict rps | Verdict P50 / P99 | `decider.serve` rps | `decider.serve` P50 / P99 | ratio rps | ratio P50 / P99 |
79
  |---|---:|---:|---:|---:|---:|---:|
80
- | 1 | 139.6 | 6.91 / 7.88 ms | 132.4 | 7.51 / 7.94 ms | 1.05× | 1.09 / 1.01× |
81
- | 8 | 436.5 | 18.23 / 20.17 ms | 314.0 | 25.41 / 26.87 ms | 1.39× | 1.39 / 1.33× |
82
- | 64 | 795.9 | 80.13 / 85.43 ms | 506.2 | 112.91 / 436.79 ms | 1.57× | 1.41 / 5.11× |
83
 
84
  ## License
85
 
 
15
 
16
  The [decider-2b](https://huggingface.co/Mapika/decider-2b) System One decision model (Qwen3.5-2B-Base, 1.9B parameters, 32768-token context, checkpoint revision v11 — the vendor's default), packaged for [Verdict](https://github.com/sgyaqing/verdict) — a high-performance serving engine for System One decision models. Serve it with one command:
17
 
 
 
 
18
  ```bash
19
+ # GPU host (the torch runtime serves on CUDA in v0.2 — no CPU image)
20
  docker run --rm --gpus all -p 8080:8080 \
21
  -v ~/.cache/verdict:/root/.cache/verdict \
22
  sgyaqing/verdict:0.2.0-cuda serve verdict-models/decider-2b --host 0.0.0.0
 
23
 
24
+ # ModelScope:
25
+ docker run --rm --gpus all -p 8080:8080 \
26
+ -e VERDICT_USE_MODELSCOPE=1 \
27
+ -v ~/.cache/verdict:/root/.cache/verdict \
28
+ sgyaqing/verdict:0.2.0-cuda serve sgyaqing/decider-2b --host 0.0.0.0
29
+ ```
30
 
31
  Endpoints: the standard decision protocols `POST /v1/systemone` and `POST /v1/decisions`, native `POST /v1/choice | /v1/score | /v1/noul`, plus an OpenAI-compatible layer (`/v1/chat/completions`, `/v1/models`).
32
 
 
51
 
52
  Requirements: a CUDA GPU (the torch runtime serves on CUDA in v0.2), `python3.12` with the pinned stack (the Verdict CUDA image ships it), ~5.5 GB VRAM at load (measured: 5654 MiB in the shipped image).
53
 
54
+ ## Provenance & fidelity
55
 
56
+ This package adds only the serving manifest; the checkpoint is fetched from `Mapika/decider-2b` (revision v11) on first serve. Serving is verified against the golden standard (official `decider-ai` 1.9.0, CPU fp32, 30-case corpus): zero answer flips on CUDA (RTX 4090).
 
 
 
57
 
58
+ Accuracy PK — both servers on one RTX 4090 (CUDA), graded against the same golden standard:
59
 
60
+ | | Verdict | `decider.serve` | result |
61
+ |---|---:|---:|:--|
62
+ | worst deviation | 1.24e-2 | 1.06e-2 | tied |
63
+
64
+ ## Performance
 
65
 
66
+ Measured on one RTX 4090, end to end over HTTP, against the model author's own server (`decider.serve`). Latencies in ms; a ratio above 1x means Verdict is ahead.
 
 
 
67
 
68
+ | concurrency | Verdict rps | `decider.serve` rps | ratio | Verdict P50 / P99 (ms) | `decider.serve` P50 / P99 (ms) | ratio |
69
  |---|---:|---:|---:|---:|---:|---:|
70
+ | 1 | 139.6 | 132.4 | 1.05x | 6.91 / 7.88 | 7.51 / 7.94 | 1.09 / 1.01x |
71
+ | 8 | 436.5 | 314.0 | 1.39x | 18.23 / 20.17 | 25.41 / 26.87 | 1.39 / 1.33x |
72
+ | 64 | 795.9 | 506.2 | 1.57x | 80.13 / 85.43 | 112.91 / 436.79 | 1.41 / 5.11x |
73
 
74
  ## License
75