Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -15,21 +15,18 @@ tags:
|
|
| 15 |
|
| 16 |
The [decider-2b](https://huggingface.co/Mapika/decider-2b) System One decision model (Qwen3.5-2B-Base, 1.9B parameters, 32768-token context, checkpoint revision v11 — the vendor's default), packaged for [Verdict](https://github.com/sgyaqing/verdict) — a high-performance serving engine for System One decision models. Serve it with one command:
|
| 17 |
|
| 18 |
-
This package serves through the torch runtime, which runs on CUDA only in v0.2, so there is one
|
| 19 |
-
image and it needs a GPU. The `-v` mounts cache the checkpoint, so only the first run downloads it.
|
| 20 |
-
|
| 21 |
```bash
|
| 22 |
-
# GPU host (
|
| 23 |
docker run --rm --gpus all -p 8080:8080 \
|
| 24 |
-v ~/.cache/verdict:/root/.cache/verdict \
|
| 25 |
sgyaqing/verdict:0.2.0-cuda serve verdict-models/decider-2b --host 0.0.0.0
|
| 26 |
-
```
|
| 27 |
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
|
|
|
| 33 |
|
| 34 |
Endpoints: the standard decision protocols `POST /v1/systemone` and `POST /v1/decisions`, native `POST /v1/choice | /v1/score | /v1/noul`, plus an OpenAI-compatible layer (`/v1/chat/completions`, `/v1/models`).
|
| 35 |
|
|
@@ -54,32 +51,25 @@ The weights are **not redistributed in this repository**. The manifest names the
|
|
| 54 |
|
| 55 |
Requirements: a CUDA GPU (the torch runtime serves on CUDA in v0.2), `python3.12` with the pinned stack (the Verdict CUDA image ships it), ~5.5 GB VRAM at load (measured: 5654 MiB in the shipped image).
|
| 56 |
|
| 57 |
-
##
|
| 58 |
|
| 59 |
-
|
| 60 |
-
(choice / score / noul, English, mixed lengths up to 2063 tokens). This package's serving path
|
| 61 |
-
is graded against it end to end in the shipped configuration (torch runtime, bf16,
|
| 62 |
-
torch.compile + CUDA graphs, RTX 4090): **argmax flips: 0**.
|
| 63 |
|
| 64 |
-
|
| 65 |
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
latency ratio = baseline ÷ Verdict.
|
| 72 |
|
| 73 |
-
|
| 74 |
-
`Mapika/decider-2b` serves today. The manifest names the repository rather than a revision, so a
|
| 75 |
-
future upstream release would be picked up on a fresh download rather than silently graded against
|
| 76 |
-
these figures.
|
| 77 |
|
| 78 |
-
| concurrency | Verdict rps | Verdict P50 / P99
|
| 79 |
|---|---:|---:|---:|---:|---:|---:|
|
| 80 |
-
| 1 | 139.6 | 6.91 / 7.88
|
| 81 |
-
| 8 | 436.5 | 18.23 / 20.17
|
| 82 |
-
| 64 | 795.9 | 80.13 / 85.43
|
| 83 |
|
| 84 |
## License
|
| 85 |
|
|
|
|
| 15 |
|
| 16 |
The [decider-2b](https://huggingface.co/Mapika/decider-2b) System One decision model (Qwen3.5-2B-Base, 1.9B parameters, 32768-token context, checkpoint revision v11 — the vendor's default), packaged for [Verdict](https://github.com/sgyaqing/verdict) — a high-performance serving engine for System One decision models. Serve it with one command:
|
| 17 |
|
|
|
|
|
|
|
|
|
|
| 18 |
```bash
|
| 19 |
+
# GPU host (the torch runtime serves on CUDA in v0.2 — no CPU image)
|
| 20 |
docker run --rm --gpus all -p 8080:8080 \
|
| 21 |
-v ~/.cache/verdict:/root/.cache/verdict \
|
| 22 |
sgyaqing/verdict:0.2.0-cuda serve verdict-models/decider-2b --host 0.0.0.0
|
|
|
|
| 23 |
|
| 24 |
+
# ModelScope:
|
| 25 |
+
docker run --rm --gpus all -p 8080:8080 \
|
| 26 |
+
-e VERDICT_USE_MODELSCOPE=1 \
|
| 27 |
+
-v ~/.cache/verdict:/root/.cache/verdict \
|
| 28 |
+
sgyaqing/verdict:0.2.0-cuda serve sgyaqing/decider-2b --host 0.0.0.0
|
| 29 |
+
```
|
| 30 |
|
| 31 |
Endpoints: the standard decision protocols `POST /v1/systemone` and `POST /v1/decisions`, native `POST /v1/choice | /v1/score | /v1/noul`, plus an OpenAI-compatible layer (`/v1/chat/completions`, `/v1/models`).
|
| 32 |
|
|
|
|
| 51 |
|
| 52 |
Requirements: a CUDA GPU (the torch runtime serves on CUDA in v0.2), `python3.12` with the pinned stack (the Verdict CUDA image ships it), ~5.5 GB VRAM at load (measured: 5654 MiB in the shipped image).
|
| 53 |
|
| 54 |
+
## Provenance & fidelity
|
| 55 |
|
| 56 |
+
This package adds only the serving manifest; the checkpoint is fetched from `Mapika/decider-2b` (revision v11) on first serve. Serving is verified against the golden standard (official `decider-ai` 1.9.0, CPU fp32, 30-case corpus): zero answer flips on CUDA (RTX 4090).
|
|
|
|
|
|
|
|
|
|
| 57 |
|
| 58 |
+
Accuracy PK — both servers on one RTX 4090 (CUDA), graded against the same golden standard:
|
| 59 |
|
| 60 |
+
| | Verdict | `decider.serve` | result |
|
| 61 |
+
|---|---:|---:|:--|
|
| 62 |
+
| worst deviation | 1.24e-2 | 1.06e-2 | tied |
|
| 63 |
+
|
| 64 |
+
## Performance
|
|
|
|
| 65 |
|
| 66 |
+
Measured on one RTX 4090, end to end over HTTP, against the model author's own server (`decider.serve`). Latencies in ms; a ratio above 1x means Verdict is ahead.
|
|
|
|
|
|
|
|
|
|
| 67 |
|
| 68 |
+
| concurrency | Verdict rps | `decider.serve` rps | ratio | Verdict P50 / P99 (ms) | `decider.serve` P50 / P99 (ms) | ratio |
|
| 69 |
|---|---:|---:|---:|---:|---:|---:|
|
| 70 |
+
| 1 | 139.6 | 132.4 | 1.05x | 6.91 / 7.88 | 7.51 / 7.94 | 1.09 / 1.01x |
|
| 71 |
+
| 8 | 436.5 | 314.0 | 1.39x | 18.23 / 20.17 | 25.41 / 26.87 | 1.39 / 1.33x |
|
| 72 |
+
| 64 | 795.9 | 506.2 | 1.57x | 80.13 / 85.43 | 112.91 / 436.79 | 1.41 / 5.11x |
|
| 73 |
|
| 74 |
## License
|
| 75 |
|